Skip to content

TaggableManager.names() does not use the prefetch cache #936

@ColonelThirtyTwo

Description

@ColonelThirtyTwo

The TaggableManager.names() method (and likely the TaggleManager.slugs() method as well) will execute queries even if the tags field has been prefetched with prefetch_related. This leads to unexpected n+1 select issues when the tags of a list of items are used, even when prefetching is done to try to resolve the issue.

As a demonstration, add this test case (a small alteration to test_prefetch_related to the TaggableManagerTestCase):

    def test_prefetch_related_names(self):
        apple = self.food_model.objects.create(name="apple")
        apple.tags.add("1", "2")
        orange = self.food_model.objects.create(name="orange")
        orange.tags.add("2", "4")
        with self.assertNumQueries(2):
            list_prefetched = list(
                self.food_model.objects.prefetch_related("tags").all()
            )
        with self.assertNumQueries(0):
            foods = {f.name: set(f.tags.names()) for f in list_prefetched}
            self.assertEqual(foods, {"orange": {"2", "4"}, "apple": {"1", "2"}})

The test will fail several times. This is one instance:

======================================================================
FAIL: test_prefetch_related_names (tests.tests.TaggableManagerCustomPKTestCase.test_prefetch_related_names)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/col/Code/django-taggit/tests/tests.py", line 756, in test_prefetch_related_names
    with self.assertNumQueries(0):
  File "/home/col/Code/django-taggit/.tox/py311-dj41/lib/python3.11/site-packages/django/test/testcases.py", line 98, in __exit__
    self.test_case.assertEqual(
AssertionError: 2 != 0 : 2 queries executed, 0 expected
Captured queries were:
1. SELECT DISTINCT "taggit_tag"."name", "tests_taggedcustompk"."id" FROM "taggit_tag" INNER JOIN "tests_taggedcustompk" ON ("taggit_tag"."id" = "tests_taggedcustompk"."tag_id") INNER JOIN "django_content_type" ON ("tests_taggedcustompk"."content_type_id" = "django_content_type"."id") WHERE ("django_content_type"."app_label" = 'tests' AND "django_content_type"."model" = 'custompkfood' AND "tests_taggedcustompk"."object_id" = 'apple') ORDER BY "tests_taggedcustompk"."id" ASC
2. SELECT DISTINCT "taggit_tag"."name", "tests_taggedcustompk"."id" FROM "taggit_tag" INNER JOIN "tests_taggedcustompk" ON ("taggit_tag"."id" = "tests_taggedcustompk"."tag_id") INNER JOIN "django_content_type" ON ("tests_taggedcustompk"."content_type_id" = "django_content_type"."id") WHERE ("django_content_type"."app_label" = 'tests' AND "django_content_type"."model" = 'custompkfood' AND "tests_taggedcustompk"."object_id" = 'orange') ORDER BY "tests_taggedcustompk"."id" ASC

It's likely that the values_list call on the queryset in the names method causes Django to have to refetch the query rather than using the prefetch cache.

This is fairly straightforward to work around - just fetch tag.name manually instead of using the names method - but it's annoying and a footgun.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions