-
-
Notifications
You must be signed in to change notification settings - Fork 622
Description
The TaggableManager.names() method (and likely the TaggleManager.slugs() method as well) will execute queries even if the tags field has been prefetched with prefetch_related. This leads to unexpected n+1 select issues when the tags of a list of items are used, even when prefetching is done to try to resolve the issue.
As a demonstration, add this test case (a small alteration to test_prefetch_related to the TaggableManagerTestCase):
def test_prefetch_related_names(self):
apple = self.food_model.objects.create(name="apple")
apple.tags.add("1", "2")
orange = self.food_model.objects.create(name="orange")
orange.tags.add("2", "4")
with self.assertNumQueries(2):
list_prefetched = list(
self.food_model.objects.prefetch_related("tags").all()
)
with self.assertNumQueries(0):
foods = {f.name: set(f.tags.names()) for f in list_prefetched}
self.assertEqual(foods, {"orange": {"2", "4"}, "apple": {"1", "2"}})The test will fail several times. This is one instance:
======================================================================
FAIL: test_prefetch_related_names (tests.tests.TaggableManagerCustomPKTestCase.test_prefetch_related_names)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/col/Code/django-taggit/tests/tests.py", line 756, in test_prefetch_related_names
with self.assertNumQueries(0):
File "/home/col/Code/django-taggit/.tox/py311-dj41/lib/python3.11/site-packages/django/test/testcases.py", line 98, in __exit__
self.test_case.assertEqual(
AssertionError: 2 != 0 : 2 queries executed, 0 expected
Captured queries were:
1. SELECT DISTINCT "taggit_tag"."name", "tests_taggedcustompk"."id" FROM "taggit_tag" INNER JOIN "tests_taggedcustompk" ON ("taggit_tag"."id" = "tests_taggedcustompk"."tag_id") INNER JOIN "django_content_type" ON ("tests_taggedcustompk"."content_type_id" = "django_content_type"."id") WHERE ("django_content_type"."app_label" = 'tests' AND "django_content_type"."model" = 'custompkfood' AND "tests_taggedcustompk"."object_id" = 'apple') ORDER BY "tests_taggedcustompk"."id" ASC
2. SELECT DISTINCT "taggit_tag"."name", "tests_taggedcustompk"."id" FROM "taggit_tag" INNER JOIN "tests_taggedcustompk" ON ("taggit_tag"."id" = "tests_taggedcustompk"."tag_id") INNER JOIN "django_content_type" ON ("tests_taggedcustompk"."content_type_id" = "django_content_type"."id") WHERE ("django_content_type"."app_label" = 'tests' AND "django_content_type"."model" = 'custompkfood' AND "tests_taggedcustompk"."object_id" = 'orange') ORDER BY "tests_taggedcustompk"."id" ASC
It's likely that the values_list call on the queryset in the names method causes Django to have to refetch the query rather than using the prefetch cache.
This is fairly straightforward to work around - just fetch tag.name manually instead of using the names method - but it's annoying and a footgun.