Add serializer-derived field introspection to ValuesViewset#14327
Add serializer-derived field introspection to ValuesViewset#14327rtibbles wants to merge 3 commits intolearningequality:release-v0.19.xfrom
Conversation
Adds an integration benchmark for comparing ValuesViewset serialization approaches, measuring performance across synthetic and real-world data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enables ValuesViewset to automatically derive its values() tuple, field_map, and consolidation logic from the serializer_class, removing the need for manually maintained field configuration. Includes serializer introspection utility, comprehensive test suite, and updated documentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes explicit values tuples and field_map dicts from FacilityUserViewSet, PublicFacilityUserViewSet, and FacilityUserSignUpViewSet, relying on the serializer to drive field selection and transformation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
github-actions
bot
added
DEV: backend
Build Artifacts
|
rtibblesbot
left a comment
rtibblesbot
left a comment
There was a problem hiding this comment.
Well-structured first step toward serializer-derived field introspection for ValuesViewset. The core introspection module is solid, the FacilityUser viewset migrations are clean, and the 58-test suite plus benchmark script provide good coverage and regression tooling.
CI passing. No UI changes.
- suggestion:
deferred_fieldssemantics for non-nested-serializer fields (see inline) - suggestion: Thread safety comment precision (see inline)
- praise: Ordering preservation in
_auto_consolidate(see inline) - praise: Type-inference optimization in
_field_matches_inferred_type(see inline) - praise: Removal of
FacilityUserConsolidateMixinand its manual re-sort — the new auto-consolidate is more robust
@rtibblesbot's comments are generated by an LLM, and should be evaluated accordingly
How was this generated?
Reviewed the pull request diff checking for:
- Correctness: bugs, edge cases, undocumented behavior, resource leaks, hardcoded values
- Design: unnecessary complexity, naming, readability, comment accuracy, redundant state
- Architecture: duplicated concerns, minimal interfaces, composition over inheritance
- Testing: behavior-based assertions, mocks only at hard boundaries, accurate coverage
- Completeness: missing dependencies, unupdated usages, i18n, accessibility, security
- Principles: DRY (same reason to change), SRP, Rule of Three (no premature abstraction)
- Checked CI status and linked issue acceptance criteria
- For UI changes: inspected screenshots for layout, visual completeness, and consistency
| field_map = { | ||
| "is_superuser": lambda x: bool(x.pop("devicepermissions__is_superuser")), | ||
| } | ||
| deferred_fields = ("roles",) |
There was a problem hiding this comment.
suggestion: deferred_fields = ("roles",) here works correctly — the field is skipped during introspection and populated manually in consolidate(). However, roles on PublicFacilityUserSerializer is a ListField(child=CharField()), not a nested ModelSerializer. The deferred_fields docstring and documentation describe it as "nested serializer field names that should be fetched separately," so this usage is semantically imprecise.
This won't cause bugs (the introspection code handles it: ListField fails _is_nested_serializer(), hits the field_name in deferred_fields: continue branch, and gets skipped), but it could confuse future developers who expect deferred_fields entries to be usable with serialize_queryset().
Consider either:
- Adding a brief comment here explaining that
rolesis deferred because it's populated manually inconsolidate()(not viaserialize_queryset()), or - Documenting in
api_patterns.rstthatdeferred_fieldscan also be used for non-nested fields that are populated manually
| Deferred from __init_subclass__ to avoid instantiating serializers | ||
| (which may reference querysets) at class definition / import time. | ||
|
|
||
| No lock needed under CPython's GIL: the work is idempotent — |
There was a problem hiding this comment.
suggestion: The thread-safety reasoning is correct in outcome but could be more precise. Under CPython's GIL, two threads can both read _initialized = False and execute the initialization block concurrently (the check-then-act at line 238 is not atomic across bytecodes). The code is safe because the work is idempotent — both threads produce identical results, so the final class state is correct regardless. The waste is only redundant computation, not corruption.
Consider rewording to: "Safe without a lock under CPython's GIL: concurrent threads may redundantly execute this block, but the work is idempotent — identical serializer always produces identical values/field_map, so the final state is correct regardless of execution order."
| self.__class__._pk_getter = pk_getter | ||
|
|
||
| joined_many = self._joined_many | ||
| original_pk_order = list(dict.fromkeys(pk_getter(item) for item in items)) |
There was a problem hiding this comment.
praise: Capturing original_pk_order before sorting for groupby, then restoring it at line 616, is an elegant solution. This correctly preserves the queryset's ordering (applied by ValuesViewsetOrderingFilter or the default order_by) through the groupby consolidation — a significant improvement over the old FacilityUserConsolidateMixin which had to manually re-sort based on request query params.
| current_model = related_model | ||
|
|
||
|
|
||
| def _field_matches_inferred_type( |
There was a problem hiding this comment.
praise: Good design decision to use exact class matching (type(declared_field) is inferred_class) rather than isinstance. This correctly handles the case where a subclass overrides to_representation() — such fields will always go through the transform path, avoiding silent type mismatches. The FK-field handling (lines 92-100) is also well-reasoned: skipping to_representation for PrimaryKeyRelatedField avoids calling it with a raw value instead of a model instance.
| return attrs | ||
|
|
||
|
|
||
| class FacilityUserRoleSerializer(serializers.ModelSerializer): |
There was a problem hiding this comment.
praise: Clean separation of FacilityUserRoleSerializer (read-only, excludes user) from the existing RoleSerializer (includes user for write operations). This avoids leaking the parent user ID into nested role objects in API responses.
rtibblesbot
left a comment
rtibblesbot
left a comment
There was a problem hiding this comment.
Solid infrastructure for serializer-derived field introspection. The core introspection module, auto-consolidation, and FacilityUser viewset migrations are well-structured.
CI passing. No UI changes.
Delta from prior review:
- ACKNOWLEDGED:
deferred_fieldssemantics for non-nested-serializer fields — behavior is correct as-is (ListFieldindeferred_fieldsis simply skipped fromvalues(), populated byconsolidate()) - ACKNOWLEDGED: Thread safety comment precision — the comment at
_ensure_initializedaccurately describes CPython GIL safety and free-threaded Python considerations - 3 prior praise finding(s) stand; not re-posted.
New findings:
- blocking:
_build_validation_schemacrashes onListFieldchildren in DEBUG mode (see inline) - praise:
_get_ownclassmethod pattern (see inline)
@rtibblesbot's comments are generated by an LLM, and should be evaluated accordingly
How was this generated?
Compared the current PR state against findings from a prior review:
- Retrieved prior bot reviews deterministically via the GitHub API
- Classified each prior finding as RESOLVED, UNADDRESSED, ACKNOWLEDGED, or CONTESTED
- Only raised NEW findings for newly introduced code
- Reviewed the pull request diff checking for correctness, design, architecture, testing, completeness, and adherence to DRY/SRP principles
- Checked CI status and linked issue acceptance criteria
| if getattr(field, "write_only", False): | ||
| continue | ||
| expected_fields.add(field_name) | ||
| if hasattr(field, "child"): |
There was a problem hiding this comment.
blocking: _build_validation_schema crashes in DEBUG mode for serializers with ListField(child=...) fields.
hasattr(field, "child") matches both ListSerializer (intended — its .child is a Serializer with .fields) and ListField (unintended — its .child is a plain Field like CharField, which has no .fields attribute).
Concrete trigger: PublicFacilityUserSerializer.roles is ListField(child=CharField()). When PublicFacilityUserViewSet initializes in DEBUG mode, it calls _build_validation_schema, hits this branch, recurses into CharField, and CharField has no .fields → AttributeError.
This isn't caught by tests because Django test settings typically have DEBUG=False, and the test suite's @override_settings(DEBUG=True) tests use custom serializers that don't include ListField.
Fix: guard the recursion to only enter serializer children, e.g.:
if hasattr(field, "child") and hasattr(field.child, "fields"):or more explicitly:
from rest_framework.serializers import BaseSerializer
if hasattr(field, "child") and isinstance(field.child, BaseSerializer):The same pattern should also be checked in _validate_items_against_schema — if a ListField value (list of primitives like ["admin", "coach"]) reaches the nested validation path, it would try to call .keys() on strings.
| Prevents dynamically cached class attributes (e.g. serializer_class | ||
| set by get_serializer_class()) from leaking to child classes. | ||
| """ | ||
| return cls.__dict__.get(attr, default) |
There was a problem hiding this comment.
praise: _get_own via cls.__dict__.get() is a clean solution for preventing dynamically-cached class attributes from leaking through the MRO to child classes — a common and subtle pitfall with Django class-based views and DRF viewsets.
3 participants
Summary
Enables
ValuesViewsetto automatically derivevalues(),field_map, andmany=Trueconsolidation fromserializer_class— no more manually maintained field tuples that drift out of sync.FacilityUserViewSet,PublicFacilityUserViewSet, andFacilityUserSignUpViewSetas first validation (all 166 existing auth tests pass unchanged)References
Partial work towards #14036
Reviewer guidance
kolibri/core/utils/serializer_introspection.py— the core derivation logicBaseValuesViewset._ensure_initializedinkolibri/core/api.py— where derivation is triggered and cachedkolibri/core/auth/api.py) shows the pattern in practice_auto_consolidate(groupby dedup correctness),_field_matches_inferred_type(false negatives cause silent type mismatches)AI usage
Developed collaboratively with Claude Code (Opus 4.6). Used for initial implementation, test suite, and iterative refinement based on benchmark results and review feedback. All code reviewed and verified against the existing auth test suite.