Improve optimizer for polymorphism and connections: post-fetch prefetching, InheritanceManager support, and fewer N+1s

[stygmate]

Summary
This PR substantially improves the Django optimizer to better support polymorphic models, nested reverse relations, and Relay connections. It introduces a post-fetch prefetching pass that batches reverse relations after queryset evaluation and injects them into Django’s _prefetched_objects_cache. It also adds dedicated handling for InheritanceManager (django-polymorphic) and enhances Relay connection resolution to avoid extra queries (including totalCount).

Key changes

• Add post-fetch prefetching
  • New module strawberry_django/postfetch.py and integration into default_qs_hook.
  • Defers certain reverse lookups to a postfetch phase, batching by FK and filling nested caches.
  • Supports both child-level and parent-level postfetch branches collected during optimization.
• Polymorphism and InheritanceManager
  • Rewrite prefetch paths after select_subclasses so subclass instances get correct lookups and avoid redundant queries.
  • Collect and merge postfetch hints across interface/union selections and concrete types.
• Relay/pagination
  • If a queryset is optimized-by-prefetching, use the specialized connection path, otherwise compute totalCount via a window Count(1) in the main query when requested, avoiding a separate count() query.
  • Materialize querysets that carry postfetch hints before connection construction to reuse caches in nested connections.
• Field and fetch hooks
  • Connection/list resolvers can reuse an existing _prefetched_objects_cache on the source instance.
  • Single-object fields avoid .get() when a prefetched result cache is present to prevent LIMIT queries discarding prefetched data.
• Tests and docs
  • Extensive new tests for polymorphism, Relay, and InheritanceManager, including postfetch branches and query behavior.
  • Documentation updates in docs/guide/optimizer.md.

Configuration

• OptimizerConfig retains existing flags (enable_only, enable_select_related, enable_prefetch_related, enable_annotate, enable_nested_relations_prefetch) and adds support pathways for postfetch_prefetch and parent-branch hints carried via OptimizerStore.
• No breaking changes to the public API; the postfetch mechanism is internal and triggered automatically by the optimizer.

Performance and behavior

• Reduces N+1s on nested reverse relations (particularly under polymorphic models and nested connections) by batching and caching after evaluation.
• Avoids redundant or path-mismatched prefetches when using InheritanceManager.select_subclasses by rewriting prefetch paths relative to the subclass.
• Prevents extra count() queries for totalCount by using SQL window functions when the field is selected.

Related issues

• Likely fixes or improves: Polymorphic interfaces break prefetch when a nested field has filters applied #593, N+1 when using a prefetch_related without explicit ordering #772, N+1 queries due to queryset.count() if cache is empty #788.
• Likely improves generic N+1 patterns around connections/fragments: N+1 if query with a fragment and custom connection class (introduced in 0.59.0) #771.
• Possibly affected (needs validation): Wrong connection total_count when use DISTINCT #792 (DISTINCT counts), Optimizer: annotations do not work in nested fields when select_related is used. #743/Query Optimizer Annotate not working when used in nested query #549 (nested annotations).

Notes

• The PR keeps the optimizer conservative on already-evaluated querysets and lists (no re-optimization, respects result_cache).
• The new logic is exercised by the added test suites, especially under tests/polymorphism*, tests/polymorphism_relay*, and tests/polymorphism_inheritancemanager*.

Review note
This one’s a heavyweight and touches the optimizer’s core. It needs a reviewer who knows the library well and can take the time for a slow, careful pass @bellini666 ☕🙂. Also, if anyone wants to experiment with this version in their projects, please do—real-world mileage reports are pure gold.

[sourcery-ai]

Sorry @stygmate, your pull request is larger than the review limit of 150000 diff characters

[codecov]

Codecov Report

❌ Patch coverage is 72.60504% with 163 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.60%. Comparing base (6456a7b) to head (cc7c87d).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
strawberry_django/postfetch.py	62.36%	105 Missing ⚠️
strawberry_django/optimizer.py	76.92%	51 Missing ⚠️
strawberry_django/pagination.py	80.00%	4 Missing ⚠️
strawberry_django/fields/field.py	94.64%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #808      +/-   ##
==========================================
- Coverage   89.71%   87.60%   -2.11%     
==========================================
  Files          45       46       +1     
  Lines        4298     4882     +584     
==========================================
+ Hits         3856     4277     +421     
- Misses        442      605     +163     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[stygmate]

I realized that under certain conditions, evaluating a queryset was loading all the records from the tables. I’ll look for fixes.

[stygmate]

I fixed the bug I detected earlier today. I also did quite a bit of refactoring to meet Pyright and Ruff requirements and to make my changes clearer.

[stygmate]

I have code branches that aren’t covered by tests. Given the complexity of handling optimizations related to polymorphism, I worked iteratively (hence the large number of commits), and some parts of the code may have become unnecessary. That said, I’m seeing very promising results.