core: Optimize list users and groups api

[ryanpesek]

Attempt at optimizing users and groups api for users with a lot of groups

Details

These changes reduced the response time by a factor of two or three when listing users and groups and reduced CPU load in pods and posgres. Most noticable improvement is when include_users=true or include_groups=true.

---

Checklist

• Local tests pass (ak test authentik/)
• The code has been formatted (make lint-fix)

If an API change has been made

• The API schema has been updated (make gen-build)

If changes to the frontend have been made

• The code has been formatted (make web)

If applicable

• The documentation has been updated
• The documentation has been formatted (make docs)

[netlify]

✅ Deploy Preview for authentik-docs canceled.

Name	Link
🔨 Latest commit	d9336df
🔍 Latest deploy log	https://app.netlify.com/projects/authentik-docs/deploys/6927381428ef8b0007aa3fef

[netlify]

✅ Deploy Preview for authentik-integrations canceled.

Name	Link
🔨 Latest commit	d9336df
🔍 Latest deploy log	https://app.netlify.com/projects/authentik-integrations/deploys/692738145d282d0007122a7f

[codecov]

Codecov Report

❌ Patch coverage is 62.50000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.99%. Comparing base (d211ea1) to head (d9336df).
⚠️ Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
authentik/core/api/groups.py	50.00%	2 Missing ⚠️
authentik/core/api/users.py	75.00%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (d211ea1) and HEAD (d9336df). Click for more details.

HEAD has 14 uploads less than BASE

Flag	BASE (d211ea1)	HEAD (d9336df)
e2e	8	6
unit	10	4
unit-migrate	10	4

Additional details and impacted files

@@             Coverage Diff             @@
##             main   #18342       +/-   ##
===========================================
- Coverage   92.98%   70.99%   -21.99%     
===========================================
  Files         894      894               
  Lines       48919    48922        +3     
===========================================
- Hits        45486    34732    -10754     
- Misses       3433    14190    +10757     

Flag	Coverage Δ
e2e	43.69% <37.50%> (-1.77%)	⬇️
integration	23.12% <12.50%> (-0.06%)	⬇️
unit	67.87% <62.50%> (-23.23%)	⬇️
unit-migrate	67.93% <62.50%> (-23.22%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[netlify]

✅ Deploy Preview for authentik-storybook canceled.

Name	Link
🔨 Latest commit	d9336df
🔍 Latest deploy log	https://app.netlify.com/projects/authentik-storybook/deploys/69273814c1c69600085bb831

[rissson]

This can cause wrong data to be shown to the user, as the user might be superuser through an indirect membership. cc. @BeryJu