This issue has been migrated from [#16685](https://github.com/matrix-org/synapse/pull/16685). --- Hopefully to make it faster?