[Feature request] Running parallel/async doctrine queries

[nacholibre]

Feature Request

Q	A
New Feature	yes
RFC	yes

Summary

First of all, thanks to all maintainers of Doctrine. You make the world better. 👏

It would be neat if doctrine can run multiple queries without waiting for each to finish after executing the next one. Currently I have a use case for a search page, which queries the database several times and the total time of all queries is relatively big, but each query by itself takes no more than 300ms. Getting around this is no trivial task. Only if I could run these queries in parallel. It seems that PHP 8.1 supports something like this with Fibers.

It there currently any limitation to implement parallelism in Doctrine? I'm sure that for most people this is must have feature and it would be very beneficial for the whole community.

[derrabus]

It there currently any limitation to implement parallelism in Doctrine?

Yes, the various database extensions of PHP. Most of them don't support multiple asynchronous queries over a single connection.

It seems that PHP 8.1 supports something like this with Fibers.

I haven't tested that yet, but it should be possible to open multiple database connections and use fibers to query them in parallel. But we don't need to change anything in DBAL to enable this, do we?

[odahcam]

thanks for all the contributions in this project, it saves lives!

@derrabus that would be the idea. Additionally packages like https://packagist.org/packages/small/doctrine-dbal-swoole-pdo-driver have implemented something like it, with additional features like a connection pool, to make using Doctrine with Swoole effective. Currently there's no other library AFAIK that would provide a driver with such capabilities being compatible with something else, like RoadRunner or ReactPHP.

Every now and then I see an issue like this in the repo and it always gets closed after some time, but unsurprisingly someone opens a new one not too long after, probably while doing their own research, since it is a very advanced topic and getting the knowledge to implement such thing currently requires extensive research due to the lack of contents to study. Event-loops and mainly PHP coroutines are here to stay, so now more than ever having some official support or guidance on the topic from the Doctrine team would be amazing.

[odahcam]

I'm still researching and this week I found https://github.com/driftphp/reactphp-dbal and https://github.com/amphp/mysql-dbal (this one is not currently working) and they are very interesting approaches to solve this async problematic without touching neither DBAL or Doctrine's APIs.

[derrabus]

Thank you for the heads up.

Event-loops and mainly PHP coroutines are here to stay, so now more than ever having some official support or guidance on the topic from the Doctrine team would be amazing.

It's not like we're refusing to support async database connections, but we need something more actionable than this ticket. And somebody would need to work on the topic.

[nacholibre]

@derrabus @odahcam I've prototyped proof of concept PR #7227

[gabrielberthier]

I would also mention one attempt of mine to bring something like async in doctrine. Currently a work in progress, but with some examples working well.

https://github.com/BrashPHP/dbal