Update _read_parallel to use a queue

[Ch3ri0ur]

Purpose

Instead of reading through the docnames in batches uses a queue and workers.

This is an alternative approach to #13738.
The difference is, that I did not actually need to touch parallel.py for this variant.

p.s.: I tried a similar approach for write, but it seems to be less performant because I
needed to pickle the doctrees and send them via queues (instead of them being forked)

References

• <...>
• <...>
• <...>

[AA-Turner]

@Ch3ri0ur could you look at the test failures?

Please can you also add a CHANGES entry and add yourself to AUTHORS? It would be good to get a sense of the performance difference from this approach if possible.

A

[Ch3ri0ur]

While testing this locally I stumbled upon a issue, where the now random/no deterministic order (intermittently) surfaced problems.
I guess this is similar. The CI parameters seem quite random.

I'll see if I can replicate it.

[Ch3ri0ur]

Jup, looks like the tests are flakey (kind of expected with the random versions of python and docutil that failed earlier):

On the previous run we had

With mostly this error:

FAILED tests/test_builders/test_build_html_5_output.py::test_html5_output[subdir/images.html-.//img[@src='../_images/img1.png']-] - AssertionError: expecting at least 1 node(s) to satisfy ".//img[@src='../_images/img1.png']" in file subdir/images.html
FAILED tests/test_builders/test_build_html_5_output.py::test_html5_output[subdir/includes.html-.//img[@src='../_images/img.png']-] - AssertionError: expecting at least 1 node(s) to satisfy ".//img[@src='../_images/img.png']" in file subdir/includes.html
====== 2 failed, 2324 passed, 26 skipped, 10 warnings in 84.01s (0:01:24) ======
Error: Process completed with exit code 1.

This time its random other python/docutils versions failed (with the same error) but somewhat unexpected:
We got two new failed runs (Docutils and Latex) with a different error.

The new error:

FAILED tests/test_directives/test_directive_only.py::test_sectioning - AssertionError: Section out of place: '1.6.2. Subsection'
assert '1.6.1.1.' == '1.6.2.'
  
  - 1.6.2.
  + 1.6.1.1.
===== 1 failed, 2341 passed, 10 skipped, 222 warnings in 219.90s (0:03:39) =====