Prunning research

[andrii-rohovyi]

No description provided.

[PatrickSteil]

Hello,
Thanks for the PR.

I looked at the code and wondered whether the new pruning rule makes a difference in performance.
The idea is correct, but the big “problem” with CSA is that simply scanning and processing a single connection (once it is in the cache) has no impact on performance.
In other words, not scanning a connection when it is in the cache does not matter. It would be important not to load a connection in the first place, thus avoiding filling the cache with “unnecessary” connections.
My assumption was also confirmed by measured query times: The pruning rule for CSA adds overhead, and does not improve performance.

This is without your pruning rule:

> runTransitiveCSAQueries /home/patrick/Documents/ULTRA_branch/ULTRA/Datasets/Karlsruhe/csa.binary 10000 0 true
Loading static graph from /home/patrick/Documents/ULTRA_branch/ULTRA/Datasets/Karlsruhe/csa.binary.graph
CSA public transit data:
   Number of Stops:                  4,214
   Number of Isolated Stops:         2,137
   Number of Trips:                 94,644
   Number of Stop Events:        1,675,308
   Number of Connections:        1,580,664
   Number of Vertices:               4,214
   Number of Edges:                  6,282
   Stops without departures:            11
   Stops without arrivals:              15
   Stops without connections:            0
   First Day:                            0
   Last Day:                             2
   First Departure:                   3:11
   Last Arrival:                   2d 4:48
   Bounding Box:              [(7.92986, 48.4444) | (9.52729, 49.3522)]

Total time: 663µs
	Clear: 30µs
	Initialization: 0µs
	Connection scan: 632µs

Scanned connections: 53,852.59
Relaxed edges: 1,267.90
Updated stops by trip: 1,347.24
Updated stops by transfer: 986.65

and this is with:

> runTransitiveCSAQueries /home/patrick/Documents/ULTRA_branch/ULTRA/Datasets/Karlsruhe/csa.binary 10000 1 true
Loading static graph from /home/patrick/Documents/ULTRA_branch/ULTRA/Datasets/Karlsruhe/csa.binary.graph
CSA public transit data:
   Number of Stops:                  4,214
   Number of Isolated Stops:         2,137
   Number of Trips:                 94,644
   Number of Stop Events:        1,675,308
   Number of Connections:        1,580,664
   Number of Vertices:               4,214
   Number of Edges:                  6,282
   Stops without departures:            11
   Stops without arrivals:              15
   Stops without connections:            0
   First Day:                            0
   Last Day:                             2
   First Departure:                   3:11
   Last Arrival:                   2d 4:48
   Bounding Box:              [(7.92986, 48.4444) | (9.52729, 49.3522)]

Total time: 854µs
	Clear: 31µs
	Initialization: 0µs
	Connection scan: 822µs

Scanned connections: 53,266.26
Relaxed edges: 1,244.21
Updated stops by trip: 1,327.67
Updated stops by transfer: 964.87

A similar picture emerges with the changes to RAPTOR. If the number of outgoing transfers is rather low, then there is no point in not scanning the few footpaths. It could help if you have many footpaths per stop; I have not tested this.
On the same instance as above:

Without pruning rule:

> runTransitiveRAPTORQueries /home/patrick/Documents/ULTRA_branch/ULTRA/Datasets/Karlsruhe/raptor.binary 1000 0
Loading static graph from /home/patrick/Documents/ULTRA_branch/ULTRA/Datasets/Karlsruhe/raptor.binary.graph
RAPTOR public transit data:
   Number of Stops:                 4,214
   Number of Routes:                4,391
   Number of Trips:                94,644
   Number of Stop Events:       1,675,308
   Number of Connections:       1,580,664
   Number of Vertices:              4,214
   Number of Edges:                 6,282
   First Day:                           0
   Last Day:                            2
   Bounding Box:             [(7.92986, 48.4444) | (9.52729, 49.3522)]

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
    init            0            0            1            1                1            7µs            0µs            0µs            0µs            7µs
       0           46          690          136          114               92            7µs            0µs            7µs            2µs           20µs
       1          606       10,042          869          954              440            7µs           37µs          111µs           15µs          173µs
       2        2,051       34,445        1,483        2,042              641            7µs          167µs          357µs           30µs          565µs
       3        2,856       47,724          909        1,435              407            7µs          258µs          464µs           23µs          755µs
       4        2,232       35,210          344          593              161            7µs          174µs          334µs           11µs          529µs
       5        1,217       17,996           93          176               47            6µs           77µs          170µs            3µs          259µs
       6          488        6,846           21           42               10            5µs           25µs           63µs            1µs           97µs
       7          158        2,156            3            7                1            3µs            6µs           19µs            0µs           30µs
       8           37          507            0            1                0            3µs            1µs            4µs            0µs            9µs
       9            7          106            0            0                0            0µs            0µs            0µs            0µs            1µs
      10            2           32            0            0                0            0µs            0µs            0µs            0µs            0µs
      11            0           13            0            0                0            0µs            0µs            0µs            0µs            0µs
      12            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
   total        9,700      155,767        3,859        5,365            1,800           63µs          751µs      1ms 535µs           88µs      2ms 452µs
Total time: 2ms 456µs
Avg. rounds: 7.31
Avg. journeys: 1.66

and with:

> runTransitiveRAPTORQueries /home/patrick/Documents/ULTRA_branch/ULTRA/Datasets/Karlsruhe/raptor.binary 1000 1
Loading static graph from /home/patrick/Documents/ULTRA_branch/ULTRA/Datasets/Karlsruhe/raptor.binary.graph
RAPTOR public transit data:
   Number of Stops:                 4,214
   Number of Routes:                4,391
   Number of Trips:                94,644
   Number of Stop Events:       1,675,308
   Number of Connections:       1,580,664
   Number of Vertices:              4,214
   Number of Edges:                 6,282
   First Day:                           0
   Last Day:                            2
   Bounding Box:             [(7.92986, 48.4444) | (9.52729, 49.3522)]

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
    init            0            0            1            1                1            7µs            0µs            0µs            0µs            8µs
       0           46          690          133          115               92            7µs            0µs            7µs            2µs           20µs
       1          606       10,050          830          960              439            7µs           38µs          111µs           16µs          174µs
       2        2,053       34,470        1,378        2,037              642            7µs          172µs          356µs           30µs          568µs
       3        2,857       47,761          858        1,438              407            7µs          266µs          461µs           23µs          761µs
       4        2,235       35,288          327          595              160            7µs          179µs          332µs           11µs          533µs
       5        1,218       17,997           90          176               47            6µs           80µs          169µs            4µs          262µs
       6          489        6,846           20           42               10            5µs           25µs           63µs            1µs           97µs
       7          158        2,156            3            7                1            3µs            6µs           19µs            0µs           30µs
       8           37          508            0            1                0            1µs            1µs            4µs            0µs            7µs
       9            7          106            0            0                0            0µs            0µs            0µs            0µs            1µs
      10            2           32            0            0                0            0µs            0µs            0µs            0µs            0µs
      11            0           13            0            0                0            0µs            0µs            0µs            0µs            0µs
      12            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
   total        9,708      155,917        3,640        5,372            1,799           63µs          773µs      1ms 528µs           89µs      2ms 467µs
Total time: 2ms 472µs
Avg. rounds: 7.31
Avg. journeys: 1.66

[andrii-rohovyi]

Dear @PatrickSteil,

Big thank you for your message, and I sincerely apologise for the unprepared PR. I made it coincidentally during my research.

I was exploring the possibility of pruning the loop in the iterative footpath search part of algorithms that are not based on graphs, such as CSA and RAPTOR. I got the same results as you for CSA and did not see any performance improvement. However, there was a speed boost for RATPTOR and ULTRA-RAPTOR on the London and Switzerland datasets, which you shared publicly. The likely reason is that London and Switzerland are larger than Karlsruhe, and as you mentioned, there might be more transfer connections in the graph, which could be a factor. So, in larger cities, we might see speed-ups (+~25% for RAPTOR) (+~15% for ULTRA-RAPTOR) for these algorithms.

Here are the results:

• RAPTOR (Switzerland)

> checkRAPTORPruning /Users/andriirohovyi/dev/unsw/ULTRA/Networks/Switzerland/Walking/Transitive/raptor.binary 1000
Loading static graph from /Users/andriirohovyi/dev/unsw/ULTRA/Networks/Switzerland/Walking/Transitive/raptor.binary.graph
--- Running with No Pruning (Rule 0) ---
--- Statistics for No Pruning (Rule 0) ---

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs           73µs
    init            0            0          102            1              101          124µs            0µs            0µs            9µs          139µs
       0          130        1,275       34,562          284            2,695          127µs           48µs           85µs          504µs          775µs
       1        1,516       15,994      273,334        2,930            8,263          133µs          552µs          973µs      3ms 550µs      5ms 220µs
       2        4,907       55,108      645,096        7,222           10,847          129µs      2ms  62µs      3ms 197µs      8ms  52µs     13ms 454µs
       3        7,423       84,882      647,372        7,386            8,092          122µs      3ms 330µs      4ms 704µs      7ms 967µs     16ms 138µs
       4        6,681       76,032      367,475        4,046            3,354          122µs      2ms 940µs      4ms   6µs      4ms 488µs     11ms 571µs
       5        3,800       42,206      125,172        1,360              997          120µs      1ms 572µs      2ms 182µs      1ms 545µs      5ms 431µs
       6        1,487       16,113       29,627          338              222          106µs          569µs          826µs          372µs      1ms 881µs
       7          422        4,522        5,194           69               42           79µs          153µs          225µs           68µs          531µs
       8           91          989          514           10                4           45µs           31µs           45µs            7µs          133µs
       9           14          163           15            1                0           20µs            4µs            7µs            0µs           34µs
      10            1           18            1            0                0            7µs            0µs            0µs            0µs            9µs
      11            0            2            0            0                0            1µs            0µs            0µs            0µs            2µs
      12            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
   total       26,472      297,304    2,128,464       23,647           34,617      1ms 142µs     11ms 265µs     16ms 256µs     26ms 568µs     55ms 398µs
Total time: 55ms 419µs
Avg. rounds: 8.00

--- Running with Pruning Rule 1 ---
--- Statistics for Pruning Rule 1 ---

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs          186µs
    init            0            0          102            1              100          111µs            0µs            0µs            9µs          125µs
       0          129        1,267       34,300          277            2,728          111µs           31µs           64µs          523µs          739µs
       1        1,514       15,976      250,253        2,878            8,039          111µs          387µs          713µs      3ms 449µs      4ms 671µs
       2        4,896       54,962      502,344        7,133           10,494          108µs      1ms 426µs      2ms 337µs      6ms 864µs     10ms 747µs
       3        7,422       84,877      433,056        7,289            7,811          109µs      2ms 278µs      3ms 408µs      6ms  16µs     11ms 823µs
       4        6,682       76,046      202,170        3,998            3,231          108µs      2ms  32µs      2ms 923µs      2ms 890µs      7ms 966µs
       5        3,800       42,210       61,507        1,350              968          106µs      1ms  74µs      1ms 591µs          915µs      3ms 698µs
       6        1,489       16,128       13,418          336              215           93µs          390µs          596µs          209µs      1ms 296µs
       7          421        4,517        2,132           68               41           68µs          103µs          164µs           35µs          377µs
       8           91          989          221           10                4           41µs           20µs           34µs            4µs          103µs
       9           14          163            6            1                0           20µs            3µs            5µs            0µs           31µs
      10            1           18            0            0                0            7µs            0µs            0µs            0µs            9µs
      11            0            2            0            0                0            1µs            0µs            0µs            0µs            2µs
      12            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
   total       26,459      297,155    1,499,509       23,341           33,631      1ms   1µs      7ms 750µs     11ms 840µs     20ms 919µs     41ms 779µs
Total time: 41ms 799µs
Avg. rounds: 8.00

--- Comparison Results ---
Pruning rule 1 results match no-pruning results. The pruning is correct.

• RAPTOR (London)

> checkRAPTORPruning /Users/andriirohovyi/dev/unsw/ULTRA/Networks/London/Walking/Transitive/raptor.binary 1000                                                                                
Loading static graph from /Users/andriirohovyi/dev/unsw/ULTRA/Networks/London/Walking/Transitive/raptor.binary.graph
--- Running with No Pruning (Rule 0) ---
--- Statistics for No Pruning (Rule 0) ---

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs           60µs
    init            0            0          141            1              140          102µs            0µs            0µs           11µs          118µs
       0           66        1,488      143,334          622           10,386           95µs           37µs          140µs      1ms 815µs      2ms  96µs
       1          505       14,030      682,110        4,006           19,414           94µs          445µs      1ms 103µs      8ms  13µs      9ms 666µs
       2        1,029       31,481      914,423        5,953           14,109           95µs      1ms 140µs      2ms 184µs     10ms 391µs     13ms 823µs
       3        1,114       34,728      549,646        3,687            6,432          106µs      1ms 279µs      2ms 213µs      6ms 111µs      9ms 720µs
       4          877       26,183      233,675        1,528            2,509          100µs          835µs      1ms 614µs      2ms 593µs      5ms 154µs
       5          542       15,405       83,741          512              840           95µs          405µs          947µs          935µs      2ms 391µs
       6          256        7,012       24,687          155              239           82µs          155µs          426µs          276µs          947µs
       7           98        2,613        6,140           35               55           62µs           49µs          156µs           68µs          342µs
       8           30          773        1,703            8               12           35µs           12µs           45µs           18µs          115µs
       9            7          193          301            1                1           15µs            3µs           12µs            3µs           35µs
      10            1           41           25            0                0            4µs            0µs            2µs            0µs            8µs
      11            0            6            0            0                0            1µs            0µs            0µs            0µs            1µs
      12            0            1            0            0                0            0µs            0µs            0µs            0µs            0µs
   total        4,525      133,954    2,639,926       16,508           54,137          892µs      4ms 365µs      8ms 848µs     30ms 238µs     44ms 484µs
Total time: 44ms 503µs
Avg. rounds: 7.71

--- Running with Pruning Rule 1 ---
--- Statistics for Pruning Rule 1 ---

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs          142µs
    init            0            0          141            1              137           94µs            0µs            0µs           11µs          109µs
       0           65        1,469      138,929          590           10,098           91µs           27µs          108µs      1ms 973µs      2ms 208µs
       1          503       13,963      544,459        3,941           19,043           89µs          351µs          854µs      7ms 321µs      8ms 627µs
       2        1,028       31,454      561,725        5,879           14,002           87µs          900µs      1ms 684µs      7ms 623µs     10ms 308µs
       3        1,113       34,680      208,672        3,679            6,407           86µs          976µs      1ms 661µs      3ms  45µs      5ms 780µs
       4          877       26,175       65,476        1,526            2,493           83µs          628µs      1ms 176µs      1ms  27µs      2ms 926µs
       5          541       15,384       19,389          512              840           78µs          298µs          680µs          328µs      1ms 394µs
       6          256        7,015        5,074          155              239           71µs          115µs          308µs           90µs          592µs
       7           98        2,613          964           35               54           51µs           36µs          114µs           19µs          226µs
       8           30          772          238            8               12           29µs            9µs           34µs            4µs           80µs
       9            7          193           38            1                1           12µs            2µs            9µs            0µs           26µs
      10            1           41            3            0                0            4µs            0µs            2µs            0µs            7µs
      11            0            6            0            0                0            1µs            0µs            0µs            0µs            1µs
      12            0            1            0            0                0            0µs            0µs            0µs            0µs            0µs
   total        4,519      133,766    1,545,108       16,327           53,326          782µs      3ms 349µs      6ms 634µs     21ms 446µs     32ms 432µs
Total time: 32ms 450µs
Avg. rounds: 7.71

--- Comparison Results ---
Pruning rule 1 results match no-pruning results. The pruning is correct.

Also, here is the results for:

• ULTRA-RAPTOR (Switzerland)

> checkULTRARAPTORPruning /Users/andriirohovyi/dev/unsw/ULTRA/Networks/Switzerland/Walking/Shortcuts/raptor.binary /Users/andriirohovyi/dev/unsw/ULTRA/Networks/Switzerland/Walking/CH/ch 1000
Loading static graph from /Users/andriirohovyi/dev/unsw/ULTRA/Networks/Switzerland/Walking/Shortcuts/raptor.binary.graph
Loading static graph from /Users/andriirohovyi/dev/unsw/ULTRA/Networks/Switzerland/Walking/CH/ch.forward
Loading static graph from /Users/andriirohovyi/dev/unsw/ULTRA/Networks/Switzerland/Walking/CH/ch.backward
--- Statistics for No Pruning (Rule 0) ---

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs           84µs
    init            0            0            0            0           10,772          102µs            0µs            0µs      7ms 655µs      7ms 766µs
       0        4,900       57,002       38,981        4,951            4,967          119µs      2ms 595µs      2ms 470µs      1ms 237µs      6ms 438µs
       1        6,972       80,202       55,666       11,498            5,278          119µs      2ms 182µs      3ms 698µs      1ms 749µs      7ms 766µs
       2        8,189       96,184       44,733        9,587            3,551          115µs      2ms 823µs      4ms 265µs      1ms 473µs      8ms 694µs
       3        7,462       86,247       24,175        5,366            1,551          116µs      2ms 412µs      3ms 676µs          840µs      7ms  61µs
       4        4,948       55,288        8,205        1,842              440          113µs      1ms 415µs      2ms 308µs          311µs      4ms 162µs
       5        2,111       22,621        1,896          436               94          106µs          530µs          938µs           82µs      1ms 668µs
       6          580        6,065          305           75               14           83µs          133µs          239µs           15µs          480µs
       7          114        1,191           48           12                2           50µs           24µs           44µs            2µs          125µs
       8           17          188            4            1                0           21µs            3µs            6µs            0µs           33µs
       9            2           23            0            0                0            5µs            0µs            0µs            0µs            7µs
      10            0            1            0            0                0            1µs            0µs            0µs            0µs            1µs
      11            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
   total       35,295      405,012      174,013       33,768           26,669          956µs     12ms 123µs     17ms 649µs     13ms 368µs     44ms 290µs
Total time: 44ms 314µs
Avg. rounds: 7.28
--- Statistics for Pruning Rule 1 ---

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs          201µs
    init            0            0            0            0           10,772          101µs            0µs            0µs      7ms 733µs      7ms 842µs
       0        4,900       57,002       32,858        4,951            4,967          126µs      2ms 367µs      2ms 223µs          927µs      5ms 658µs
       1        6,972       80,202       35,187       11,409            5,248          125µs      1ms 997µs      3ms 319µs      1ms 162µs      6ms 617µs
       2        8,185       96,157       22,726        9,509            3,509          112µs      2ms 560µs      3ms 778µs          910µs      7ms 377µs
       3        7,460       86,220       10,691        5,328            1,534          112µs      2ms 170µs      3ms 207µs          511µs      6ms  16µs
       4        4,946       55,267        3,324        1,830              436          109µs      1ms 265µs      2ms  11µs          193µs      3ms 591µs
       5        2,109       22,607          708          434               93          103µs          471µs          824µs           52µs      1ms 462µs
       6          579        6,054          103           75               14           84µs          118µs          213µs           10µs          434µs
       7          114        1,188           14           12                1           50µs           22µs           39µs            1µs          117µs
       8           17          188            1            1                0           20µs            3µs            6µs            0µs           32µs
       9            2           23            0            0                0            6µs            0µs            0µs            0µs            8µs
      10            0            1            0            0                0            1µs            0µs            0µs            0µs            1µs
      11            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
   total       35,284      404,909      105,612       33,549           26,574          955µs     10ms 978µs     15ms 626µs     11ms 505µs     39ms 361µs
Total time: 39ms 384µs
Avg. rounds: 7.28

-ULTRA-RAPTOR (London)

> CheckULTRARAPTORPruning
RAPTOR input file> /Users/andriirohovyi/dev/unsw/ULTRA/Networks/London/Walking/Shortcuts/raptor.binary   
CH data> /Users/andriirohovyi/dev/unsw/ULTRA/Networks/London/Walking/CH/ch                  
Number of queries> 1000
Loading static graph from /Users/andriirohovyi/dev/unsw/ULTRA/Networks/London/Walking/Shortcuts/raptor.binary.graph
Loading static graph from /Users/andriirohovyi/dev/unsw/ULTRA/Networks/London/Walking/CH/ch.forward
Loading static graph from /Users/andriirohovyi/dev/unsw/ULTRA/Networks/London/Walking/CH/ch.backward
--- Statistics for No Pruning (Rule 0) ---

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs          101µs
    init            0            0            0            0            9,120           86µs            0µs            0µs     10ms 276µs     10ms 370µs
       0          869       27,211       54,828        5,169            9,178           98µs      1ms 499µs      2ms 146µs      1ms 740µs      5ms 496µs
       1        1,130       35,785       59,279        6,923            8,492          100µs      1ms 526µs      2ms 600µs      1ms 971µs      6ms 210µs
       2        1,149       36,327       37,338        4,544            4,697           99µs      1ms 535µs      2ms 485µs      1ms 206µs      5ms 338µs
       3        1,010       30,603       18,087        2,246            2,035           96µs      1ms  46µs      1ms 895µs          629µs      3ms 679µs
       4          711       20,369        6,692          838              698           92µs          557µs      1ms 249µs          246µs      2ms 155µs
       5          380       10,371        1,926          246              188           82µs          229µs          640µs           80µs      1ms  40µs
       6          150        3,928          440           58               41           66µs           71µs          233µs           20µs          396µs
       7           44        1,116           77           10                6           44µs           17µs           62µs            4µs          132µs
       8           10          250            8            1                0           22µs            3µs           13µs            0µs           41µs
       9            1           39            1            0                0            7µs            0µs            1µs            0µs           10µs
      10            0            5            0            0                0            1µs            0µs            0µs            0µs            2µs
      11            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
   total        5,454      166,004      178,676       20,035           34,455          800µs      6ms 487µs     11ms 328µs     16ms 177µs     34ms 978µs
Total time: 34ms 996µs
Avg. rounds: 7.24
--- Statistics for Pruning Rule 1 ---

Statistics:
   Round       Routes     Segments        Edges Stops (trip) Stops (transfer)           Init        Collect           Scan      Transfers          Total
   clear            0            0            0            0                0            0µs            0µs            0µs            0µs          127µs
    init            0            0            0            0            9,120           75µs            0µs            0µs      9ms 898µs      9ms 980µs
       0          869       27,211       46,903        5,169            9,178           86µs      1ms  29µs      1ms 581µs      1ms 210µs      3ms 918µs
       1        1,130       35,785       44,438        6,890            8,424           86µs      1ms  37µs      1ms 775µs      1ms 213µs      4ms 123µs
       2        1,149       36,294       24,496        4,519            4,670           85µs      1ms  33µs      1ms 601µs          742µs      3ms 473µs
       3        1,010       30,591       10,718        2,237            2,029           84µs          718µs      1ms 274µs          360µs      2ms 447µs
       4          711       20,371        3,677          836              698           81µs          383µs          843µs          138µs      1ms 455µs
       5          380       10,369          983          245              188           73µs          156µs          430µs           42µs          709µs
       6          150        3,925          209           58               41           58µs           49µs          160µs           10µs          284µs
       7           44        1,115           36           10                6           39µs           12µs           43µs            2µs           99µs
       8           10          250            3            1                0           19µs            2µs            9µs            0µs           33µs
       9            1           39            0            0                0            7µs            0µs            1µs            0µs            9µs
      10            0            5            0            0                0            1µs            0µs            0µs            0µs            1µs
      11            0            0            0            0                0            0µs            0µs            0µs            0µs            0µs
   total        5,454      165,955      131,463       19,965           34,354          699µs      4ms 423µs      7ms 720µs     13ms 619µs     26ms 664µs
Total time: 26ms 680µs
Avg. rounds: 7.24

P.S. I'll also add my code updates in this PR. I made some slight optimisations. Please let me know if you observe the same improvements on larger datasets.

Big thank you for your quick response and validation of my idea, I really appreciate it!

[PatrickSteil]

If you’d like to further tune and optimise your ULTRA branch, feel free to check out my PR #24 ;especially the commit fc26d7f. It might add some performance for your use case.

[andrii-rohovyi]

Thanks heaps, @PatrickSteil, for sharing! I'll review it.

Is it alright if I add this RAPTOR and ULTRA RAPTOR modification to your PR?

[PatrickSteil]

Thanks for the offer! I’d prefer to keep them separate so Jonas and I can more easily distinguish what each branch does and who contributed which changes.

[andrii-rohovyi]

Sure, thank you! Yeah, you are right. I will then review your modification and attempt to combine the ideas while keeping them separate. Big thank you for your help and suggestions!