Skip to content

Commit df04aac

Browse files
bwbushbwbush
authored
Add sim-cli 1.4.0 to regression experiment (#608)
* Added leios-mempool-size-bytes parameter to config * Added 1.4.0 to regression tests * Completed 1.4.0 simulation * Analyzed 1.4.0 results and compared to previous versions
1 parent 961bcf2 commit df04aac

File tree

5 files changed

+360
-521
lines changed

5 files changed

+360
-521
lines changed

‎analysis/sims/regression/analysis.ipynb‎

Lines changed: 266 additions & 521 deletions
Large diffs are not rendered by default.

‎analysis/sims/regression/experiments/1.4.0/case.csv‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
sim-cli,Tx start [s],Tx stop [s],Sim stop [s]
2+
1.4.0,60,660,1200

‎analysis/sims/regression/experiments/1.4.0/run.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../run.sh

‎analysis/sims/regression/experiments/1.4.0/summary.txt‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
INFO praos: sim_cli::events: 60001 transactions(s) were generated in total.
2+
INFO praos: sim_cli::events: 64 naive praos block(s) were published.
3+
INFO praos: sim_cli::events: 1136 slot(s) had no naive praos blocks.
4+
INFO praos: sim_cli::events: 60001 transaction(s) (90.00 MB) finalized in a naive praos block.
5+
INFO praos: sim_cli::events: 0 transaction(s) (0 B) did not reach a naive praos block.
6+
INFO praos: sim_cli::events: Pool 3 published 1 naive praos block(s)
7+
INFO praos: sim_cli::events: Pool 4 published 1 naive praos block(s)
8+
INFO praos: sim_cli::events: Pool 38 published 1 naive praos block(s)
9+
INFO praos: sim_cli::events: Pool 39 published 1 naive praos block(s)
10+
INFO praos: sim_cli::events: Pool 40 published 1 naive praos block(s)
11+
INFO praos: sim_cli::events: Pool 41 published 2 naive praos block(s)
12+
INFO praos: sim_cli::events: Pool 42 published 1 naive praos block(s)
13+
INFO praos: sim_cli::events: Pool 45 published 1 naive praos block(s)
14+
INFO praos: sim_cli::events: Pool 46 published 1 naive praos block(s)
15+
INFO praos: sim_cli::events: Pool 50 published 1 naive praos block(s)
16+
INFO praos: sim_cli::events: Pool 53 published 2 naive praos block(s)
17+
INFO praos: sim_cli::events: Pool 54 published 1 naive praos block(s)
18+
INFO praos: sim_cli::events: Pool 58 published 1 naive praos block(s)
19+
INFO praos: sim_cli::events: Pool 59 published 2 naive praos block(s)
20+
INFO praos: sim_cli::events: Pool 67 published 1 naive praos block(s)
21+
INFO praos: sim_cli::events: Pool 68 failed to publish 1 naive praos block(s) due to slot battles.
22+
INFO praos: sim_cli::events: Pool 70 published 1 naive praos block(s)
23+
INFO praos: sim_cli::events: Pool 94 published 2 naive praos block(s)
24+
INFO praos: sim_cli::events: Pool 102 published 1 naive praos block(s)
25+
INFO praos: sim_cli::events: Pool 104 published 1 naive praos block(s)
26+
INFO praos: sim_cli::events: Pool 105 published 1 naive praos block(s)
27+
INFO praos: sim_cli::events: Pool 106 published 1 naive praos block(s)
28+
INFO praos: sim_cli::events: Pool 109 published 1 naive praos block(s)
29+
INFO praos: sim_cli::events: Pool 110 published 1 naive praos block(s)
30+
INFO praos: sim_cli::events: Pool 121 published 1 naive praos block(s)
31+
INFO praos: sim_cli::events: Pool 122 published 2 naive praos block(s)
32+
INFO praos: sim_cli::events: Pool 136 published 1 naive praos block(s)
33+
INFO praos: sim_cli::events: Pool 138 published 1 naive praos block(s)
34+
INFO praos: sim_cli::events: Pool 351 published 1 naive praos block(s)
35+
INFO praos: sim_cli::events: Pool 361 published 1 naive praos block(s)
36+
INFO praos: sim_cli::events: Pool 371 published 1 naive praos block(s)
37+
INFO praos: sim_cli::events: Pool 376 published 1 naive praos block(s)
38+
INFO praos: sim_cli::events: Pool 429 published 1 naive praos block(s)
39+
INFO praos: sim_cli::events: Pool 443 failed to publish 1 naive praos block(s) due to slot battles.
40+
INFO praos: sim_cli::events: Pool 446 published 1 naive praos block(s)
41+
INFO praos: sim_cli::events: Pool 477 published 1 naive praos block(s)
42+
INFO praos: sim_cli::events: Pool 478 published 1 naive praos block(s)
43+
INFO praos: sim_cli::events: Pool 481 published 2 naive praos block(s)
44+
INFO praos: sim_cli::events: Pool 509 published 1 naive praos block(s)
45+
INFO praos: sim_cli::events: Pool 510 published 1 naive praos block(s)
46+
INFO praos: sim_cli::events: Pool 511 published 1 naive praos block(s)
47+
INFO praos: sim_cli::events: Pool 515 published 1 naive praos block(s)
48+
INFO praos: sim_cli::events: Pool 518 published 2 naive praos block(s)
49+
INFO praos: sim_cli::events: Pool 525 published 1 naive praos block(s)
50+
INFO praos: sim_cli::events: Pool 527 published 2 naive praos block(s)
51+
INFO praos: sim_cli::events: Pool 537 published 2 naive praos block(s)
52+
INFO praos: sim_cli::events: Pool 540 published 1 naive praos block(s)
53+
INFO praos: sim_cli::events: Pool 545 published 2 naive praos block(s)
54+
INFO praos: sim_cli::events: Pool 556 published 1 naive praos block(s)
55+
INFO praos: sim_cli::events: Pool 560 published 1 naive praos block(s)
56+
INFO praos: sim_cli::events: Pool 562 published 1 naive praos block(s)
57+
INFO praos: sim_cli::events: Pool 567 published 1 naive praos block(s)
58+
INFO praos: sim_cli::events: Pool 741 published 2 naive praos block(s)
59+
INFO praos: sim_cli::events: Pool 742 published 1 naive praos block(s)
60+
INFO praos: sim_cli::events: Pool 745 published 1 naive praos block(s)
61+
INFO leios: sim_cli::events: 0 IB(s) were generated, on average 0.000 IB(s) per slot.
62+
INFO leios: sim_cli::events: 0 out of 60001 transaction(s) were included in at least one IB.
63+
INFO leios: sim_cli::events: The average age of the pending transactions is NaNs (stddev NaN).
64+
INFO leios: sim_cli::events: Each transaction was included in an average of NaN IB(s) (stddev NaN).
65+
INFO leios: sim_cli::events: Each IB contained an average of NaN transaction(s) (stddev NaN) and an average of 0 B (stddev 0 B). 0 IB(s) were empty.
66+
INFO leios: sim_cli::events: Each node received an average of 0.000 IB(s) (stddev 0.000).
67+
INFO leios: sim_cli::events: 33 EB(s) were generated; on average there were 0.028 EB(s) per slot.
68+
INFO leios: sim_cli::events: Each EB contained an average of 3200.424 transaction(s) (stddev 1469.795). 0 EB(s) were empty.
69+
INFO leios: sim_cli::events: Each EB contained an average of 0.000 IB(s) (stddev 0.000). 0 EB(s) were empty.
70+
INFO leios: sim_cli::events: Each IB was included in an average of NaN EB(s) (stddev NaN).
71+
INFO leios: sim_cli::events: 0 out of 0 IBs were included in at least one EB.
72+
INFO leios: sim_cli::events: 0 out of 0 IBs expired before they reached an EB.
73+
INFO leios: sim_cli::events: 17 out of 33 EBs expired before an EB from their stage reached an RB.
74+
INFO leios: sim_cli::events: 59941 out of 60001 transaction(s) were included in at least one EB.
75+
INFO leios: sim_cli::events: 17273 total votes were generated.
76+
INFO leios: sim_cli::events: Each stake pool produced an average of 79.968 vote(s) (stddev 23.324).
77+
INFO leios: sim_cli::events: Each EB received an average of 539.781 vote(s) (stddev 140.711).
78+
INFO leios: sim_cli::events: There were 5766 bundle(s) of votes. Each bundle contained 2.996 vote(s) (stddev 1.626).
79+
INFO leios: sim_cli::events: 17 L1 block(s) had a Leios endorsement.
80+
INFO leios: sim_cli::events: 58981 tx(s) (88.47 MB) were referenced by a Leios endorsement.
81+
INFO leios: sim_cli::events: 1020 tx(s) (1.53 MB) were included directly in a Praos block.
82+
INFO leios: sim_cli::events: Spatial efficiency: 88.47 MB/3.52 MB (2510.850%) of Leios bytes were unique transactions.
83+
INFO leios: sim_cli::events: 0 tx(s) (0.000%) referenced by a Leios endorsement were redundant.
84+
INFO leios: sim_cli::events: Each transaction took an average of NaNs (stddev NaN) to be included in an IB.
85+
INFO leios: sim_cli::events: Each transaction took an average of 16.748s (stddev 13.864) to be included in an EB.
86+
INFO leios: sim_cli::events: Each transaction took an average of 54.381s (stddev 20.233) to be included in a block.
87+
INFO network: sim_cli::events: 44940749 TX message(s) were sent. 44940749 of them were received (100.000%).
88+
INFO network: sim_cli::events: 0 IB message(s) were sent. 0 of them were received (NaN%).
89+
INFO network: sim_cli::events: 24260 EB message(s) were sent. 24260 of them were received (100.000%).
90+
INFO network: sim_cli::events: 4318734 Vote message(s) were sent. 4318734 of them were received (100.000%).

‎analysis/sims/regression/experiments/config.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ relay-strategy: request-from-first
1313
treat-blocks-as-full: false
1414
eb-diffusion-strategy: peer-order
1515
eb-referenced-txs-max-size-bytes: 12000000
16+
leios-mempool-size-bytes: 24000000
1617
eb-size-bytes-constant: 240
1718
eb-size-bytes-per-ib: 32
1819
leios-header-diffusion-time-ms: 1000.0

0 commit comments

Comments
 (0)