keeping built/installed historical versions between atime CI runs #6528
Description
now that we have 10+ performance tests, the amount of time taken by performance testing CI is getting larger, 8min last I checked. I believe most of this time is taken installing different historical/reference versions of data.table, which do not change between runs of the performance testing CI jobs.
in other words, each CI job needs to compile and install a bunch of different git versions of data.table, and some of these versions are different in each run (master, merge-base, HEAD, CRAN), but many are the same (historical versions).
https://github.com/actions/upload-artifact could be used at the end of our CI job, to upload compiled/installed historical versions of data.table, then https://github.com/actions/download-artifact could be used at the begining of each CI job to download previously compiled/installed historical versions of data.table.
There is some trade-off between storage space and Ci run time. I don't think it is currently worth our time to actually implement this, because https://github.com/organizations/Rdatatable/settings/billing/summary says we are nowhere close to our quota (0 usage in all categories, quota resets in 18 days). But one day we may need to consider implementing this.
https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions#included-storage-and-minutes says the free plan includes 500 MB, and 2000 minutes per month. I estimate ~10MB per installed historical version of data.table, which means we could get speedups for free up to 50 historical versions. Below we see file sizes of 13 installed historical versions of data.table on my local computer, total ~100MB, so average per version a bit less than ~10MB.
(base) tdhock@tdhock-MacBook:~/R/atime[combine]$ du -ksc /home/tdhock/lib/R/library/data.table.*/*|sort -n
4 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/cc
4 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/INDEX
4 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/cc
4 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/INDEX
4 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/cc
4 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/INDEX
4 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/cc
4 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/INDEX
4 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/cc
4 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/INDEX
4 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/cc
4 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/INDEX
4 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/cc
4 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/INDEX
4 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/cc
4 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/INDEX
4 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/cc
4 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/INDEX
4 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/cc
4 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/INDEX
4 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/cc
4 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/INDEX
4 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/cc
4 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/INDEX
4 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/cc
4 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/INDEX
8 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/include
8 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/include
8 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/include
8 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/include
8 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/include
8 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/include
8 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/include
8 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/include
8 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/include
8 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/include
8 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/NAMESPACE
8 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/DESCRIPTION
8 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/include
8 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/NAMESPACE
20 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/LICENSE
20 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/LICENSE
20 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/LICENSE
20 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/LICENSE
20 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/LICENSE
20 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/LICENSE
20 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/LICENSE
20 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/LICENSE
20 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/LICENSE
20 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/LICENSE
20 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/LICENSE
20 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/LICENSE
20 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/LICENSE
28 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/Meta
28 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/Meta
28 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/Meta
28 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/Meta
28 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/Meta
28 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/Meta
28 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/Meta
28 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/Meta
28 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/Meta
28 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/Meta
28 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/Meta
28 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/Meta
28 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/Meta
40 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/html
40 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/html
40 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/html
40 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/html
40 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/html
40 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/html
40 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/html
40 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/html
44 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/html
44 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/html
44 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/html
44 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/html
44 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/html
92 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/NEWS.md
92 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/NEWS.md
96 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/NEWS.md
96 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/NEWS.md
96 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/NEWS.md
160 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/NEWS.md
164 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/NEWS.md
188 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/NEWS.md
188 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/NEWS.md
204 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/NEWS.md
204 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/NEWS.md
244 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/NEWS.md
248 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/NEWS.md
388 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/help
388 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/help
392 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/help
392 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/help
404 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/help
404 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/help
416 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/help
416 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/help
428 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/help
428 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/help
432 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/help
432 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/help
432 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/help
472 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/po
472 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/po
484 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/po
484 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/po
484 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/po
484 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/po
484 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/po
484 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/po
532 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/R
532 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/R
532 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/R
532 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/R
552 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/R
552 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/R
572 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/R
576 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/R
596 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/R
596 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/R
604 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/R
604 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/R
604 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/R
1056 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/po
1056 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/po
1304 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/libs
1308 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/libs
1320 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/po
1320 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/po
1320 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/po
1340 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/libs
1512 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/tests
1520 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/tests
1544 /home/tdhock/lib/R/library/data.table.be2f72e6f5c90622fe72e1c315ca05769a9dc854/tests
1544 /home/tdhock/lib/R/library/data.table.e793f53466d99f86e70fc2611b708ae8c601a451/tests
1624 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/tests
1624 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/tests
1720 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/tests
1724 /home/tdhock/lib/R/library/data.table.4c633907a651f25a4e02c86e6bebb6f5c8c37fa8/tests
1816 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/tests
1816 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/tests
1900 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/tests
1900 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/tests
1900 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/tests
1976 /home/tdhock/lib/R/library/data.table.58409197426ced4714af842650b0cc3b9e2cb842/libs
2532 /home/tdhock/lib/R/library/data.table.e9087ce9860bac77c51467b19e92cf4b72ca78c7/libs
3164 /home/tdhock/lib/R/library/data.table.b1b1832b0d2d4032b46477d9fe6efb15006664f4/libs
3264 /home/tdhock/lib/R/library/data.table.fd24a3105953f7785ea7414678ed8e04524e6955/libs
3392 /home/tdhock/lib/R/library/data.table.8973959eed39eb1622cef683ac811977cd10dcfd/libs
4220 /home/tdhock/lib/R/library/data.table.a77e8c22e44e904835d7b34b047df2eff069d1f2/libs
4664 /home/tdhock/lib/R/library/data.table.9d3b9202fddb980345025a4f6ac451ed26a423be/libs
4836 /home/tdhock/lib/R/library/data.table.ed72e398df76a0fcfd134a4ad92356690e4210ea/libs
6356 /home/tdhock/lib/R/library/data.table.a60b749f9439cfc8e622115d9083a85fde1d051a/libs
8052 /home/tdhock/lib/R/library/data.table.3b2d5bc777b43f54d72f448169c893172530ac9a/libs
94844 total
(base) tdhock@tdhock-MacBook:~/R/atime[combine]$ du -ks /home/tdhock/lib/R/library/data.table.*|wc -l
13