Tests are taking too long

[tomchor]

It's not a major issue at the moment, but test are currently taking about 2.5 hours. See for example #195

This is because tests in Oceanostics have nested loops that test pretty much every combination of grids, closures, etc. In a way that's good because it covers a huge number of cases and combinations, but on the other hand it might be unnecessarily thorough, and as the package grows and tests multiply, this becomes exponentially slow.

[glwagner]

I'm not sure what this does:

Oceanostics.jl/.github/workflows/ci.yml

Line 26 in 875f330

- uses: julia-actions/julia-runtest@v1

but one way you can reduce test time is to run with optimization level -O0. Testing is probably 99% compile time, and O0 can cut compile time in half.

I also agree that it's nice but not necessary to test everything. We should test common and representative cases, and add edge cases if we anticipate or find a problem.

[tomchor]

I'm not sure what this does:

Oceanostics.jl/.github/workflows/ci.yml

Line 26 in 875f330

• uses: julia-actions/julia-runtest@v1

It's what actually runs the tests: https://github.com/julia-actions/julia-runtest. Not sure how to change the compile flag there though...

[glwagner]

Sorry yeah I gathered it ran the tests, but I don't know what command it uses. However, you can basically just write

julia --project -O0 -e 'using Pkg; Pkg.test()'

if you want to configure things manually. Or maybe there is a way to pass options to that utility

[glwagner]

You can also break up the test into different actions

[tomchor]

You can also break up the test into different actions

Yeah that's a good point! I'll still investigate how to pass an optimization flag to julia though.

[tomchor]

Two things:

1. Currently there's no way to pass optimization flag to julia-runtest, but there's a PR to implement that: add optimize as an input julia-actions/julia-runtest#140
2. Separating the tests helps quite a bit. After Separate tests #197 they went from 2.5 hours to 1.5 hours, and the tests that take the longest are the tracer tests and KE tests. I'll consider this issue closed once we get the tests down to an hour or less, which can be down by further separating them, or by being a bit smarter about what we test. I'll do that in another PR though.