ci: restructure workflow into parallel jobs

[ismoiliy98]

Split the monolithic CI workflow into separate parallel jobs for better efficiency and clarity. The workflow now includes:

• A prepare job for setting up tools
• Parallel linting jobs for eslint and types
• Parallel testing jobs for unit, e2e and build tests
• A publish job that depends on successful linting and testing

[schiller-manuel]

@lachlancollins can you please have a look whether this makes sense?

[lachlancollins]

@ismoiliy98 I'm a bit unsure about this, I can't see how it would save time over the Nx distributed execution. Is there some reference for an improvement?

[ismoiliy98]

@lachlancollins you are right. Nx Cloud's distributed execution is excellent for speeding up test and build pipelines. This PR isn't trying to replace that, though - it's more about getting the workflow better organized in terms of scalability and maintainability (IMHO)

Motivations behind this change:

• Parallelization at the job level:
  • By splitting out lint, test, and build into matrix jobs, we unlock concurrent execution across independent runners.
  • This allows GitHub Actions to schedule them in parallel without depending solely on Nx agents (which still run sequential steps internally, even if distributed).
• Status checks & Visibility:
  • GitHub's UI shows which exact phase (lint, test, build) failed or passed.
  • Easier debugging and quicker feedback during development and PR reviews.
• Still Nx-compatible:
  • We can bring back nx-cloud start-ci-run inside the matrix jobs, especially inside the build and test steps.
  • We can combine Nx's distributed tasks within each matrix job to get the best of both worlds.
• Faster feedback loop for contributors: Linting and type checks run independently of heavy tests/builds, giving early red/green signals.

[lachlancollins]

I'm still not sure I'm seeing the benefits here.

Firstly, (in theory) most of the tasks in the CI workflow should already be cached in Nx from the PR workflow. If 100% of the cache matches, the workflow can finish pretty quickly once it spins up.

In terms of parallelisation/concurrency, running each task (e.g. test:lint and test:lib) separately will actually be slower, as each task needs to restore the same dependent jobs on each worker (mostly building dependencies). When running 5 separate tasks, that's a lot of duplication. The Nx workflow gets distributed over 8+ machines, each running up to 3 tasks, so that's 24+ parallel jobs which don't get duplicated.

Regarding visibility, the Nx workflow generates a page that lets you view the real-time logs, and shows the specific sub-tasks which have failed all in one interface. If this was split into 5 workflows, you might need to go through each interface to find the errors.

There is a non-negligible startup time for Nx distributed workflows, but i imagine a similar hurdle would be encountered by spinning up multiple GitHub actions. Once Nx is initialised, it really does zoom through the tasks.