feat: Speculative Availability

[timwu20]

Description

Introduces Speculative Availability chunk requests within the Availability Distribution subsystem. In Availability Distibution subsystem we listen on ActiveLeavesUpdate and call Prospective Parachains to gather backable candidates that we can start fetching their erasure encoded chunks from the backing group before the candidates are actually backed on chain. This feature is currently enabled by running the node with the --speculative-availability CLI flag.

Review Notes

Note: This PR is a reimplementation of PR#9444 with less changes to the handling of fetch tasks, overall less diffs, and added CLI flag to enable feature.

• Moves request_backale_candidates out of Provisioner subsystem to subystem-util crate to be used as well in Availability Distribution subsystem.
• Introduces CoreInfo and CoreInfoOrigin private types in availability-distribution::requester module. CoreInfo is used to create fetch tasks now instead of using available cores directly. We construct CoreInfo instances from calling Prospective Parachains for backable candidates and we denote the origin of these CoreInfo instances with CoreInfo::Scheduled. CoreInfo instances created from candidates that have been backed on chain have CoreInfo::Occupied origin.
• Modifies Availability Store subsystem to accept new AvailabilityStoreMessage::NoteBackableCandidates message type.
  • This new message type is handled by the Availability Store subsystem to write the meta data to pass validation when accepting the actual chunk for the associated candidate hash. It was discovered during development, that storing the chunk from scheduled/early fetch requests were not being persisted due to not having the meta data previously stored for said candidate.
• Includes Zombienet test that utilizes two parachains which support elastic scaling. Assertions are done on the polkadot_parachain_fetched_chunks_total[origin="scheduled"] metric.
• --speculative-availability CLI flag support.

TODO

• Benchmark with speculative availability enabled.

Checklist

• My PR includes a detailed description as outlined in the "Description" and its two subsections above.
• My PR follows the labeling requirements of this project (at minimum one label for T required)
  • External contributors: Use /cmd label <label-name> to add labels
  • Maintainers can also add labels manually
• I have made corresponding changes to the documentation (if applicable)
• I have added tests that prove my fix is effective or that my feature works (if applicable)

[cla-bot-2021]

User @timwu20, please sign the CLA here.

[timwu20]

/cmd prdoc

[github-actions]

Command "prdoc" has failed ❌! See logs here

[timwu20]

/cmd label T0-node T8-polkadot

[github-actions]

Command "label T0-node T8-polkadot" has failed ❌! See logs here