Optimize importing archive packages

[SpyCheese]

No description provided.

[github-actions]

Blocking Issues

• validator/import-db-slice-local.cpp:551-560 – The new pipelined path now calls Db::add_handle_to_archive (which ultimately writes a handle into the archive index and flips the handle_moved_to_archive flag) before we know that the block will actually be applied (apply_block_async_3 / ValidatorManager::new_block). If new_block later rejects the block (bad state, timeout, etc.), apply_blocks_async propagates the failure and the importer exits, but at that point the archive already contains the new handle even though the block was never applied or fully archived. On the next retry the handle will be seen as “already moved”, so BlockArchiver will skip the add‑handle step and you’re left with an inconsistent archive (no files, moved_to_archive == false, but the LT index already advanced). Previously the add‑handle happened inside BlockArchiver, i.e. only after new_block triggered archiving and only if that succeeded. We now need either to defer add_handle_to_archive until after new_block completes, or to implement a rollback for the handle/lt‑db entry when later phases fail; otherwise a transient failure strands half‑archived handles and the import can’t be retried cleanly.

• validator/import-db-slice-local.cpp:99-104 – process_package now silently swallows any FileReference::create error and just return true;. Before this change we set S = F.move_as_error() and aborted the package, so the caller logged which file name was malformed/corrupted. With the new behaviour a broken entry (e.g. truncated proof filename) is skipped without any warning, the package reports success, and the importer only fails much later with a generic “no block data/proof” error – or worse, if that file was optional, the block is silently missing. At minimum we should keep surfacing the parsing error so operators know the archive on disk is corrupted; silently ignoring unknown file names makes diagnosis of archive issues almost impossible.

[github-actions]

No blocking bugs jumped out from this diff. The refactoring is large but the new coroutine‑based importer, async archiving changes, and option plumbing all look internally consistent, and I didn’t spot a correctness regression in the code paths I inspected.

Residual risks / things to double‑check:

• validator/import-db-slice-local.cpp:551 – apply_block_async_2 explicitly states that add_handle must be performed in order for LTD index updates to stay monotonic, but the current implementation launches every Db::add_handle_to_archive through td::actor::all with no serialization. If the actor scheduler ever reorders those asks, older seqnos could be treated as duplicates and skipped inside ArchiveSlice::add_handle.
• validator/import-db-slice-local.cpp pipeline still calls ValidatorManager::new_block, which unconditionally runs Db::archive. Because the importer now pre‑archives each handle via Db::add_handle_to_archive, BlockArchiver still replays the entire proof/data copying step, defeating the “Optimize importing archive packages” goal and potentially doubling the I/O for each block. Consider short‑circuiting ValidatorManager::new_block when the handle is already archived to avoid undoing the optimization.