Shrink persisted fork choice data #7805
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
michaelsproul
added
work-in-progress
dapplion
reviewed
Jul 29, 2025
|
Closes #7760 or we want to go further? |
|
This isn't finished yet, I'm working on the compression now and it gets the size down to 1.9MB total |
dapplion
reviewed
Aug 4, 2025
michaelsproul
added
ready-for-review
|
This pull request has merge conflicts. Could you please resolve them @michaelsproul? 🙏 |
mergify
bot
added
waiting-on-author
mergify
bot
added
ready-for-review
jimmygchen
reviewed
Aug 15, 2025
jimmygchen
reviewed
Aug 15, 2025
jimmygchen
reviewed
Aug 15, 2025
| @@ -162,9 +164,9 @@ where | |||
| /// It is assumed that `anchor` is already persisted in `store`. | |||
| pub fn get_forkchoice_store( | |||
| store: Arc<HotColdDB<E, Hot, Cold>>, | |||
| anchor: &BeaconSnapshot<E>, | |||
| anchor: BeaconSnapshot<E>, | |||
There was a problem hiding this comment.
Do we need ownership here? we're cloning the beacon state below, and i dont see this moved anywhere
jimmygchen
reviewed
Aug 15, 2025
| ) | ||
| .map_err(|e| { | ||
| Error::MigrationError(format!( | ||
| "Error loading fork choise store from persisted: {e:?}" |
jimmygchen
approved these changes
Aug 15, 2025
|
Will leave it to you to merge if you'd like to do more testing or another pair of eyes (maybe @dapplion ?) |
michaelsproul
added
ready-for-merge
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue Addressed
Closes:
Proposed Changes
balances_cachefromPersistedForkChoiceStore(~65 MB saving on mainnet)justified_balancesfromPersistedForkChoiceStore(~16 MB saving on mainnet)balancesfromProtoArray/SszContainer.Testing Results
Total bytes written is way down:
Time spent writing to disk is also way down: