Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Now prunes tries properly (and more aggressively!) without hack #97

Merged
merged 10 commits into from
Mar 11, 2024

Conversation

BGluth
Copy link
Contributor

@BGluth BGluth commented Mar 8, 2024

This PR is the last piece to tie together all of the pruning optimizations/fixes that have been going on over the past month. I ended up having to cherry-pick from a testing branch, so apologies if this is a bit messy (eg. Requested PR changes for #39 somehow never got merged and was on the tip of another in progress branch that I needed, but I'm just going to leave that here for now).

The critical bits in this PR are:

  • Logic in the decoder to handle the branch collapse edge case properly.
  • Fix in creating trie-subsets with when to hash out extension nodes.
  • The order of calls in into_txn_proof_gen_ir has changed in order to use delta_out (just contains info on collapsed nodes) when creating partial tries. I'm going to admit upfront that this section got a lot more complex and will get a nice refactor pretty soon in an upcoming PR.
  • Logic to initialize empty storage tries finally got moved outside of apply_deltas_to_trie_state. It really had no reason to be in here.

Almost all blocks that I've tested are not able to generate proofs. I was hoping to say all, but 19240780 currently fails.

BGluth added 7 commits March 8, 2024 13:58
- However it will only not hash the child if the key matches the key
  piece of the extension.
- Fixes overall deal with calling the new pruning query in `mpt_trie`
  and changing the order of calls when generating proof IR.
- Realized that we needed a bit more logic to get the key to the
  remaining child after a branch collapse.
- Ended up adding a bool to `TriePathIter`.
- Also includes one core fix that got left out.
@BGluth BGluth requested review from muursh and Nashtare as code owners March 8, 2024 21:55
@github-actions github-actions bot added crate: trace_decoder Anything related to the trace_decoder crate. crate: mpt_trie Anything related to the mpt_trie crate. labels Mar 8, 2024
@muursh
Copy link
Contributor

muursh commented Mar 9, 2024

Almost all blocks that I've tested are not able to generate proofs.

Not able?

Copy link
Collaborator

@Nashtare Nashtare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Brendan, overall LGTM minus a few nits. Also if you could add an entry to the CHANGELOG, that'd be great!

@@ -26,6 +26,11 @@ pub type Nibble = u8;
/// Used for the internal representation of a sequence of nibbles.
pub type NibblesIntern = U512;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed it now, but why do we need U512? For the 65-Nibbles edge case? We probably would want to refactor it at some point, seems a bit wasteful to leverage U512 everywhere here (though out of scope for this PR).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah exactly. It's for the 65 Nibble edge case.

I don't think I have a Github issue open for this (there was one in Jira), so I'll create one. Also worth having a discussion on the best way to implement this. If we care about performance, using a byte array might perform worse. I know H256 uses a [u8; 32], but U256s use [u64; 4]. If I had to guess, since H256 don't support common operators (eg. +, -), they probably found that working with u64s allowed them to implement a lot of operations with 64-bit instructions (vs. doing a bunch of operations to individual bytes). Probably worth talking to @dvdplm about this, as IIRC he worked on ethereum-types a good amount.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created an issue for this (#104).

@Nashtare Nashtare linked an issue Mar 11, 2024 that may be closed by this pull request
@pgebheim pgebheim added this to the Type 1 - Q1 2024 milestone Mar 11, 2024
Robin's suggested changes for PR #97

Co-authored-by: Robin Salen <30937548+Nashtare@users.noreply.github.com>
@BGluth
Copy link
Contributor Author

BGluth commented Mar 11, 2024

Almost all blocks that I've tested are not able to generate proofs.

Not able?

Sorry just really want to correct myself as I accidentally negated myself here. 😅 There's just one block that does not work so far (out of ~10).

@BGluth BGluth enabled auto-merge (squash) March 11, 2024 19:39
@BGluth BGluth merged commit 5c8d516 into main Mar 11, 2024
6 checks passed
@BGluth BGluth deleted the aggressive_trie_pruning_finish branch March 11, 2024 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crate: mpt_trie Anything related to the mpt_trie crate. crate: trace_decoder Anything related to the trace_decoder crate.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Address over-aggressive sub-trie pruning
4 participants