refactor: one zero package #625

0xaatif · 2024-09-13T10:57:19Z

As a repo we've often reached for creating a new crate when a module or even a single rust file would do.
Let's start to roll that complecting back.
This unlocks further refactoring of common code, and e.g moving trace_decoder out of a crate.
Eventually I want just a slim cargo build-all-features workflow

…arithmetization/benches/fibonacci_25m_gas.rs evm_arithmetization/benches/stack_manipulation.rs evm_arithmetization/src/all_stark.rs evm_arithmetization/src/arithmetic/addcy.rs evm_arithmetization/src/arithmetic/arithmetic_stark.rs evm_arithmetization/src/arithmetic/byte.rs evm_arithmetization/src/arithmetic/columns.rs evm_arithmetization/src/arithmetic/divmod.rs evm_arithmetization/src/arithmetic/mod.rs evm_arithmetization/src/arithmetic/modular.rs evm_arithmetization/src/arithmetic/mul.rs evm_arithmetization/src/arithmetic/shift.rs evm_arithmetization/src/arithmetic/utils.rs evm_arithmetization/src/bin/assemble.rs evm_arithmetization/src/byte_packing/byte_packing_stark.rs evm_arithmetization/src/byte_packing/columns.rs evm_arithmetization/src/byte_packing/mod.rs evm_arithmetization/src/cpu/byte_unpacking.rs evm_arithmetization/src/cpu/clock.rs evm_arithmetization/src/cpu/columns/general.rs evm_arithmetization/src/cpu/columns/mod.rs evm_arithmetization/src/cpu/columns/ops.rs evm_arithmetization/src/cpu/contextops.rs evm_arithmetization/src/cpu/control_flow.rs evm_arithmetization/src/cpu/cpu_stark.rs evm_arithmetization/src/cpu/decode.rs evm_arithmetization/src/cpu/dup_swap.rs evm_arithmetization/src/cpu/gas.rs evm_arithmetization/src/cpu/halt.rs evm_arithmetization/src/cpu/jumps.rs evm_arithmetization/src/cpu/kernel/aggregator.rs evm_arithmetization/src/cpu/kernel/assembler.rs evm_arithmetization/src/cpu/kernel/ast.rs evm_arithmetization/src/cpu/kernel/constants/context_metadata.rs evm_arithmetization/src/cpu/kernel/constants/exc_bitfields.rs evm_arithmetization/src/cpu/kernel/constants/global_metadata.rs evm_arithmetization/src/cpu/kernel/constants/journal_entry.rs evm_arithmetization/src/cpu/kernel/constants/mod.rs evm_arithmetization/src/cpu/kernel/constants/trie_type.rs evm_arithmetization/src/cpu/kernel/constants/txn_fields.rs evm_arithmetization/src/cpu/kernel/cost_estimator.rs evm_arithmetization/src/cpu/kernel/interpreter.rs evm_arithmetization/src/cpu/kernel/keccak_util.rs evm_arithmetization/src/cpu/kernel/mod.rs evm_arithmetization/src/cpu/kernel/opcodes.rs evm_arithmetization/src/cpu/kernel/optimizer.rs evm_arithmetization/src/cpu/kernel/parser.rs evm_arithmetization/src/cpu/kernel/stack/mod.rs evm_arithmetization/src/cpu/kernel/stack/permutations.rs evm_arithmetization/src/cpu/kernel/stack/stack_manipulation.rs evm_arithmetization/src/cpu/kernel/tests/account_code.rs evm_arithmetization/src/cpu/kernel/tests/add11.rs evm_arithmetization/src/cpu/kernel/tests/balance.rs evm_arithmetization/src/cpu/kernel/tests/bignum/mod.rs evm_arithmetization/src/cpu/kernel/tests/blake2_f.rs evm_arithmetization/src/cpu/kernel/tests/blobhash.rs evm_arithmetization/src/cpu/kernel/tests/block_hash.rs evm_arithmetization/src/cpu/kernel/tests/bls381.rs evm_arithmetization/src/cpu/kernel/tests/bn254.rs evm_arithmetization/src/cpu/kernel/tests/core/access_lists.rs evm_arithmetization/src/cpu/kernel/tests/core/create_addresses.rs evm_arithmetization/src/cpu/kernel/tests/core/intrinsic_gas.rs evm_arithmetization/src/cpu/kernel/tests/core/jumpdest_analysis.rs evm_arithmetization/src/cpu/kernel/tests/core/mod.rs evm_arithmetization/src/cpu/kernel/tests/ecc/curve_ops.rs evm_arithmetization/src/cpu/kernel/tests/ecc/ecrecover.rs evm_arithmetization/src/cpu/kernel/tests/ecc/mod.rs evm_arithmetization/src/cpu/kernel/tests/exp.rs evm_arithmetization/src/cpu/kernel/tests/hash.rs evm_arithmetization/src/cpu/kernel/tests/init_exc_stop.rs evm_arithmetization/src/cpu/kernel/tests/kernel_consistency.rs evm_arithmetization/src/cpu/kernel/tests/log.rs evm_arithmetization/src/cpu/kernel/tests/mcopy.rs evm_arithmetization/src/cpu/kernel/tests/mod.rs evm_arithmetization/src/cpu/kernel/tests/mpt/delete.rs evm_arithmetization/src/cpu/kernel/tests/mpt/hash.rs evm_arithmetization/src/cpu/kernel/tests/mpt/hex_prefix.rs evm_arithmetization/src/cpu/kernel/tests/mpt/insert.rs evm_arithmetization/src/cpu/kernel/tests/mpt/linked_list.rs evm_arithmetization/src/cpu/kernel/tests/mpt/load.rs evm_arithmetization/src/cpu/kernel/tests/mpt/mod.rs evm_arithmetization/src/cpu/kernel/tests/mpt/read.rs evm_arithmetization/src/cpu/kernel/tests/packing.rs evm_arithmetization/src/cpu/kernel/tests/receipt.rs evm_arithmetization/src/cpu/kernel/tests/rlp/decode.rs evm_arithmetization/src/cpu/kernel/tests/rlp/encode.rs evm_arithmetization/src/cpu/kernel/tests/rlp/mod.rs evm_arithmetization/src/cpu/kernel/tests/rlp/num_bytes.rs evm_arithmetization/src/cpu/kernel/tests/signed_syscalls.rs evm_arithmetization/src/cpu/kernel/tests/transaction_parsing/mod.rs evm_arithmetization/src/cpu/kernel/tests/transaction_parsing/parse_type_0_txn.rs evm_arithmetization/src/cpu/kernel/tests/transaction_parsing/parse_type_1_txn.rs evm_arithmetization/src/cpu/kernel/tests/transaction_parsing/parse_type_2_txn.rs evm_arithmetization/src/cpu/kernel/tests/transaction_parsing/parse_type_3_txn.rs evm_arithmetization/src/cpu/kernel/tests/transient_storage.rs evm_arithmetization/src/cpu/kernel/utils.rs evm_arithmetization/src/cpu/membus.rs evm_arithmetization/src/cpu/memio.rs evm_arithmetization/src/cpu/mod.rs evm_arithmetization/src/cpu/modfp254.rs evm_arithmetization/src/cpu/pc.rs evm_arithmetization/src/cpu/push0.rs evm_arithmetization/src/cpu/shift.rs evm_arithmetization/src/cpu/simple_logic/eq_iszero.rs evm_arithmetization/src/cpu/simple_logic/mod.rs evm_arithmetization/src/cpu/simple_logic/not.rs evm_arithmetization/src/cpu/stack.rs evm_arithmetization/src/cpu/syscalls_exceptions.rs evm_arithmetization/src/curve_pairings.rs evm_arithmetization/src/extension_tower.rs evm_arithmetization/src/fixed_recursive_verifier.rs evm_arithmetization/src/generation/linked_list.rs evm_arithmetization/src/generation/mod.rs evm_arithmetization/src/generation/mpt.rs evm_arithmetization/src/generation/prover_input.rs evm_arithmetization/src/generation/rlp.rs evm_arithmetization/src/generation/segments.rs evm_arithmetization/src/generation/state.rs evm_arithmetization/src/generation/trie_extractor.rs evm_arithmetization/src/get_challenges.rs evm_arithmetization/src/keccak/columns.rs evm_arithmetization/src/keccak/constants.rs evm_arithmetization/src/keccak/keccak_stark.rs evm_arithmetization/src/keccak/logic.rs evm_arithmetization/src/keccak/mod.rs evm_arithmetization/src/keccak/round_flags.rs evm_arithmetization/src/keccak_sponge/columns.rs evm_arithmetization/src/keccak_sponge/keccak_sponge_stark.rs evm_arithmetization/src/keccak_sponge/mod.rs evm_arithmetization/src/lib.rs evm_arithmetization/src/logic.rs evm_arithmetization/src/memory/columns.rs evm_arithmetization/src/memory/memory_stark.rs evm_arithmetization/src/memory/mod.rs evm_arithmetization/src/memory/segments.rs evm_arithmetization/src/memory_continuation/columns.rs evm_arithmetization/src/memory_continuation/memory_continuation_stark.rs evm_arithmetization/src/memory_continuation/mod.rs evm_arithmetization/src/poseidon/columns.rs evm_arithmetization/src/poseidon/mod.rs evm_arithmetization/src/poseidon/poseidon_stark.rs evm_arithmetization/src/proof.rs evm_arithmetization/src/prover.rs evm_arithmetization/src/recursive_verifier.rs evm_arithmetization/src/testing_utils.rs evm_arithmetization/src/util.rs evm_arithmetization/src/verifier.rs evm_arithmetization/src/witness/errors.rs evm_arithmetization/src/witness/gas.rs evm_arithmetization/src/witness/memory.rs evm_arithmetization/src/witness/mod.rs evm_arithmetization/src/witness/operation.rs evm_arithmetization/src/witness/state.rs evm_arithmetization/src/witness/traces.rs evm_arithmetization/src/witness/transition.rs evm_arithmetization/src/witness/util.rs evm_arithmetization/tests/add11_yml.rs evm_arithmetization/tests/erc20.rs evm_arithmetization/tests/erc721.rs evm_arithmetization/tests/global_exit_root.rs evm_arithmetization/tests/log_opcode.rs evm_arithmetization/tests/selfdestruct.rs evm_arithmetization/tests/simple_transfer.rs evm_arithmetization/tests/two_to_one_block.rs evm_arithmetization/tests/withdrawals.rs mpt_trie/examples/ethereum_trie.rs mpt_trie/examples/hash_nodes.rs mpt_trie/examples/simple.rs mpt_trie/src/builder.rs mpt_trie/src/debug_tools/diff.rs mpt_trie/src/debug_tools/mod.rs mpt_trie/src/debug_tools/query.rs mpt_trie/src/debug_tools/stats.rs mpt_trie/src/lib.rs mpt_trie/src/nibbles.rs mpt_trie/src/partial_trie.rs mpt_trie/src/special_query.rs mpt_trie/src/testing_utils.rs mpt_trie/src/trie_hashing.rs mpt_trie/src/trie_ops.rs mpt_trie/src/trie_subsets.rs mpt_trie/src/utils.rs proc_macro/src/common.rs proc_macro/src/impls/columns.rs proc_macro/src/impls/deref_columns.rs proc_macro/src/impls/mod.rs proc_macro/src/lib.rs proc_macro/tests/compile/mod.rs proc_macro/tests/compile/test_impls.rs proc_macro/tests/compile_fail/not_repr_c.rs proc_macro/tests/compile_fail/not_repr_c_deref.rs proc_macro/tests/compile_fail/not_struct.rs proc_macro/tests/compile_fail/not_struct_deref.rs proc_macro/tests/compiletest.rs proof_gen/src/constants.rs proof_gen/src/lib.rs proof_gen/src/proof_gen.rs proof_gen/src/proof_types.rs proof_gen/src/prover_state.rs proof_gen/src/types.rs proof_gen/src/verifier_state.rs smt_trie/src/bits.rs smt_trie/src/code.rs smt_trie/src/db.rs smt_trie/src/keys.rs smt_trie/src/lib.rs smt_trie/src/smt.rs smt_trie/src/smt_test.rs smt_trie/src/utils.rs trace_decoder/benches/block_processing.rs trace_decoder/src/core.rs trace_decoder/src/interface.rs trace_decoder/src/lib.rs trace_decoder/src/type1.rs trace_decoder/src/type2.rs trace_decoder/src/typed_mpt.rs trace_decoder/src/wire.rs trace_decoder/tests/common/mod.rs trace_decoder/tests/consistent-with-header.rs trace_decoder/tests/simulate-execution.rs zero/build.rs zero/src/bin/leader.rs zero/src/bin/leader/cli.rs zero/src/bin/leader/client.rs zero/src/bin/leader/http.rs zero/src/bin/leader/stdio.rs zero/src/bin/rpc.rs zero/src/bin/verifier.rs zero/src/bin/verifier/cli.rs zero/src/bin/verifier/init.rs zero/src/bin/worker.rs zero/src/block_interval.rs zero/src/debug_utils.rs zero/src/env.rs zero/src/fs.rs zero/src/lib.rs zero/src/ops.rs zero/src/parsing.rs zero/src/pre_checks.rs zero/src/prover.rs zero/src/prover/cli.rs zero/src/prover_state/circuit.rs zero/src/prover_state/cli.rs zero/src/prover_state/mod.rs zero/src/prover_state/persistence.rs zero/src/provider.rs zero/src/rpc/jerigon.rs zero/src/rpc/mod.rs zero/src/rpc/native/mod.rs zero/src/rpc/native/state.rs zero/src/rpc/native/txn.rs zero/src/rpc/retry.rs zero/src/tracing.rs zero/src/version.rs **/*.yml zero/tools/prove_rpc.sh zero/tools/prove_stdio.sh

Nashtare

Maybe it'd be worth having an actual discussion with everyone on the project before spending time and ressources in such work. I don't think we're all aligned in the "1 crate is better" vision

atanmarko · 2024-09-13T15:17:50Z

Maybe it'd be worth having an actual discussion with everyone on the project before spending time and ressources in such work. I don't think we're all aligned in the "1 crate is better" vision

I haven't looked at the PR yet, but we had discussion about this topic on previous rust sync meeting. I agree with Aatif that we went to the extreme with creating a new crate for every zero-bin module - that includes problems with many inter-dependencies multiplied by features, publishing all crates at once etc.

0xaatif · 2024-09-16T16:42:10Z

@Nashtare @muursh I would appreciate speedy reviews for this PR - such a sweeping change is liable to accumulating conflicts.

If there are PRs which you believe should come before this one, let me know and I can rebase ahead of time.

Nashtare · 2024-09-16T16:44:04Z

.github/labeler.yml

+# Add 'crate: zero_bin' label to any changes within 'zero' folder.
 'crate: zero_bin':


nit: we may want to rename the lint to crate: zero as well?

I kept this the same for convenience (e.g looking at historical issues). Happy to change :)

.gitignore

trace_decoder/Cargo.toml

trace_decoder/src/lib.rs

muursh

No qualms

0xaatif added 4 commits September 13, 2024 00:46

mark: 0xaatif/fewer-crates

eb593c7

refactor: remove ops

36d4798

refactor: remove prover

4361949

refactor: remove worker

67fcf23

github-actions bot added crate: trace_decoder Anything related to the trace_decoder crate. crate: proof_gen Anything related to the proof_gen crate. crate: zero_bin Anything related to the zero-bin subcrates. labels Sep 13, 2024

0xaatif changed the title ~~refactor: fewer crates~~ refactor: collapse the zero_bin crates Sep 13, 2024

0xaatif added 3 commits September 13, 2024 12:01

chore: update CI

9e7ac1a

chore: update doc

19d770f

fix: features are going to ruin my life

7406442

github-actions bot added the ci label Sep 13, 2024

chore: remove features from trace_decoder

9e59fbb

0xaatif changed the title ~~refactor: collapse the zero_bin crates~~ refactor: collapse the zero_bin crates, vol 1 Sep 13, 2024

0xaatif added 4 commits September 13, 2024 12:40

refactor: remvoe verifier

5d4f457

refactor: remove rpc

60cd1cd

wip: move leader

16f11f3

refactor: remove leader

6dfca34

0xaatif force-pushed the 0xaatif/fewer-crates branch from 708144d to 6dfca34 Compare September 13, 2024 12:10

0xaatif added 5 commits September 13, 2024 15:09

chore: ci

adef43c

fix: clippy on zero_bin_common

5a004f3

wip: zero_bin_common -> zero

d218d36

chore: ci

02f044d

Nashtare reviewed Sep 13, 2024

View reviewed changes

0xaatif changed the title ~~refactor: collapse the zero_bin crates, vol 1~~ refactor: collapse the zero_bin crates Sep 16, 2024

0xaatif changed the title ~~refactor: collapse the zero_bin crates~~ refactor: one zero package Sep 16, 2024

0xaatif added 2 commits September 16, 2024 14:28

chore: docs

1ba78f7

chore: move zero_bin/tools

685a833

0xaatif and others added 4 commits September 16, 2024 15:30

wip

0c1b225

chore: ci

f0036f4

chore: more ci

ee3c8bb

chore: more gitignore

ddeeeef

0xaatif self-assigned this Sep 16, 2024

Merge remote-tracking branch 'origin/develop' into 0xaatif/fewer-crates

82a0865

0xaatif force-pushed the 0xaatif/fewer-crates branch from 227b551 to 82a0865 Compare September 16, 2024 15:55

0xaatif added 2 commits September 16, 2024 17:02

run: cargo fmt

0805482

chore: fix exe paths

bd8c3b2

0xaatif marked this pull request as ready for review September 16, 2024 16:20

0xaatif requested review from atanmarko and muursh as code owners September 16, 2024 16:20

0xaatif mentioned this pull request Sep 16, 2024

Aatif's Personal Hit List #270

Open

18 tasks

Nashtare reviewed Sep 16, 2024

View reviewed changes

review: markups

8a3f56b

0xaatif force-pushed the 0xaatif/fewer-crates branch from 0898fc1 to 8a3f56b Compare September 16, 2024 16:53

Nashtare approved these changes Sep 16, 2024

View reviewed changes

trace_decoder/src/lib.rs Show resolved Hide resolved

muursh approved these changes Sep 16, 2024

View reviewed changes

0xaatif merged commit 6932ab3 into develop Sep 16, 2024
18 checks passed

0xaatif deleted the 0xaatif/fewer-crates branch September 16, 2024 18:35

Nashtare added this to the System strengthening milestone Sep 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: one zero package #625

refactor: one zero package #625

0xaatif commented Sep 13, 2024 •

edited

Loading

Nashtare left a comment

atanmarko commented Sep 13, 2024 •

edited

Loading

0xaatif commented Sep 16, 2024

Nashtare Sep 16, 2024

0xaatif Sep 16, 2024

muursh left a comment

		# Add 'crate: zero_bin' label to any changes within 'zero' folder.
		'crate: zero_bin':

refactor: one zero package #625

refactor: one zero package #625

Conversation

0xaatif commented Sep 13, 2024 • edited Loading

Nashtare left a comment

Choose a reason for hiding this comment

atanmarko commented Sep 13, 2024 • edited Loading

0xaatif commented Sep 16, 2024

Nashtare Sep 16, 2024

Choose a reason for hiding this comment

0xaatif Sep 16, 2024

Choose a reason for hiding this comment

muursh left a comment

Choose a reason for hiding this comment

0xaatif commented Sep 13, 2024 •

edited

Loading

atanmarko commented Sep 13, 2024 •

edited

Loading