Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: balances and commitment mapper improvements #286

Closed

Conversation

PetarKirov
Copy link
Collaborator

No description provided.

PetarKirov and others added 30 commits February 19, 2024 19:59
@Xearty Xearty force-pushed the feat/balances_and_commitment_mapper_improvements branch from 07727e3 to 8f2cdec Compare February 29, 2024 12:50
feat(plonky2): Make commitment mapper redis serialization hex

non-working stuff

more changes

got the commitment mapper inner proofs working

Use reduce

Rename addInnerProofs to updateBranches and don't call it for every batch

Delete an obsolete branch

change order of task parameters

continually process the head slot's epoch

Implement tasks fetching data for old epochs

Add a script for pruning data for old epochs

Add search merkle path extractor script & store data in Redis as a json data type instead of string. Fix commitment mapper zero proofs computation

Added epoch syncing up to head

Delete an unused constant

Fixed spamming zero validator tasks on script start

Added slot as an optional argument for balances script

Delete unused code

Fixed fetching many validators and writing to Redis

Make balance_verification build and make it use json redis data type

Change the indexing scheme for balance verification tree and reduce the
size of the inner tasks

fix balances generation for validator_balacnce_input

fix(commitment_mapper): save leaf validator proofs before calculating them

feat(balance_verification): command-line option to delete intermediary proofs

refactor(balance_verification): delete logging

feat(beacon-api): fetch validators from a different source depending on the requesteded count

feat(scripts): create a common config for default options

perf(commitment_mapper): optimize validator changed predicate

improve(commitment_mapper): logging in get_changed_validators.ts script

feat(config): use common_config.json in the worker

improve(balances): improve logging and disconnect from ioredis instance

refactor(crud): delete unnecessary syntex

refactor(commitment_mapper): remove CommitmentMapperTaskType::None and use options instead

refactor(crud): delete an unused function validator_proof_to_json

refactor(commitment_mapper): refactor, rename and split into multiple files

refactor(crud): delete unused imports

improve(commitment_mapper): improve error logging

fix(balance_verification): fix crash on inner level proofs if lower level not yet completed

improve(balance_verification): improve logging

refactor(): renaming

refactor(commitment_mapper): partially remove dependency on the redis package and move commitment mapper to a separate file

fix(commitment_mapper): wrong hash when odd number of validators

fix(final_layer): patch the final layer proof generation

refactor(commitment_mapper): rename utils.ts to scheduler.ts

refactor(commitment_mapper): no longer implement the dispose interface

feat(commitment_mapper): add incomplete zero tasks to redis and improve logging

refactor: delete obsolete parameter from the commitment_mapper inner
level circuit

feat: offset option

refactor: use just relay/redis in the get_balances_input script

improve: colorful logging

fix(commitment_mapper): save zero validator proof in redis when not calculated

fix: push leased tasks in from of the work queue

fix: refetch proof on each retry

feat: make light clean duration configurable

refactor: less hacky way to query validators and don't fetch 0 validator on startup

fix(redis): waitForConnection

fix: load validators for an epoch parameter wihtout duplicating validators

fix: pass a missing argument

feat(commitment_mapper/balance_verification): light cleaner script that returns tasks to the front of the queue

refactor(commitment_mapper/balance_verification): use ProofStorage

Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
@PetarKirov PetarKirov changed the base branch from main to feat/commitment_mapper_builder_nix_package February 29, 2024 15:15
@PetarKirov PetarKirov force-pushed the feat/balances_and_commitment_mapper_improvements branch from 8f2cdec to 3adea3f Compare February 29, 2024 15:15
@Xearty Xearty force-pushed the feat/balances_and_commitment_mapper_improvements branch from a928737 to 3adea3f Compare February 29, 2024 15:54
@PetarKirov PetarKirov force-pushed the feat/balances_and_commitment_mapper_improvements branch from b34a06e to 97d1fa6 Compare March 1, 2024 14:16
@Xearty Xearty force-pushed the feat/balances_and_commitment_mapper_improvements branch from 97d1fa6 to 574ce0b Compare March 1, 2024 16:18
@PetarKirov PetarKirov force-pushed the feat/commitment_mapper_builder_nix_package branch from 950bede to 7c13894 Compare March 8, 2024 12:41
Xearty and others added 11 commits March 18, 2024 15:25
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
…ommitment mapper old epoch deletion

Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
Co-authored-by: Aneta Tsvetkova <anetastsvetkova@gmail.com>
@Xearty Xearty force-pushed the feat/balances_and_commitment_mapper_improvements branch from 574ce0b to ea0c8cc Compare March 18, 2024 13:27
@PetarKirov PetarKirov force-pushed the feat/commitment_mapper_builder_nix_package branch 6 times, most recently from e188d5e to ee4dc2f Compare March 22, 2024 16:53
Base automatically changed from feat/commitment_mapper_builder_nix_package to main March 22, 2024 21:24
@Xearty Xearty closed this Apr 19, 2024
@Xearty Xearty deleted the feat/balances_and_commitment_mapper_improvements branch April 19, 2024 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants