feature wait 50 blocks to process Ethereum events #338

juniuszhou · 2020-12-02T23:34:48Z

For Ethereum, the block not finalized and the reorg happened sometimes. After ebrelayer received an envent from Ethereum, we put it on the event queue, process them one by one after 30 blocks produced.

cmd/ebrelayer/relayer/ethereum.go

docs/adr-002-ebrelayer-eth-subscribe.md

ElliotFriedman · 2020-12-05T00:29:36Z

Testing locally, ceth going across the bridge works perfectly! Will now test wiring up erowan and making sure it gets minted on the cosmos side as rowan after 30 blocks.

cmd/ebrelayer/relayer/ethereum.go

ElliotFriedman · 2020-12-07T19:26:09Z

cmd/ebrelayer/relayer/ethereum.go

+
+			// Iterate find out older enough events
+			for key, value := range sub.EventsBuffer.Buffer {
+				sub.Logger.Info(fmt.Sprintf("New header is %d stored header is %d", newHead.Number, key))


I just realized that we have a logic error here. This picks up events and stores them in the q, but it doesn't check that this event is still valid and that TX did not end up getting dis included in a block because the block was orphaned, which was the problem we are trying to solve.

Here, we will need to check on the transaction that was included in this event and make sure that it is in fact in a block that is 30 blocks ago, if it is not in a previous block, we delete it from the q, if it got included in a later block, we add it again to the queue.

I believe the we can check it via code logic before. If we get the event, which block hash not the same with the buffered one. Then we removed all events with old block hash.
if ok && events.BlockHash != newHead.Hash() {
// Deal with block reorg, delete all events with old hash value
sub.Logger.Info(fmt.Sprintf("Block reorg found old hash is %v new hash is %v", events.BlockHash, newHead.Hash()))
delete(sub.EventsBuffer.Buffer, newHead.Number)
}

That solution sounds good to me.

We have to somehow query to check if this old block was valid, if it is invalid, like you said, we throw all of the old stuff out.

…hain/sifnode into feature/wait-blocks-confirm

ElliotFriedman · 2020-12-11T19:24:35Z

smart-contracts/scripts/sendTransfer.js

+  const web3 = new Web3(provider);
+  try {
+    const accounts = await web3.eth.getAccounts();
+    await web3.eth.sendTransaction({from: accounts[8], to: accounts[9], value: 1})


Could we please loop over this 30 times in this file vs having the python file trigger this 30 times? It should make things a lot faster as truffle exec has a bunch of overhead of spinning up the compiler and checking on the artifacts and then connecting to the network.

This loop over 30 times added into python script.

I think we need this the other way around - there should be a yarn command that generates N blocks. Like @ElliotFriedman said, doing this in python is really slow, but we also want to be able to do it in shell scripts.

@juniuszhou Please add a loop inside of the javascript file to make sendTransfer.js mine 30 blocks and remove the loop in the python file. This way things will execute much faster when tests are run.

You can use the advance block function from openzeppelin to advance blocks https://forum.openzeppelin.com/t/how-to-simulate-a-date-in-the-future-with-truffle-tests/908/3

https://docs.openzeppelin.com/test-helpers/0.5/api#advanceblock

You can use the advance block function from openzeppelin to advance blocks https://forum.openzeppelin.com/t/how-to-simulate-a-date-in-the-future-with-truffle-tests/908/3

https://docs.openzeppelin.com/test-helpers/0.5/api#advanceblock

Does this bignumber issue matter to us? OpenZeppelin/openzeppelin-test-helpers#70

banshee

I think this is one case where we really do want to improve the integration tests before we merge this. We should definitely have tests that check the balances before and after 30 blocks, for example.

banshee · 2020-12-16T02:15:25Z

smart-contracts/scripts/sendTransfer.js

+  const web3 = new Web3(provider);
+  try {
+    const accounts = await web3.eth.getAccounts();
+    await web3.eth.sendTransaction({from: accounts[8], to: accounts[9], value: 1})


I think we need this the other way around - there should be a yarn command that generates N blocks. Like @ElliotFriedman said, doing this in python is really slow, but we also want to be able to do it in shell scripts.

test/integration/start-integration-env.sh

ElliotFriedman · 2020-12-16T18:12:18Z

I'm not seeing anything here that looks incorrect to me. I talked with the peggy althea team, they use 50 blocks before allowing transactions through.

I think we should bump up to 50 blocks waited for security reasons.

Before I run a final test against this PR, please fix all of the integration tests and then tag me so I can run a final test before signing off on it.

cmd/ebrelayer/relayer/ethereum.go

ElliotFriedman · 2020-12-16T18:34:40Z

Apparently we can check the status of an event that has previously mined. I would prefer our implementation uses this method by querying the event and tx hash instead of the recursive check that we currently use. https://ethereum.stackexchange.com/questions/68012/is-it-possible-to-get-duplicates-of-the-log-events-on-ethereum-when-we-have-a-fo

…t.py from develop (commit bb368ec) We're going to do block advancement a different way

…node into feature/wait-blocks-confirm

ElliotFriedman · 2020-12-18T23:59:53Z

cmd/ebrelayer/relayer/ethereum.go

+			sub.EventsBuffer.AddHeader(newHead.Number, newHead.Hash(), newHead.ParentHash)
+
+			for {
+				thirty := big.NewInt(30)


This needs to be 50 blocks per the new requirement.

* develop: [Frontend] Apply testing to CLP API. Apply fixes for breaking changes. (#409) Fix/peggy docs (#406)

…ure/wait-blocks-confirm * 'develop' of ssh://github.com/Sifchain/sifnode: Integration scripts now do all transfers in python and have better waiting (#410)

…ure/permissioned-validators * feature/test-whitelist-validators: Add support for calling sifnoded add-genesis-validators to integration tests [Frontend] Feature/Notifications (#389) fix issues 7 and 8 auditors found (#411) chains/ethereum: yarn add truffle (incl for local) (#413) feature wait 50 blocks to process Ethereum events (#338) Integration scripts now do all transfers in python and have better waiting (#410) [Frontend] Apply testing to CLP API. Apply fixes for breaking changes. (#409) Fix/peggy docs (#406) Tests for block delay (#407) Smart Contract Clean up #1, silence test warnings, fix smart contract issues (#403) Extract assets out of config (#390) Fix to format the tag name. (#400) added asset details to query liquidity provider response (#398) [Peggy] Test improvments and add test for ceth => eth (#396) Clp tut (#395) Workflow fixes. (#399) [ChainOps] feature/k8s-tutorial (#397)

* create event queue for delay send tx to sifchain. * set delay blocks as 30. * add adr file to describe ebrelayer. * update account according to new init.sh * update docs * new algorithm to handle orphan block. * fix linter issue. * fix two errors. * add scripts to generate 30 new blocks. * print out balance for check. * print out balance for check. * remove the eth lock in integration env, it impact the balance of ceth. * make the transfer script send transaction according to input. * add sleep time to make sure new block produced. * Pull in peggy-basic-test-docker.py and test/integration/peggy-e2e-test.py from develop (commit bb368ec) We're going to do block advancement a different way * remove send transfer, create advanceBlock script * fix the conflict. * Don't wrap block advance test in an exception handler * Require 50 block delay instead of 30 * update docs Co-authored-by: Elliot Friedman <elliotfriedman3@gmail.com> Co-authored-by: utx0_ <90531+utx0@users.noreply.github.com> Co-authored-by: Elliot <34463580+ElliotFriedman@users.noreply.github.com> Co-authored-by: James Moore <james@sifchain.finance>

juniuszhou added 2 commits December 2, 2020 11:36

create event queue for delay send tx to sifchain.

158c3e8

set delay blocks as 30.

6bd3ad5

juniuszhou requested review from utx0, ilyabukalov and ElliotFriedman December 2, 2020 23:39

ElliotFriedman suggested changes Dec 3, 2020

View reviewed changes

cmd/ebrelayer/relayer/ethereum.go Outdated Show resolved Hide resolved

juniuszhou added 2 commits December 4, 2020 08:25

add adr file to describe ebrelayer.

7dff3d6

update account according to new init.sh

b826e67

ElliotFriedman suggested changes Dec 4, 2020

View reviewed changes

docs/adr-002-ebrelayer-eth-subscribe.md Outdated Show resolved Hide resolved

ElliotFriedman previously approved these changes Dec 5, 2020

View reviewed changes

ElliotFriedman added 2 commits December 4, 2020 21:32

Merge branch 'develop' into feature/wait-blocks-confirm

9281f0e

update docs

97d4378

ElliotFriedman dismissed their stale review via 97d4378 December 5, 2020 02:35

ElliotFriedman suggested changes Dec 5, 2020

View reviewed changes

cmd/ebrelayer/relayer/ethereum.go Show resolved Hide resolved

utx0 added the in progress In progress. label Dec 7, 2020

ElliotFriedman suggested changes Dec 7, 2020

View reviewed changes

utx0 and others added 10 commits December 9, 2020 15:18

Merge branch 'develop' into feature/wait-blocks-confirm

e41df80

new algorithm to handle orphan block.

8092120

Merge branch 'feature/wait-blocks-confirm' of https://github.com/sifc…

0fc75c8

…hain/sifnode into feature/wait-blocks-confirm

fix linter issue.

bedfd69

Merge branch 'develop' into feature/wait-blocks-confirm

61e53fc

fix two errors.

67d0a7d

add scripts to generate 30 new blocks.

619bdcf

print out balance for check.

7143e5e

print out balance for check.

47a5bd6

Merge branch 'develop' into feature/wait-blocks-confirm

cd109ae

ElliotFriedman suggested changes Dec 11, 2020

View reviewed changes

ElliotFriedman added 2 commits December 13, 2020 21:30

Merge branch 'develop' into feature/wait-blocks-confirm

b149e54

Merge branch 'develop' into feature/wait-blocks-confirm

38d510e

remove the eth lock in integration env, it impact the balance of ceth.

f9c37a1

juniuszhou requested a review from ElliotFriedman December 16, 2020 01:35

banshee suggested changes Dec 16, 2020

View reviewed changes

make the transfer script send transaction according to input.

990fe7c

banshee suggested changes Dec 16, 2020

View reviewed changes

cmd/ebrelayer/relayer/ethereum.go Outdated Show resolved Hide resolved

juniuszhou and others added 6 commits December 17, 2020 10:25

add sleep time to make sure new block produced.

616a4e3

Pull in peggy-basic-test-docker.py and test/integration/peggy-e2e-tes…

8087b40

…t.py from develop (commit bb368ec) We're going to do block advancement a different way

remove send transfer, create advanceBlock script

85a5388

Merge branch 'feature/wait-blocks-confirm' of github.com:Sifchain/sif…

6524980

…node into feature/wait-blocks-confirm

rebase pr with develop branch.

065d83c

fix the conflict.

0cc027b

ElliotFriedman suggested changes Dec 18, 2020

View reviewed changes

banshee and others added 5 commits December 18, 2020 16:18

Merge branch 'develop' into feature/wait-blocks-confirm

0c4791c

* develop: [Frontend] Apply testing to CLP API. Apply fixes for breaking changes. (#409) Fix/peggy docs (#406)

Merge branch 'develop' of ssh://github.com/Sifchain/sifnode into feat…

29ee90f

…ure/wait-blocks-confirm * 'develop' of ssh://github.com/Sifchain/sifnode: Integration scripts now do all transfers in python and have better waiting (#410)

Don't wrap block advance test in an exception handler

90cfcaf

Require 50 block delay instead of 30

c42372d

update docs

c080913

ElliotFriedman changed the title ~~feature wait 30 blocks to process Ethereum events~~ feature wait 50 blocks to process Ethereum events Dec 19, 2020

ElliotFriedman approved these changes Dec 19, 2020

View reviewed changes

banshee approved these changes Dec 19, 2020

View reviewed changes

ElliotFriedman merged commit 7349f7b into develop Dec 19, 2020

juniuszhou deleted the feature/wait-blocks-confirm branch April 6, 2021 06:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature wait 50 blocks to process Ethereum events #338

feature wait 50 blocks to process Ethereum events #338

juniuszhou commented Dec 2, 2020

ElliotFriedman commented Dec 5, 2020

ElliotFriedman Dec 7, 2020

juniuszhou Dec 7, 2020

ElliotFriedman Dec 8, 2020

ElliotFriedman Dec 8, 2020

ElliotFriedman Dec 11, 2020

juniuszhou Dec 16, 2020

banshee Dec 16, 2020

ElliotFriedman Dec 16, 2020

ElliotFriedman Dec 16, 2020

banshee Dec 16, 2020 •

edited

Loading

banshee left a comment

banshee Dec 16, 2020

ElliotFriedman commented Dec 16, 2020

ElliotFriedman commented Dec 16, 2020

ElliotFriedman Dec 18, 2020

feature wait 50 blocks to process Ethereum events #338

feature wait 50 blocks to process Ethereum events #338

Conversation

juniuszhou commented Dec 2, 2020

ElliotFriedman commented Dec 5, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

banshee Dec 16, 2020 • edited Loading

Choose a reason for hiding this comment

banshee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ElliotFriedman commented Dec 16, 2020

ElliotFriedman commented Dec 16, 2020

Choose a reason for hiding this comment

banshee Dec 16, 2020 •

edited

Loading