Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(levm): add refunded gas to revm's gas used when writing the ef-test report #2011

Closed
wants to merge 9 commits into from

Conversation

lima-limon-inc
Copy link
Contributor

@lima-limon-inc lima-limon-inc commented Feb 19, 2025

Motivation

Revm subtracts refunded gas from the gas used when it returns a report result. This meant that we were showing gas differences when in reality we were spending the same amount of gas.

Description
Add refunded_gas to gas_used in REVM's transaction result.

@lima-limon-inc lima-limon-inc self-assigned this Feb 19, 2025
@lima-limon-inc lima-limon-inc requested a review from a team as a code owner February 19, 2025 17:09
Copy link

github-actions bot commented Feb 19, 2025

Total lines added: 0
Total lines removed: 0
Total lines changed: 0

@lima-limon-inc lima-limon-inc marked this pull request as draft February 19, 2025 17:23
Copy link

github-actions bot commented Feb 19, 2025

Benchmark Results Comparison

PR Results

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Factorial 238.0 ± 0.8 237.2 239.4 1.00
levm_Factorial 904.1 ± 9.4 890.8 921.0 3.80 ± 0.04

Benchmark Results: Factorial - Recursive

Command Mean [s] Min [s] Max [s] Relative
revm_FactorialRecursive 1.482 ± 0.088 1.369 1.596 1.00
levm_FactorialRecursive 15.704 ± 0.112 15.585 15.879 10.60 ± 0.63

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Fibonacci 211.0 ± 1.2 209.1 213.0 1.00
levm_Fibonacci 894.8 ± 12.1 883.5 919.1 4.24 ± 0.06

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ManyHashes 8.6 ± 0.0 8.6 8.7 1.00
levm_ManyHashes 18.1 ± 0.1 17.9 18.4 2.10 ± 0.02

Benchmark Results: BubbleSort

Command Mean [s] Min [s] Max [s] Relative
revm_BubbleSort 3.256 ± 0.021 3.235 3.301 1.00
levm_BubbleSort 6.093 ± 0.032 6.034 6.143 1.87 ± 0.02

Benchmark Results: ERC20 - Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Transfer 253.4 ± 2.0 250.8 257.1 1.00
levm_ERC20Transfer 537.9 ± 3.6 533.7 546.7 2.12 ± 0.02

Benchmark Results: ERC20 - Mint

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Mint 142.8 ± 1.3 141.2 145.9 1.00
levm_ERC20Mint 353.9 ± 5.9 349.4 364.9 2.48 ± 0.05

Benchmark Results: ERC20 - Approval

Command Mean [s] Min [s] Max [s] Relative
revm_ERC20Approval 1.057 ± 0.011 1.042 1.079 1.00
levm_ERC20Approval 2.031 ± 0.032 1.999 2.118 1.92 ± 0.04

Main Results

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Factorial 238.0 ± 1.2 236.5 239.9 1.00
levm_Factorial 900.7 ± 10.3 890.2 917.1 3.78 ± 0.05

Benchmark Results: Factorial - Recursive

Command Mean [s] Min [s] Max [s] Relative
revm_FactorialRecursive 1.441 ± 0.071 1.371 1.587 1.00
levm_FactorialRecursive 15.652 ± 0.108 15.525 15.796 10.86 ± 0.54

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Fibonacci 210.4 ± 1.1 209.4 212.3 1.00
levm_Fibonacci 896.3 ± 10.9 882.0 916.7 4.26 ± 0.06

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ManyHashes 8.7 ± 0.1 8.6 8.8 1.00
levm_ManyHashes 18.0 ± 0.1 17.9 18.1 2.08 ± 0.02

Benchmark Results: BubbleSort

Command Mean [s] Min [s] Max [s] Relative
revm_BubbleSort 3.238 ± 0.041 3.210 3.335 1.00
levm_BubbleSort 6.067 ± 0.027 6.033 6.126 1.87 ± 0.02

Benchmark Results: ERC20 - Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Transfer 249.9 ± 1.8 247.6 253.2 1.00
levm_ERC20Transfer 537.6 ± 3.8 531.7 542.7 2.15 ± 0.02

Benchmark Results: ERC20 - Mint

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Mint 142.2 ± 1.8 140.8 146.5 1.00
levm_ERC20Mint 350.2 ± 2.0 347.4 353.6 2.46 ± 0.03

Benchmark Results: ERC20 - Approval

Command Mean [s] Min [s] Max [s] Relative
revm_ERC20Approval 1.044 ± 0.010 1.034 1.062 1.00
levm_ERC20Approval 2.010 ± 0.015 1.988 2.037 1.92 ± 0.02

@lima-limon-inc lima-limon-inc changed the title Add Revm's refunded gas and gas used when showing differences fix(levm): add refunded gas to revm's gas used when writing the ef-test report Feb 19, 2025
@lima-limon-inc lima-limon-inc added the levm Lambda EVM implementation label Feb 19, 2025
@lima-limon-inc lima-limon-inc marked this pull request as ready for review February 19, 2025 18:20
@lambdaclass lambdaclass deleted a comment from github-actions bot Feb 19, 2025
@lima-limon-inc lima-limon-inc marked this pull request as draft February 19, 2025 19:23
Copy link

EF Tests Comparison

Test Name MAIN PR DIFF
Summary: 16315/16773 (97.27%) 16292/16773 (97.13%) ⬇️️ -23
Prague: 2373/2373 (100.00%) 2350/2373 (99.03%) ⬇️️ -23
Cancun: 3579/3579 (100.00%) 3579/3579 (100.00%) ➖️
Shanghai: 221/221 (100.00%) 221/221 (100.00%) ➖️
Paris: 62/62 (100.00%) 62/62 (100.00%) ➖️
London: 39/39 (100.00%) 39/39 (100.00%) ➖️
Berlin: 35/35 (100.00%) 35/35 (100.00%) ➖️
Istanbul: 35/35 (100.00%) 35/35 (100.00%) ➖️
Petersburg: 2363/2399 (98.50%) 2363/2399 (98.50%) ➖️
Constantinople: 2323/2405 (96.59%) 2323/2405 (96.59%) ➖️
Byzantium: 2294/2330 (98.45%) 2294/2330 (98.45%) ➖️
SpuriousDragon: 579/579 (100.00%) 579/579 (100.00%) ➖️
Tangerine: 563/650 (86.62%) 563/650 (86.62%) ➖️
Homestead: 1208/1324 (91.24%) 1208/1324 (91.24%) ➖️
Frontier: 641/742 (86.39%) 641/742 (86.39%) ➖️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
levm Lambda EVM implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants