Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bench: add tokio spawn_blocking + std fs benchmark #375

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

MrCroxx
Copy link

@MrCroxx MrCroxx commented Feb 14, 2025

Hi. Thank you for providing the crate compio. 🥰

This PR introduces tokio spawn_blocking + std case for fs benchmark.

The benchmark shows that compio works really good on reads. However, the write part is slower than tokio spawn_blocking + std. I'm not sure if I've done it correctly. Could you please help? 🙏

Raw output:

read-random/std         time:   [32.088 µs 32.115 µs 32.144 µs]
                        change: [-0.3149% -0.1749% -0.0343%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
read-random/tokio       time:   [1.0739 ms 1.1160 ms 1.1572 ms]
                        change: [-7.0959% -3.4174% +0.6847%] (p = 0.10 > 0.05)
                        No change in performance detected.
read-random/tokio_std   time:   [645.81 µs 675.20 µs 703.76 µs]
                        change: [-19.822% -16.300% -12.883%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) low mild
read-random/compio      time:   [55.984 µs 56.038 µs 56.099 µs]
                        change: [+1.2718% +1.4833% +1.7111%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  5 (5.00%) low severe
  3 (3.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe
read-random/compio-join time:   [37.399 µs 37.426 µs 37.456 µs]
                        change: [-7.4126% -6.6096% -5.7807%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
read-random/monoio      time:   [52.061 µs 52.112 µs 52.168 µs]
                        change: [-0.1142% -0.0114% +0.0913%] (p = 0.83 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low mild
  8 (8.00%) high mild
  2 (2.00%) high severe

read-all/std            time:   [1.2974 µs 1.3010 µs 1.3047 µs]
                        thrpt:  [2993.9 GiB/s 3002.5 GiB/s 3010.8 GiB/s]
                 change:
                        time:   [-1.4735% -1.2075% -0.9169%] (p = 0.00 < 0.05)
                        thrpt:  [+0.9254% +1.2222% +1.4955%]
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
read-all/tokio          time:   [17.116 µs 17.746 µs 18.336 µs]
                        thrpt:  [213.04 GiB/s 220.12 GiB/s 228.22 GiB/s]
                 change:
                        time:   [-2.7325% +1.0449% +4.9322%] (p = 0.58 > 0.05)
                        thrpt:  [-4.7003% -1.0341% +2.8093%]
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  8 (8.00%) low mild
read-all/tokio_std      time:   [14.740 µs 15.265 µs 15.755 µs]
                        thrpt:  [247.94 GiB/s 255.90 GiB/s 265.01 GiB/s]
Found 13 outliers among 100 measurements (13.00%)
  13 (13.00%) low mild
read-all/compio         time:   [864.57 ns 865.31 ns 866.07 ns]
                        thrpt:  [4510.3 GiB/s 4514.3 GiB/s 4518.1 GiB/s]
                 change:
                        time:   [+3.0212% +3.3689% +3.7015%] (p = 0.00 < 0.05)
                        thrpt:  [-3.5694% -3.2591% -2.9326%]
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe
read-all/monoio         time:   [797.30 ns 798.28 ns 799.35 ns]
                        thrpt:  [4886.8 GiB/s 4893.3 GiB/s 4899.3 GiB/s]
                 change:
                        time:   [+0.1896% +0.3742% +0.5816%] (p = 0.00 < 0.05)
                        thrpt:  [-0.5783% -0.3728% -0.1893%]
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high mild

write-random/std        time:   [43.241 µs 43.292 µs 43.351 µs]
                        change: [+1.8799% +2.1045% +2.3321%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
write-random/tokio      time:   [1.0925 ms 1.1352 ms 1.1770 ms]
                        change: [-2.9237% +0.7864% +4.3320%] (p = 0.68 > 0.05)
                        No change in performance detected.
write-random/tokio_std  time:   [66.256 µs 72.390 µs 78.288 µs]
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high mild
write-random/compio     time:   [433.36 µs 453.40 µs 473.76 µs]
                        change: [+2.2686% +7.0361% +12.247%] (p = 0.01 < 0.05)
                        Performance has regressed.
write-random/compio-join
                        time:   [126.05 µs 127.59 µs 128.94 µs]
                        change: [+3.8459% +5.4051% +6.8271%] (p = 0.00 < 0.05)
                        Performance has regressed.
write-random/monoio     time:   [429.43 µs 447.35 µs 465.80 µs]
                        change: [+7.3082% +12.674% +18.043%] (p = 0.00 < 0.05)
                        Performance has regressed.
write-random/monoio-join
                        time:   [97.917 µs 104.98 µs 112.43 µs]
                        change: [-9.4316% -5.7760% -2.5335%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  12 (12.00%) low severe

write-all/std           time:   [11.692 µs 11.704 µs 11.719 µs]
                        thrpt:  [5.2083 GiB/s 5.2147 GiB/s 5.2202 GiB/s]
                 change:
                        time:   [+0.0185% +0.3369% +0.8174%] (p = 0.15 > 0.05)
                        thrpt:  [-0.8108% -0.3358% -0.0185%]
                        No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
  7 (7.00%) high mild
  6 (6.00%) high severe
write-all/tokio         time:   [157.17 µs 164.08 µs 170.56 µs]
                        thrpt:  [366.44 MiB/s 380.92 MiB/s 397.65 MiB/s]
                 change:
                        time:   [-4.8568% -0.7049% +3.3899%] (p = 0.74 > 0.05)
                        thrpt:  [-3.2788% +0.7099% +5.1047%]
                        No change in performance detected.
write-all/tokio_std     time:   [23.723 µs 25.241 µs 26.837 µs]
                        thrpt:  [2.2743 GiB/s 2.4181 GiB/s 2.5728 GiB/s]
write-all/compio        time:   [105.73 µs 110.97 µs 116.38 µs]
                        thrpt:  [537.05 MiB/s 563.21 MiB/s 591.14 MiB/s]
                 change:
                        time:   [-2.9022% +2.3375% +7.9584%] (p = 0.38 > 0.05)
                        thrpt:  [-7.3717% -2.2841% +2.9890%]
                        No change in performance detected.
write-all/monoio        time:   [101.60 µs 105.76 µs 110.00 µs]
                        thrpt:  [568.21 MiB/s 590.99 MiB/s 615.18 MiB/s]
                 change:
                        time:   [-16.455% -12.631% -8.4236%] (p = 0.00 < 0.05)
                        thrpt:  [+9.1984% +14.458% +19.695%]
                        Performance has improved.

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant