Aida Stochastic SDb

Aida Stochastic SDb

Overview

This is a stochastic tool for producing random workloads using Markovian Processes for

1. fuzzing to increase the test coverage, find unexpected behaviour and bugs, and
2. testing runtime/space behaviour for future workloads.

The stochastic tool uses a random generator, parameterised distributions, and Markovian processes to mimic real-world test data. For fuzzing, a uniform Markovian process and simple parameters are used. For projecting future runtime/space behaviour, the Markovian process and parameters are learnt from historical data.

The workflow of the stochastic tool is shown below:

We have two ways to produce an events file depending on the use case. If we would like to use the stochastic tool for fuzzing, we generate an events file via the Uniform Generate tool; otherwise, we record an events file from historical data of the blockchain using the Record tool. The statistics of the events file can be visualised in the Visualizer tool. For running the Replay tool, the events file is to be converted to a simulation file using the Estimator tool.

Classification

• System Under Test
  • StateDB
• Configurable Functional Tests
  • Check results of StateDB operations with a reference implementation
• Non-Functional Tests
  • Memory-Consumption
  • Disk-Space
  • Runtime of operations
• Data set
  • Synthetic

Requirements

You need a configured Go language environment to build the CLI application. Please check the Go documentation for the details of installing the language compiler on your system.

TODO

Build

To build the aida-stochastic-sdb application, run make aida-stochastic-sdb.

The build process downloads all the needed modules and libraries, you don't need to install these manually. The aida-stochastic-sdb executable application will be created in /build folder.

Run

To use Aida Stochastic, execute the compiled binary with the command and flags for the desired operation.

./build/aida-stochastic-sdb command [command options] [arguments...]

command	description
estimate	Estimates parameters of access distributions and produces a simulation file
generate	Generate uniform events file
record	Record StateDB events while processing blocks
replay	Simulates StateDB operations using a random generator with realistic distributions
visualize	Produces a graphical view of the estimated parameters for various distributions

Generate Command

The generate command produces uniform/simple statistics, and an events file is written.

The following command,

./build/aida-stochastic-sdb generate                        # generate a uniform/simple events file

produces an events.json file with uniform parameters.

Options

replay:
    --block-length          defines the number of transactions per block (default: 10)
    --sync-period           defines the number of blocks per sync-period (default: 300)
    --transaction-length    determines the length of a transaction indirectly (default: 10)
    --num-contracts         number of contracts to create (default: 1_000)
    --num-keys              number of keys to generate (default: 1_000)
    --num-values            number of values to generate (default: 1_000)
    --snapshot-depth        depth of snapshot history (default: 100)

Record Command

Recorder collects events while running the block processor on historical data. As a result an events file is produced that can be converted to a simulation file by the estimator command.

./build/aida-stochastic-sdb record <blockNumFirst> <blockNumLast>

The stochastic record command requires two arguments: <blockNumFirst> <blockNumLast>

<blockNumFirst> and <blockNumLast> are the first and last block for recording events.

Options

record:
    --aida-db            set substate, updateset and deleted accounts directory
    --cpu-profile        enables CPU profiling
    --quiet              disable progress report (default: false)
    --sync-period        defines the number of blocks per sync-period (default: 300)
    --chainid            ChainID for replayer (default: 250)
    --output             output path
    --workers            number of worker threads that execute in parallel (default: 4)
    --substate-db        data directory for substate recorder/replayer

Estimate Command

The estimate command produces a simulation file from an events file. The estimator calculates the parameters of distributions based on the statistics found in the events file. The stochastic estimator command requires one argument: <events.json>

<events.json> is the event file produced by the stochastic recorder.

Replay Command

Replay performs random walk / does sampling for executing StateDB operations. It requires a simulation file for its execution.

./build/aida-stochastic-sdb replay <block#> <simulation.json>

The stochastic replay command requires two arguments: <simulationLength> <simulation.json>

<simulationLength> determines the number of blocks <simulation.json> contains the simulation parameters produced by the stochastic estimator.

Options

replay:
    --aida-db               set substate, updateset and deleted accounts directory
    --carmen-schema         select the DB schema used by Carmen's current state DB (default: 0)
    --continue-on-failure   continue execute after validation failure detected (default: false)
    --cpu-profile           enables CPU profiling
    --debug-from            sets the first block to print trace debug (default: 0)
    --quiet                 disable progress report (default: false)
    --memory-breakdown      enables printing of memory usage breakdown (default: false)
    --random-seed           set random seed (default: -1)
    --db-impl               select state DB implementation (default: "geth")
    --db-variant            select a state DB variant
    --db-tmp                sets the temporary directory where to place state DB data; uses system default if empty
    --db-logging            enable logging of all DB operations (default: false)
    --trace                 enable tracing
    --trace-debug           enable debug output for tracing
    --trace-file            set storage trace's output directory
    --shadow-db             use this flag when using an existing ShadowDb
    --db-shadow-impl        select state DB implementation to shadow the prime DB implementation
    --db-shadow-variant     select a state DB variant to shadow the prime DB implementation
    --balance-range         sets the balance range of the stochastic simulation
    --nonce-range           sets nonce range for stochastic simulation
    --log                   level of the logging of the app action ("critical", "error", "warning", "notice", "info", "debug"; default: INFO)

Visualize Command

Visualize collected events and estimation parameters. Uses web-browser for visualization.

On http://localhost:PORT displays:

• Counting/queuing statistics
• Stationary distributions
• Markov chains
• Etc.

./build/aida-stochastic-sdb record <blockNumFirst> <blockNumLast>
./build/aida-stochastic-sdb visualize events.json

The stochastic visualize command requires one argument: <events.json>

<events.json> is the event file produced by the stochastic recorder.`

Options

visualize:
    --port  enable visualization on `PORT` (default: 8080)

Use Cases

Fuzzing

The following sequence of commands would run the fuzzing for <block#> blocks:

./build/aida-stochastic-sdb generate                        # generate a uniform/simple events file
./build/aida-stochastic-sdb estimate events.json            # generate a simulation file
./build/aida-stochastic-sdb replay <block#> simulation.json # run simulation for <block#> blocks

Simulating historical data

The following sequence of commands would run the fuzzing for <block#> blocks:

./build/aida-stochastic-sdb record <begin> <end>            # record historical between <begin> and <end> block range and write events file
./build/aida-stochastic-sdb estimate events.json            # generate a simulation file
./build/aida-stochastic-sdb replay <block#> simulation.json # run simulation for <block#> blocks