perf(fuzz): parallel stateless fuzzing

[yash-atreya]

Motivation

Stacked on #11769

Solution

• Introduce SharedFuzzState which consists of global values of runs, timer, fail_fast and rejects to determine whether the fuzz test should_continue using the SharedFuzzState::should_continue
• Fuzz workers atomically update the values in SharedFuzzState
• For timed campaigns, timeout specified in foundry.toml is applied to each worker as-is to fit as many runs as possible i.e, timeout value is NOT divided by number of workers.
• fuzz.runs config value is divided by the number of workers i.e, worker_runs = fuzz.runs / num_worker — reducing the time required for the campaign
• fuzz.seed set in config is altered per worker so that the workers are covering different inputs. For worker0 / master worker, the provided seed is used as is. For workers with WorkerID > 0, the seed is set to keccak256(fuzz.seed || WorkerID)
• Currently, the corpus SYNC_INTERVAL is naively set to 1000, we can adjust this after some more empirical feedback

TODO

• Add max_rejects in SharedFuzzState to track total rejects across workers and fail accordingly 40263fd
• Set failures in run_worker via SharedState::try_claim_failure - This will intercept other workers as well and stop fuzzing entirely e4c6060
• Introduce sync_interval for the WorkerCorpus and call WorkerCorpus::sync ed234d8
• Introduce GlobalCorpusMetrics and sync worker corpus metrics d973b79
• Determine number of workers - all available cores / --jobs e991826
• Integrate into fn fuzz and run workers using rayon::IntoParallelIterator
• Aggregate worker results 47bd12d
• Address breaking tests

Benchmarks

1. Uniswap/v4-core - 1000 fuzz runs

hyperfine "forge test --match-test 'test[^(]*\([^)]+\)'" "forge-pf test --match-test 'test[^(]*\([^)]+\)'" --warmup 1 --setup "forge b"
Benchmark 1: forge test --match-test 'test[^(]*\([^)]+\)'
  Time (mean ± σ):      6.147 s ±  0.355 s    [User: 24.952 s, System: 0.484 s]
  Range (min … max):    5.335 s …  6.545 s    10 runs
 
Benchmark 2: forge-pf test --match-test 'test[^(]*\([^)]+\)'
  Time (mean ± σ):      4.494 s ±  0.140 s    [User: 30.911 s, System: 0.517 s]
  Range (min … max):    4.305 s …  4.714 s    10 runs
 
Summary
  forge-pf test --match-test 'test[^(]*\([^)]+\)' ran
    1.37 ± 0.09 times faster than forge test --match-test 'test[^(]*\([^)]+\)'

With coverage-guided fuzzing enabled:

hyperfine "forge test --match-test 'test[^(]*\([^)]+\)'" "forge-pf test --match-test 'test[^(]*\([^)]+\)'" --warmup 1 --setup "forge b" --runs 5 --conclude 'rm -rf corpus/'
Benchmark 1: forge test --match-test 'test[^(]*\([^)]+\)'
  Time (mean ± σ):     14.171 s ±  2.630 s    [User: 56.604 s, System: 1.333 s]
  Range (min … max):   11.709 s … 18.144 s    5 runs
 
Benchmark 2: forge-pf test --match-test 'test[^(]*\([^)]+\)'
  Time (mean ± σ):     12.531 s ±  0.644 s    [User: 61.457 s, System: 2.709 s]
  Range (min … max):   11.943 s … 13.344 s    5 runs
 
Summary
  forge-pf test --match-test 'test[^(]*\([^)]+\)' ran
    1.13 ± 0.22 times faster than forge test --match-test 'test[^(]*\([^)]+\)'

2. Ithacaxyz/account - 1000 fuzz runs

hyperfine "forge test --match-test 'test[^(]*\([^)]+\)'" "forge-pf test --match-test 'test[^(]*\([^)]+\)'" --warmup 1 --setup "forge b" --runs 10
Benchmark 1: forge t  --mt 'test[^(]*\([^)]+\)'
  Time (mean ± σ):     15.754 s ±  2.097 s    [User: 67.377 s, System: 0.297 s]
  Range (min … max):   13.318 s … 18.829 s    10 runs
 
Benchmark 2: forge-pf test --match-test 'test[^(]*\([^)]+\)'
  Time (mean ± σ):     11.183 s ±  0.783 s    [User: 80.358 s, System: 0.339 s]
  Range (min … max):   10.260 s … 12.490 s    10 runs
 
Summary
  forge-pf test --match-test 'test[^(]*\([^)]+\)' ran
    1.41 ± 0.21 times faster than forge t  --mt 'test[^(]*\([^)]+\)'

PR Checklist

• Added Tests
• Added Documentation
• Breaking changes