bench_tools: add criterion benchmark names to benchmark config

[AvivYossef-starkware]

No description provided.

[reviewable-StarkWare]

This change is 

[AvivYossef-starkware]

• bench_tools: add absolute limits to benchmark runs #9861 : 3 dependent PRs (#9526 , #9874 , #9876 )
• bench_tools: dont capture output when running a benchmark #9860
• ci: use benchtools to benchmark the committer #9707
• bench_tools: fix committer benchmark config #9807
• bench_tools: run and compare benchmark #9699
• bench_tools: support deserialization of criterion change #9698
• bench_tools: save specific benchmark result #9742
• bench_tools: add criterion benchmark names to benchmark config #9697 👈 (View in Graphite)
• bench_tools: add bench tools to allowed scopes #9696
• ci: run benchmark command #9636
• infra: copy dir rec #9802
• main-v0.14.1

This stack of pull requests is managed by Graphite. Learn more about stacking.

[avi-starkware]

@avi-starkware reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @meship-starkware)

---

crates/bench_tools/src/types/benchmark_config.rs line 13 at r1 (raw file):

    /// If None, assumes a single benchmark with the same name as the config.
    /// Used for regression checking to know which criterion directories to check.
    pub criterion_benchmark_names: Option<&'static [&'static str]>,

I don't understand this new config

Code quote:

    /// Optional list of Criterion benchmark names that this benchmark suite produces.
    /// If None, assumes a single benchmark with the same name as the config.
    /// Used for regression checking to know which criterion directories to check.
    pub criterion_benchmark_names: Option<&'static [&'static str]>,

[graphite-app]

Merge activity

• Oct 29, 12:19 PM UTC: Graphite rebased this pull request, because this pull request is set to merge when ready.

[AvivYossef-starkware]

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @avi-starkware and @meship-starkware)

---

crates/bench_tools/src/types/benchmark_config.rs line 13 at r1 (raw file):

Previously, avi-starkware (Avi Cohen) wrote…

I don't understand this new config

look here

BenchmarkConfig {
        name: "dummy_benchmark",
        package: "bench_tools",
        cmd_args: &["bench", "-p", "bench_tools", "--bench", "dummy_bench"],
        input_dir: Some("crates/bench_tools/data/dummy_bench_input"),
        criterion_benchmark_names: Some(&[
            "dummy_sum_100",
            "dummy_sum_1000",
            "dummy_process_small_input",
            "dummy_process_large_input",
        ]),
    },

when running cargo bench -p bench_tools --bench dummy_bench
4 benches are actually run.
It happens because of adding a bench function like that

c.bench_function("dummy_process_large_input", |b| {
        b.iter(|| black_box(process_input(&large_input)))
    });

[avi-starkware]

Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @meship-starkware)