ci: use benchtools to benchmark the committer

[AvivYossef-starkware]

No description provided.

[reviewable-StarkWare]

This change is 

[AvivYossef-starkware]

• bench_tools: add absolute limits to benchmark runs #9861 : 3 dependent PRs (#9526 , #9874 , #9876 )
• bench_tools: dont capture output when running a benchmark #9860
• ci: use benchtools to benchmark the committer #9707 👈 (View in Graphite)
• bench_tools: fix committer benchmark config #9807
• bench_tools: run and compare benchmark #9699
• bench_tools: support deserialization of criterion change #9698
• bench_tools: save specific benchmark result #9742
• bench_tools: add criterion benchmark names to benchmark config #9697
• bench_tools: add bench tools to allowed scopes #9696
• ci: run benchmark command #9636
• infra: copy dir rec #9802
• main-v0.14.1

This stack of pull requests is managed by Graphite. Learn more about stacking.

[avi-starkware]

.github/workflows/committer_ci.yml line 134 at r1 (raw file):

      - run: bash ./crates/starknet_committer_and_os_cli/benches/bench_split_and_prepare_post.sh benchmarks_list.txt benchmark_comment_body.txt

      - run: echo BENCHES_RESULT=$(cat benchmark_comment_body.txt) >> $GITHUB_ENV

After your change, you no longer post a comment on github.
Is that intentional?

Code quote:

      # Benchmark the new code, splitting the benchmarks, and prepare the results for posting a comment.
      - run: bash ./crates/starknet_committer_and_os_cli/benches/bench_split_and_prepare_post.sh benchmarks_list.txt benchmark_comment_body.txt

      - run: echo BENCHES_RESULT=$(cat benchmark_comment_body.txt) >> $GITHUB_ENV

[AvivYossef-starkware]

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @avi-starkware)

---

.github/workflows/committer_ci.yml line 134 at r1 (raw file):

Previously, avi-starkware (Avi Cohen) wrote…

After your change, you no longer post a comment on github.
Is that intentional?

Yes, I don't find it useful.
I added informative prints to the command instead

[avi-starkware]

Do you plan to keep this in the committer_ci ?
What about the gateway benchmark you added in the config?

@avi-starkware reviewed all commit messages.
Reviewable status: 0 of 1 files reviewed, all discussions resolved

[avi-starkware]

@avi-starkware reviewed 1 of 1 files at r1.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @AvivYossef-starkware)

[avi-starkware]

.github/workflows/committer_ci.yml line 111 at r1 (raw file):

      # Benchmark the current branch and compare to the previous run.
      # (downloads inputs from GCS, fails if regression > 8%).
      - run: cargo run -p bench_tools -- run-and-compare --package starknet_committer_and_os_cli --out /tmp/new_results --regression-limit 8.0

Can we cache the inputs?

Code quote:

      # (downloads inputs from GCS, fails if regression > 8%).
      - run: cargo run -p bench_tools -- run-and-compare --package starknet_committer_and_os_cli --out /tmp/new_results --regression-limit 8.0

[AvivYossef-starkware]

In the committer-ci, we will run it with
--package starknet_committer_and_os_cli
No one asked me to add the gateway benchmark to the CI.
I have added it to the benchmark config before I had access to the committer input files,
to verify that it works

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @avi-starkware)

---

.github/workflows/committer_ci.yml line 111 at r1 (raw file):

Previously, avi-starkware (Avi Cohen) wrote…

Can we cache the inputs?

Yes, we can.
The CLI tool doesn't support it right now.
But I can add bash commands for it.

[avi-starkware]

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @AvivYossef-starkware)

---

.github/workflows/committer_ci.yml line 111 at r1 (raw file):

Previously, AvivYossef-starkware wrote…

Yes, we can.
The CLI tool doesn't support it right now.
But I can add bash commands for it.

Try to use the inputs from cache at least in this second command. No need to download the files twice, and you already have the command to read inputs from a local dir.

[AvivYossef-starkware]

Done

[avi-starkware]

@avi-starkware reviewed 1 of 2 files at r2, 1 of 1 files at r3, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @AvivYossef-starkware)

[AvivYossef-starkware]

@AvivYossef-starkware reviewed all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @AvivYossef-starkware)