Skip to content

Commit 16c6fd3

Browse files
committed
chore: update benchmark
1 parent 26c7bdc commit 16c6fd3

File tree

9 files changed

+70
-137
lines changed

9 files changed

+70
-137
lines changed

.github/workflows/benchmark.yml

Lines changed: 0 additions & 26 deletions
This file was deleted.

testing/README.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,4 @@ pre-commit try-repo ./.. clang-tidy --verbose --all-files
99

1010
## Benchmark
1111

12-
```bash
13-
python3 testing/benchmark_hooks.py
14-
```
12+
See [benchmark](./benchmark.md).

testing/benchmark.md

Lines changed: 55 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,26 +4,69 @@ This document outlines the benchmarking process for comparing the performance of
44

55
> About tests performance can be found at: [![CodSpeed Badge](https://img.shields.io/endpoint?url=https://codspeed.io/badge.json)](https://codspeed.io/cpp-linter/cpp-linter-hooks)
66
7-
## Running the Benchmark
7+
## Run benchmark separately
88

99
```bash
10-
python3 testing/benchmark_hooks.py
10+
rm -rf examples || true
11+
git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
1112

12-
# or
13+
pre-commit clean
14+
hyperfine --warmup 1 -r 10 'pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files'
1315

14-
hyperfine -i --warmup 1 -r 5 'pre-commit run --config ../testing/benchmark_hook_1.yaml --all-files' 'pre-commit run --config ../testing/benchmark_hook_2.yaml --all-files'
16+
rm -rf examples || true
17+
git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
18+
pre-commit clean
19+
20+
hyperfine --warmup 1 -r 10 'pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files'
21+
```
22+
23+
Results:
24+
25+
```bash
26+
# Updated on 2025-09-19
27+
28+
Cleaned /home/sxp/.cache/pre-commit.
29+
Benchmark 1: pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files
30+
Time (mean ± σ): 150.2 ms ± 1.8 ms [User: 121.7 ms, System: 29.2 ms]
31+
Range (min … max): 148.3 ms … 153.9 ms 10 runs
32+
33+
Cleaned /home/sxp/.cache/pre-commit.
34+
Benchmark 1: pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files
35+
Time (mean ± σ): 122.6 ms ± 1.9 ms [User: 98.0 ms, System: 24.7 ms]
36+
Range (min … max): 120.3 ms … 125.5 ms 10 runs
37+
```
38+
39+
### Run benchmark comparison
40+
41+
Compare the results of both commands.
42+
43+
```bash
44+
rm -rf examples || true
45+
git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
46+
47+
hyperfine -i --warmup 1 -r 20 'pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' 'pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
1548
```
1649

17-
## Results
50+
Results:
1851

1952
```bash
20-
# Updated on 2025-09-02
21-
Benchmark Results:
53+
# Updated on 2025-09-19
54+
Benchmark 1: pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files
55+
Time (mean ± σ): 84.1 ms ± 3.2 ms [User: 73.5 ms, System: 10.2 ms]
56+
Range (min … max): 79.7 ms … 95.2 ms 20 runs
57+
58+
Warning: Ignoring non-zero exit code.
2259

23-
Hook | Avg (s) | Std (s) | Min (s) | Max (s) | Runs
24-
---------------------+------------------+------------------+------------------+------------------+-----------------
25-
cpp-linter-hooks | 12.473 | 1.738 | 11.334 | 15.514 | 5
26-
mirrors-clang-format | 4.960 | 0.229 | 4.645 | 5.284 | 5
60+
Benchmark 2: pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files
61+
Time (mean ± σ): 85.0 ms ± 3.0 ms [User: 71.8 ms, System: 13.3 ms]
62+
Range (min … max): 81.0 ms … 91.0 ms 20 runs
2763

28-
Results saved to testing/benchmark_results.txt
64+
Warning: Ignoring non-zero exit code.
65+
66+
Summary
67+
'pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' ran
68+
1.01 ± 0.05 times faster than 'pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
2969
```
70+
71+
> [!NOTE]
72+
> The results may vary based on the system and environment where the benchmarks are run.

testing/benchmark_hooks.py

Lines changed: 0 additions & 84 deletions
This file was deleted.

testing/benchmark_results.txt

Lines changed: 0 additions & 6 deletions
This file was deleted.

testing/good.c

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
#include <stdio.h>
2-
int main() {
3-
for (;;) break;
4-
printf("Hello world!\n");
5-
return 0;
2+
int main()
3+
{
4+
for (;;)
5+
break;
6+
printf("Hello world!\n");
7+
return 0;
68
}

testing/main.c

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,8 @@
11
#include <stdio.h>
2-
int main() {for (;;) break; printf("Hello world!\n");return 0;}
2+
int main()
3+
{
4+
for (;;)
5+
break;
6+
printf("Hello world!\n");
7+
return 0;
8+
}

testing/benchmark_hook_1.yaml renamed to testing/test-cpp-linter-hooks.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@ repos:
22
- repo: https://github.com/cpp-linter/cpp-linter-hooks
33
rev: v1.1.3
44
hooks:
5-
- id: clang-format
5+
- id: clang-format # v 21.1.0
66
args: [--style=file]
File renamed without changes.

0 commit comments

Comments
 (0)