Skip to content

Simplify fuzzing coverage (CI) scripts somewhat #3925

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Simplify fuzzing coverage (CI) scripts somewhat #3925

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4e3d54e
Simplify fuzzing coverage (CI) scripts somewhat
TheBlueMatt Jul 14, 2025
08795f2
Switch to codecov CLI
TheBlueMatt Jul 15, 2025
37cd32d
Separate codecov reports out between testing and fuzzing
TheBlueMatt Jul 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,13 +125,20 @@ jobs:
cargo install cargo-llvm-cov
export RUSTFLAGS="-Coverflow-checks=off"
cargo llvm-cov --features rest-client,rpc-client,tokio,serde --codecov --hide-instantiations --output-path=target/codecov.json
curl --verbose -O https://cli.codecov.io/latest/linux/codecov
chmod +x codecov
# Could you use this to fake the coverage report for your PR? Sure.
# Will anyone be impressed by your amazing coverage? No
# Maybe if codecov wasn't broken we wouldn't need to do this...
bash <(curl -s https://codecov.io/bash) -f target/codecov.json -t "f421b687-4dc2-4387-ac3d-dc3b2528af57"
./codecov --verbose upload-process --disable-search --fail-on-error -f target/codecov.json -t "f421b687-4dc2-4387-ac3d-dc3b2528af57" -F 'tests'
cargo clean
- name: Run fuzz coverage generation
run: |
./ci/ci-fuzz.sh
./contrib/generate_fuzz_coverage.sh --output-dir `pwd` --output-codecov-json
# Could you use this to fake the coverage report for your PR? Sure.
# Will anyone be impressed by your amazing coverage? No
# Maybe if codecov wasn't broken we wouldn't need to do this...
./codecov --verbose upload-process --disable-search --fail-on-error -f fuzz-codecov.json -t "f421b687-4dc2-4387-ac3d-dc3b2528af57" -F 'fuzzing'

benchmark:
runs-on: ubuntu-latest
Expand Down
24 changes: 0 additions & 24 deletions ci/ci-fuzz.sh
View file
Open in desktop

This file was deleted.

36 changes: 9 additions & 27 deletions contrib/generate_fuzz_coverage.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,19 @@
#!/bin/bash
set -e
set -x
set -ex

# Parse command line arguments
OUTPUT_DIR="coverage-report"
OUTPUT_CODECOV_JSON=0
while [[ $# -gt 0 ]]; do
case $1 in
--output-dir)
OUTPUT_DIR="$2"
shift 2
;;
--output-codecov-json)
OUTPUT_CODECOV_JSON=1
shift 1
;;
*)
echo "Unknown option: $1"
echo "Usage: $0 [--output-dir OUTPUT_DIRECTORY]"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usage information is missing the newly added --output-codecov-json flag. Please update the help text to include this option:

echo "Usage: $0 [--output-dir OUTPUT_DIRECTORY] [--output-codecov-json]"

This will help users understand all available command-line options.

Suggested change
echo "Usage: $0 [--output-dir OUTPUT_DIRECTORY]"
echo "Usage: $0 [--output-dir OUTPUT_DIRECTORY] [--output-codecov-json]"

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a CI-only flag so not really worth documenting.

Expand Down Expand Up @@ -54,34 +58,12 @@ mkdir -p "$OUTPUT_DIR"
export RUSTFLAGS="--cfg=fuzzing --cfg=secp256k1_fuzz --cfg=hashes_fuzz"

# dont run this command when running in CI
if [ "$CI" != "true" ] && [ "$GITHUB_ACTIONS" != "true" ]; then
if [ "$OUTPUT_CODECOV_JSON" = "0" ]; then
cargo llvm-cov --html --ignore-filename-regex "fuzz/" --output-dir "$OUTPUT_DIR"

# Check if coverage report was generated successfully
# The report is generated in $OUTPUT_DIR/html/index.html when using --html --output-dir
if [ ! -f "$OUTPUT_DIR/html/index.html" ]; then
echo "Error: Failed to generate coverage report at $OUTPUT_DIR/html/index.html"
echo "Contents of $OUTPUT_DIR:"
ls -la "$OUTPUT_DIR" || echo "Directory $OUTPUT_DIR does not exist"
if [ -d "$OUTPUT_DIR/html" ]; then
echo "Contents of $OUTPUT_DIR/html:"
ls -la "$OUTPUT_DIR/html"
fi
exit 1
fi
echo "Coverage report generated in $OUTPUT_DIR/html/index.html"
fi

# Generate codecov JSON format if running in CI environment
if [ "$CI" = "true" ] || [ "$GITHUB_ACTIONS" = "true" ]; then
echo "CI environment detected, generating codecov JSON format..."
else
cargo llvm-cov --codecov --ignore-filename-regex "fuzz/" --output-path "$OUTPUT_DIR/fuzz-codecov.json"

if [ -f "$OUTPUT_DIR/fuzz-codecov.json" ] && [[ "$OUTPUT_DIR" == *"target/"* ]]; then
TARGET_DIR="../target"
cp "$OUTPUT_DIR/fuzz-codecov.json" "$TARGET_DIR/fuzz-codecov.json"
echo "Fuzz codecov report copied to $TARGET_DIR/fuzz-codecov.json"
fi
echo "Fuzz codecov report available at $OUTPUT_DIR/fuzz-codecov.json"
fi


Expand Down
Loading