fix(ci): measure coverage over all test targets, not just --lib - #26
Merged
Conversation
Completes the fleet rollout. These two repos were held back when the other twelve landed, because their mains were red at the time and a gate change cannot be verified against a build that does not run. Both are green now. `cargo llvm-cov --lib` builds only the lib target's own unit tests. Integration tests under tests/ are separate crates producing separate binaries, so `--lib` never builds them and records nothing they cover -- which silently narrows the claim the gate makes from "this code is tested" to "this code is tested from inside src/". bluetooth-forensic reported 54 lines uncovered on a library its own integration tests cover to 100%. Binary shells are excluded rather than counted: dropping `--lib` pulls src/bin/ into the report at 0% (built by the test profile, never executed), which would make the gate unsatisfiable for reasons unrelated to test quality. Verified against each repo's own gate rather than a generic one, since they differ: usb-forensic gates with `--fail-uncovered-lines 0`, peripheral-forensic walks the lcov and honours `// cov:unreachable`. Both pass. usb-forensic's gate was also checked for the failure mode this session kept finding -- a gate that cannot go red. Appending four deliberately uncovered lines makes it exit 1; removing them makes it exit 0. It is live, not vacuous.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes the fleet rollout — these two were held back when the other twelve landed, because their mains were red and a gate change can't be verified against a build that doesn't run. Both are green now.
cargo llvm-cov --libbuilds only the lib target's own unit tests. Integration tests undertests/are separate crates producing separate binaries, so--libnever builds them and records nothing they cover — narrowing the claim from "this code is tested" to "this code is tested from insidesrc/". bluetooth-forensic reported 54 lines uncovered on a library its own integration tests cover to 100%.Binary shells are excluded rather than counted: dropping
--libpullssrc/bin/in at 0% (built by the test profile, never executed), which would make the gate unsatisfiable for reasons unrelated to test quality.Verified against each repo's own gate, since they differ — usb-forensic uses
--fail-uncovered-lines 0, peripheral-forensic walks the lcov and honours// cov:unreachable. Both pass.usb-forensic's gate was also checked for the failure mode this session kept finding — a gate that can't go red. Appending four deliberately uncovered lines makes it exit 1; removing them makes it exit 0. Live, not vacuous.