Skip to content

test: measure coverage out-of-process for true 100%#125

Merged
svaningelgem merged 3 commits into
masterfrom
fix/coverage-startup-measurement
May 27, 2026
Merged

test: measure coverage out-of-process for true 100%#125
svaningelgem merged 3 commits into
masterfrom
fix/coverage-startup-measurement

Conversation

@svaningelgem
Copy link
Copy Markdown
Owner

Problem

Project coverage sat at ~76-79% even though the code is well tested. The cause is when coverage starts: pytest-cov begins measuring inside a pytest hook, after pytest has already imported the logprise package and its pytest plugin at startup (via the pytest11 entry point). Their module-level definitions (imports, class/function defs, appriser = Appriser()) therefore never registered as executed.

PR #97 worked around this for pytest_plugin.py with a session-scoped fixture that reloaded the module under coverage. That only patched one file — __init__.py had the same gap — and left a reload hack in the test suite.

Fix

Collect coverage out-of-process with coverage run -m pytest, which starts measurement before pytest imports anything:

  • build.yml / release.yml: coverage run -m pytest + coverage xml instead of pytest --cov.
  • pyproject.toml: drop the pytest-cov addopts (a plain pytest run stays fast and coverage-free; coverage run reads [tool.coverage.run]). Note: --cov and coverage run together conflict, so it must go.
  • conftest.py: remove the module-reload fixture — no longer needed.
  • Add tests for the two remaining real branches (notify reporting failure; an already-handled logger).

Result

logprise/__init__.py          206   0   72   0   100%
logprise/pytest_plugin.py      34   0    4   0   100%
TOTAL                         240   0   76   0   100%

Collect coverage with 'coverage run -m pytest' instead of pytest-cov.
pytest-cov starts measuring inside a pytest hook, after pytest has
already imported the logprise package and its pytest plugin at startup,
so their module-level definitions were never counted. Running coverage
out-of-process starts measurement before those imports.

- Switch build.yml and release.yml to 'coverage run -m pytest' + 'coverage xml'.
- Drop the pytest-cov addopts from pyproject (a plain 'pytest' stays
  coverage-free; 'coverage run' reads [tool.coverage.run]).
- Remove the conftest reload fixture: the out-of-process run measures the
  plugin's module-level code directly, so the reload hack is no longer needed.
- Add tests for the two remaining uncovered branches (notify reporting
  failure, and an already-handled logger), reaching 100%.
@codecov
Copy link
Copy Markdown

codecov Bot commented May 27, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (6b600e4) to head (2b31c3d).

Additional details and impacted files
@@             Coverage Diff              @@
##           master      #125       +/-   ##
============================================
+ Coverage   72.50%   100.00%   +27.50%     
============================================
  Files           2         2               
  Lines         240       240               
  Branches       38        38               
============================================
+ Hits          174       240       +66     
+ Misses         63         0       -63     
+ Partials        3         0        -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Coverage is now collected with 'coverage run -m pytest', so pytest-cov
is unused. Depend on coverage[toml] directly (previously only pulled in
transitively via pytest-cov); the [toml] extra is needed so coverage can
read its [tool.coverage] config from pyproject.toml on Python 3.10.
Also declare pytest directly, since the suite is run via 'coverage run -m pytest'.
Note that the extra is only needed for Python <= 3.10 and can be dropped
once 3.10 support is removed.
@svaningelgem svaningelgem merged commit 1df315b into master May 27, 2026
10 checks passed
@svaningelgem svaningelgem deleted the fix/coverage-startup-measurement branch May 27, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant