Skip to content

No coverage delta tracking in test scoring #18

@DarlingtonDeveloper

Description

@DarlingtonDeveloper

Context

SPEC-03's test scoring includes test_score *= (1.0 + coverage_delta * 0.5) to reward mutations that increase code coverage.

Current Behavior

discover_eval_config has a coverage_command field but it's never populated or used. Test scoring is binary: all pass = 1.0, any fail = 0.0.

Expected Behavior

Capture baseline coverage percentage, run coverage after mutation, compute delta, and apply the multiplier to the test score.

Impact

  • No incentive for mutations to improve coverage
  • Mutations that add tests without improving coverage score the same as those that do

References

  • atlas-specs/03-EVALUATION.md — coverage delta formula
  • atlas/evaluation/discovery.pycoverage_command field (always None)
  • atlas/evaluation/dimensions.pyscore_tests has no coverage logic

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions