Skip to content

feat(train): add generation and full-pool retrieval eval - #143

Merged
ppodolsky merged 1 commit into
mainfrom
audit-llm-code-and-repos
Aug 11, 2026
Merged

feat(train): add generation and full-pool retrieval eval#143
ppodolsky merged 1 commit into
mainfrom
audit-llm-code-and-repos

Conversation

@ppodolsky

Copy link
Copy Markdown
Contributor

Summary

  • add a full-pool retrieval evaluator with deterministic tie handling, rank-tail statistics, distractor shards, and duplicate-document validation
  • add free-running generation evaluation that shares prompt framing with supervised training and reports EOS, repetition, containment, and source-overlap signals
  • make retrieval rank semantics consistent in the existing evaluator and document both workflows

Verification

  • cargo test --workspace
  • cargo fmt --all -- --check
  • strict clippy via the repository pre-push hook
  • real Metal checkpoint smoke test produced a schema-v3 156-document report; the complete three-checkpoint matrix is running separately

@ppodolsky
ppodolsky merged commit df3d915 into main Aug 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant