Problem
career-ops now relies on evaluation reports as durable artifacts for downstream tooling: tracker links, pattern analysis, dashboard metadata, PDF decisions, and application follow-up. The report format has several implied contract fields, but there is no standalone validator for report files.
This means a report can silently miss key fields such as **URL:**, **Legitimacy:**, ## Machine Summary, score format, or PDF status, and later scripts only fail indirectly.
Proposed solution
Add a lightweight verify-reports.mjs script that scans reports/*.md and validates the report contract:
- header includes
**URL:**
- header includes
**Legitimacy:** {tier}
- score is parseable as
X.X/5 or a documented skip value
- PDF status is present when the report came from auto-pipeline/pipeline
## Machine Summary is present and parseable when available
- machine summary fields are from the accepted schema used by
analyze-patterns.mjs
Why this fits
This makes evaluation artifacts auditable before downstream scripts consume them. It also gives contributors a focused check to run after changing modes, Gemini output, or report generation.
Non-overlap audit
I checked open PRs before proposing this. This does not overlap with:
Suggested implementation scope
- New
verify-reports.mjs
- Optional
npm run verify:reports script
- Add focused tests to
test-all.mjs with temporary report fixtures
- Document in
docs/SCRIPTS.md
Test plan
node verify-reports.mjs --self-test
node test-all.mjs --quick
- Fixture coverage for missing URL, missing legitimacy, malformed Machine Summary, and valid report
Problem
career-ops now relies on evaluation reports as durable artifacts for downstream tooling: tracker links, pattern analysis, dashboard metadata, PDF decisions, and application follow-up. The report format has several implied contract fields, but there is no standalone validator for report files.
This means a report can silently miss key fields such as
**URL:**,**Legitimacy:**,## Machine Summary, score format, or PDF status, and later scripts only fail indirectly.Proposed solution
Add a lightweight
verify-reports.mjsscript that scansreports/*.mdand validates the report contract:**URL:****Legitimacy:** {tier}X.X/5or a documented skip value## Machine Summaryis present and parseable when availableanalyze-patterns.mjsWhy this fits
This makes evaluation artifacts auditable before downstream scripts consume them. It also gives contributors a focused check to run after changing modes, Gemini output, or report generation.
Non-overlap audit
I checked open PRs before proposing this. This does not overlap with:
Suggested implementation scope
verify-reports.mjsnpm run verify:reportsscripttest-all.mjswith temporary report fixturesdocs/SCRIPTS.mdTest plan
node verify-reports.mjs --self-testnode test-all.mjs --quick