fix(eval): gate dead links before evaluation in oferta/auto-pipeline#937
Conversation
Level-3 scan results enter pipeline.md as pending entries with no
liveness check. oferta and auto-pipeline then run the full A-G
evaluation, write a report, and generate a tailored PDF before anyone
discovers the posting is gone — a 404/expired page served as a static
fallback ("position filled", empty shell) scores against phantom
content.
Add a liveness gate at the eval entry points, mirroring the apply-mode
preflight gate (santifer#887): auto-pipeline gets a Step 0.5 that judges the
Step 0 snapshot before Step 1, and oferta gets a URL-input gate before
Block A. A closed/dead/redirected page stops the flow — no eval, report,
or CV — and marks the pipeline entry dead. JD-text input (no URL) skips
the gate. When auto-pipeline drives oferta, oferta reuses the already-
cleared snapshot instead of re-navigating.
Inline-snapshot judgment (not a check-liveness.mjs call) keeps the
convention consistent with apply.md and avoids a second navigation,
since Step 0 already holds the snapshot.
+1 regression test asserting both eval modes carry the gate.
Fixes santifer#835
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
📝 WalkthroughWalkthroughThis PR adds a liveness gate step to the job evaluation workflows. Before evaluating job postings, both ChangesLiveness Gate Documentation and Testing
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Welcome to career-ops, @maxmilian! Thanks for your first PR. A few things to know:
We'll review your PR soon. Join our Discord if you have questions. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test-all.mjs`:
- Around line 515-528: The test currently checks for presence of liveness gate
strings in ofertaMode and autoPipelineMode (variables ofertaMode,
autoPipelineMode) but not their order; add optional positional assertions to
ensure the liveness gate heading appears before the evaluation start markers:
verify indexOf('## Liveness gate (URL inputs)') < indexOf('Block A') in
ofertaMode and indexOf('## Step 0.5 — Liveness gate') < indexOf('Step 1') in
autoPipelineMode, and only pass when both presence and ordering checks succeed
(update the pass/fail branch that currently calls pass('eval modes ...') /
fail('eval modes ...')).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: e463d7eb-fe54-43d4-b8da-4f6d36fe0ee2
📒 Files selected for processing (3)
modes/auto-pipeline.mdmodes/oferta.mdtest-all.mjs
| const ofertaMode = readFile('modes/oferta.md'); | ||
| const autoPipelineMode = readFile('modes/auto-pipeline.md'); | ||
| if ( | ||
| ofertaMode.includes('## Liveness gate (URL inputs)') && | ||
| ofertaMode.includes('closed posting evidence') && | ||
| ofertaMode.includes('Do not continue to Block A until this gate is resolved') && | ||
| autoPipelineMode.includes('## Step 0.5 — Liveness gate') && | ||
| autoPipelineMode.includes('closed posting evidence') && | ||
| autoPipelineMode.includes('Do not continue to Step 1 until this gate is resolved') | ||
| ) { | ||
| pass('eval modes (oferta/auto-pipeline) gate dead links before evaluation'); | ||
| } else { | ||
| fail('eval modes missing liveness gate before evaluation'); | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | 💤 Low value
Test coverage looks good; consider optional positional validation.
The test correctly validates that both evaluation modes include the required liveness gate documentation elements:
- Specific heading text for each mode
- "closed posting evidence" classification criteria
- Gate resolution requirement before proceeding
The test follows the existing pattern established by the apply mode preflight gate test above it (lines 502-513).
Optional enhancement: The current test checks for text presence but not position. Consider adding a check that the liveness gate heading appears before the evaluation start point (before "Block A" in oferta.md and before "Step 1" in auto-pipeline.md) to ensure the gate actually runs first. However, this is a nice-to-have rather than a requirement, as the current test already validates the contract sufficiently.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@test-all.mjs` around lines 515 - 528, The test currently checks for presence
of liveness gate strings in ofertaMode and autoPipelineMode (variables
ofertaMode, autoPipelineMode) but not their order; add optional positional
assertions to ensure the liveness gate heading appears before the evaluation
start markers: verify indexOf('## Liveness gate (URL inputs)') < indexOf('Block
A') in ofertaMode and indexOf('## Step 0.5 — Liveness gate') < indexOf('Step 1')
in autoPipelineMode, and only pass when both presence and ordering checks
succeed (update the pass/fail branch that currently calls pass('eval modes ...')
/ fail('eval modes ...')).
|
Merged — this closes the loop on #835 (@PaulaBarszcz's report): no more full evaluations + PDFs burned on postings that died yesterday. Token-friendly AND tracker-friendly. With #936, #938 and the #923 fix, that's four merges today, @maxmilian — quietly becoming one of the most productive contributors here. 🚀 |
2 participants
What does this PR do?
Adds a liveness gate at the evaluation entry points so a dead/expired job link can no longer trigger a full A–G evaluation, a written report, and a tailored PDF on phantom content.
modes/auto-pipeline.md— new Step 0.5 — Liveness gate between Step 0 (Extract JD) and Step 1 (A–G eval). The Step 0 Playwright snapshot is already in hand, so the gate judges it inline; a closed/dead/redirected/404 page stops before Step 1 and marks thedata/pipeline.mdentry dead.modes/oferta.md— new Liveness gate (URL inputs) before Block A (covers direct-URL entry, e.g. runningofertaon a pending pipeline link). Whenauto-pipelinealready cleared the link in its Step 0.5, oferta reuses that snapshot instead of re-navigating. JD-text input (no URL) skips the gate.test-all.mjs— +1 regression assertion that both eval modes carry the gate (mirrors the apply-mode gate test).Related issue
Fixes #835
Notes / design choices
check-liveness.mjs.auto-pipelineStep 0 already captures a Playwright snapshot, so judging liveness inline avoids a second navigation and keeps the convention consistent withmodes/apply.md. (Thanks @PaulaBarszcz for the forensic 17/25-dead-links writeup and for surfacingcheck-liveness.mjs— the gate uses the same active/closed signal classes.)pdfmode is intentionally not gated. A standalonepdfrun renders an already-written report; the dead link must be caught upstream at the eval, which is exactly where these gates sit. Gatingpdfwould be the wrong layer.Type of change
Checklist
Fixes #835)node test-all.mjslocally — 231 passed, 0 failed (16 pre-existing warnings: cv-sync without user data + README translation false-positives, unrelated)modes/*.md,test-all.mjs)Summary by CodeRabbit
Release Notes
New Features
Tests