feat(oferta,auto-pipeline): gate evaluation on a step-0 liveness check#939
Closed
maxmilian wants to merge 1 commit into
Closed
feat(oferta,auto-pipeline): gate evaluation on a step-0 liveness check#939maxmilian wants to merge 1 commit into
maxmilian wants to merge 1 commit into
Conversation
The Level 3 scan appends WebSearch hits to pipeline.md with no liveness check, and the eval side never re-verifies: oferta / auto-pipeline run the full A-G evaluation, write a report, and generate a tailored CV before anyone discovers the posting 404s. One reporter burned a session on 17/25 dead links, including a top-scored offer whose company had rebranded months earlier. scan (step 7.5) and apply (santifer#887 preflight) already gate on liveness; this closes the remaining eval-side gap. oferta gets a Step 0 liveness gate (renumbering Archetype Detection to Step 1, which is also where Block G's snapshot reference now correctly points), and auto-pipeline's Step 0 verifies the URL before extraction. On 404/expired/closed evidence both stop before any LLM work, mark the pipeline entry dead, and fall back to check-liveness.mjs when Playwright MCP is unavailable. Closes santifer#835
📝 WalkthroughWalkthroughThe PR adds a liveness verification gate at the beginning of the URL-based job evaluation pipeline in ChangesLiveness Gate for Dead Link Detection
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
Welcome to career-ops, @maxmilian! Thanks for your first PR. A few things to know:
We'll review your PR soon. Join our Discord if you have questions. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test-all.mjs`:
- Around line 518-533: The current assertions only check substring presence;
update the tests for ofertaMode and autoPipelineMode to also verify ordering by
using index comparisons: ensure ofertaMode.indexOf('Step 0') and/or
ofertaMode.indexOf('Liveness gate') are >= 0 and less than
ofertaMode.indexOf('Step 1') before calling pass/fail, and similarly ensure
autoPipelineMode.indexOf('Liveness gate (first)') and/or
autoPipelineMode.indexOf('do not run Step 1 evaluation') are found and occur
before the indexOf('Step 1') in autoPipelineMode; use the existing variables
ofertaMode and autoPipelineMode and keep the same pass/fail calls but only pass
when both presence and correct order are satisfied.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 85704811-d116-45a2-8c17-59719417b47c
📒 Files selected for processing (3)
modes/auto-pipeline.mdmodes/oferta.mdtest-all.mjs
Comment on lines
+518
to
+533
| if ( | ||
| ofertaMode.includes('## Step 0 — Liveness gate') && | ||
| ofertaMode.includes('stop before Block A') && | ||
| ofertaMode.includes('Do not continue to Step 1 until liveness is resolved') | ||
| ) { | ||
| pass('oferta mode gates evaluation on a step-0 liveness check'); | ||
| } else { | ||
| fail('oferta mode missing the step-0 liveness gate'); | ||
| } | ||
|
|
||
| const autoPipelineMode = readFile('modes/auto-pipeline.md'); | ||
| if ( | ||
| autoPipelineMode.includes('Liveness gate (first)') && | ||
| autoPipelineMode.includes('do not run Step 1 evaluation') | ||
| ) { | ||
| pass('auto-pipeline mode gates evaluation on URL liveness'); |
There was a problem hiding this comment.
Strengthen liveness-gate assertions with ordering checks.
These checks only assert substring presence; they can pass even if the gate text is moved below Step 1. Validate relative order (Step 0/Liveness gate appears before Step 1) to enforce the contract.
Suggested hardening
const ofertaMode = readFile('modes/oferta.md');
+const ofertaStep0 = ofertaMode.indexOf('## Step 0 — Liveness gate');
+const ofertaStep1 = ofertaMode.indexOf('## Step 1 — Archetype Detection');
if (
- ofertaMode.includes('## Step 0 — Liveness gate') &&
+ ofertaStep0 !== -1 &&
+ ofertaStep1 !== -1 &&
+ ofertaStep0 < ofertaStep1 &&
ofertaMode.includes('stop before Block A') &&
ofertaMode.includes('Do not continue to Step 1 until liveness is resolved')
) {
pass('oferta mode gates evaluation on a step-0 liveness check');
} else {
fail('oferta mode missing the step-0 liveness gate');
}
const autoPipelineMode = readFile('modes/auto-pipeline.md');
+const autoStep0 = autoPipelineMode.indexOf('## Step 0 — Extract JD');
+const autoStep1 = autoPipelineMode.indexOf('## Step 1 — A-G Evaluation');
+const autoGate = autoPipelineMode.indexOf('Liveness gate (first)');
if (
- autoPipelineMode.includes('Liveness gate (first)') &&
+ autoStep0 !== -1 &&
+ autoStep1 !== -1 &&
+ autoGate !== -1 &&
+ autoStep0 <= autoGate &&
+ autoGate < autoStep1 &&
autoPipelineMode.includes('do not run Step 1 evaluation')
) {
pass('auto-pipeline mode gates evaluation on URL liveness');
} else {
fail('auto-pipeline mode missing the liveness gate before evaluation');
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@test-all.mjs` around lines 518 - 533, The current assertions only check
substring presence; update the tests for ofertaMode and autoPipelineMode to also
verify ordering by using index comparisons: ensure ofertaMode.indexOf('Step 0')
and/or ofertaMode.indexOf('Liveness gate') are >= 0 and less than
ofertaMode.indexOf('Step 1') before calling pass/fail, and similarly ensure
autoPipelineMode.indexOf('Liveness gate (first)') and/or
autoPipelineMode.indexOf('do not run Step 1 evaluation') are found and occur
before the indexOf('Step 1') in autoPipelineMode; use the existing variables
ofertaMode and autoPipelineMode and keep the same pass/fail calls but only pass
when both presence and correct order are satisfied.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds a step-0 URL liveness gate to
ofertaandauto-pipeline, so a dead posting is caught before the full A-G evaluation, the report, and the tailored CV are generated — closing the eval-side half of the liveness-gate class.Related issue
Closes #835
Why
The Level 3 (WebSearch) scan appends hits to
pipeline.mdas pending entries, and the eval side never re-verifies.oferta/auto-pipelinerun all of blocks A-G, write a report, and generate a tailored PDF CV before anyone discovers the URL 404s. The reporter burned a session on 17/25 dead links, including the session's top-scored offer (4.3/5) whose company had rebranded months earlier — full evaluation + PDF generated on phantom data.scanalready gates Level 3 results (step 7.5,scan --verify) andapplygates at its preflight (#887). This was the remaining gap: the eval side had no step-0 check, exactly as the maintainer noted in the issue thread.What users will see
ofertaor triggeringauto-pipelinenow verifies the posting is live first. On dead-posting evidence (404/410, "position closed / filled / no longer accepting applications", a hard redirect to a generic careers page, or a page with no JD), it stops before any A-G work — no report, no CV — marks the entry- [x] ~~Company | Role~~ — posting expiredinpipeline.md, and tells the user the link is dead.check-liveness.mjs <url>is the documented headless fallback when Playwright MCP isn't available.Surface area
Two mode files (
modes/oferta.md,modes/auto-pipeline.md) + integrity tests intest-all.mjs. System Layer only; no user-layer files. Inoferta.md, the new gate takes Step 0 andArchetype Detectionmoves to Step 1 — Block G's "snapshot already captured in Step 0" now correctly points at the gate (which is where thebrowser_snapshotis taken).Screenshots
N/A — mode-instruction (prompt) change.
Bug fix verification
test-all.mjsgains two integrity assertions (mirroring the existing apply-preflight check) so the gate can't silently disappear:ofertamust carry## Step 0 — Liveness gate+stop before Block A+Do not continue to Step 1 until liveness is resolved;auto-pipelinemust carryLiveness gate (first)+do not run Step 1 evaluation. Both go red without the mode-file changes and green with them.Validation
node test-all.mjs --quick→ 231 passed, 0 failed (the 16 warnings are pre-existingsantifer.iopersonal-data false-positives in README.pl/ar/ua + the cv-sync no-user-data check — unrelated).Note on scope (translations)
Scoped to the two English modes named in the issue. The Arabic
modes/ar/fursah.mdhas the sameStep 0and would benefit from the same gate — happy to extend it here or leave it for a follow-up i18n PR, your call.Checklist
node test-all.mjs --quick) and all tests pass (0 failed)Summary by CodeRabbit
New Features
Tests