Skip to content

feat(oferta,auto-pipeline): gate evaluation on a step-0 liveness check #939

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
maxmilian wants to merge 1 commit into from
+40 −1
Closed

feat(oferta,auto-pipeline): gate evaluation on a step-0 liveness check #939

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions modes/auto-pipeline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ When the user pastes a JD (text or URL) without an explicit sub-command, execute

If the input is a **URL** (not pasted JD text), follow this strategy to extract the content:

**Liveness gate (first):** Confirm the URL is live before any evaluation (#835). While navigating to read the JD, check for dead-posting evidence — 404/410, "position closed / filled / no longer accepting applications", a hard redirect to a generic careers or search page, or a page with no JD. If the posting is dead, **stop here**: do not run Step 1 evaluation, write a report, or generate a PDF. Mark the entry in `data/pipeline.md` as `- [x] ~~Company | Role~~ — posting expired` and tell the user the link is dead. The headless `check-liveness.mjs <url>` script is the fallback when Playwright MCP is unavailable.

**Priority order:**

1. **Playwright (preferred):** Most job portals (Lever, Ashby, Greenhouse, Workday) are SPAs. Use `browser_navigate` + `browser_snapshot` to render and read the JD.
Expand Down
16 changes: 15 additions & 1 deletion modes/oferta.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,21 @@

When the candidate pastes a job (text or URL), ALWAYS deliver the 7 blocks (A-F evaluation + G legitimacy):

## Step 0 — Archetype Detection
## Step 0 — Liveness gate

When the candidate provides a **URL** (not pasted JD text), confirm the posting is still live before spending any evaluation tokens. Dead links must be caught here — not after a full A-G report and a tailored CV have already been generated for a 404 (#835).

1. `browser_navigate` to the URL and take a `browser_snapshot` (this snapshot is reused by Block G — Posting Freshness).
2. Read the URL, page title, JD body, and any closed/expired signals:
- **active evidence:** role title + a real job description + an apply/submit path
- **dead evidence:** 404/410, "position closed / filled / no longer accepting applications", a hard redirect to a generic careers or search page, or a page with only nav/footer and no JD
3. If the posting is dead, **stop before Block A** — do not evaluate, write a report, or generate a CV. Mark the entry in `data/pipeline.md` as `- [x] ~~Company | Role~~ — posting expired` and tell the candidate the link is dead.
4. If liveness cannot be verified (no Playwright, or navigation blocked by a challenge/403), say so and ask the candidate to confirm the posting is active before continuing. The headless `check-liveness.mjs <url>` script is the fallback when Playwright MCP is unavailable.
5. If the input is pasted JD text (no URL), skip this gate.

Do not continue to Step 1 until liveness is resolved.

## Step 1 — Archetype Detection

Classify the job into one of the 6 archetypes (see `_shared.md`). If it is a hybrid, indicate the 2 closest ones. This determines:
- Which proof points to prioritize in block B
Expand Down
23 changes: 23 additions & 0 deletions test-all.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -512,6 +512,29 @@ if (
fail('apply mode missing liveness/role-match preflight gate');
}

// #835 — the eval side (oferta / auto-pipeline) must gate on URL liveness as
// step 0, so a dead link is caught before a full A-G report + tailored CV.
const ofertaMode = readFile('modes/oferta.md');
if (
ofertaMode.includes('## Step 0 — Liveness gate') &&
ofertaMode.includes('stop before Block A') &&
ofertaMode.includes('Do not continue to Step 1 until liveness is resolved')
) {
pass('oferta mode gates evaluation on a step-0 liveness check');
} else {
fail('oferta mode missing the step-0 liveness gate');
}

const autoPipelineMode = readFile('modes/auto-pipeline.md');
if (
autoPipelineMode.includes('Liveness gate (first)') &&
autoPipelineMode.includes('do not run Step 1 evaluation')
) {
pass('auto-pipeline mode gates evaluation on URL liveness');
Comment on lines +518 to +533

@coderabbitai coderabbitai Bot Jun 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Strengthen liveness-gate assertions with ordering checks.

These checks only assert substring presence; they can pass even if the gate text is moved below Step 1. Validate relative order (Step 0/Liveness gate appears before Step 1) to enforce the contract.

Suggested hardening 
 const ofertaMode = readFile('modes/oferta.md');
+const ofertaStep0 = ofertaMode.indexOf('## Step 0 — Liveness gate');
+const ofertaStep1 = ofertaMode.indexOf('## Step 1 — Archetype Detection');
 if (
-  ofertaMode.includes('## Step 0 — Liveness gate') &&
+  ofertaStep0 !== -1 &&
+  ofertaStep1 !== -1 &&
+  ofertaStep0 < ofertaStep1 &&
   ofertaMode.includes('stop before Block A') &&
   ofertaMode.includes('Do not continue to Step 1 until liveness is resolved')
 ) {
   pass('oferta mode gates evaluation on a step-0 liveness check');
 } else {
   fail('oferta mode missing the step-0 liveness gate');
 }

 const autoPipelineMode = readFile('modes/auto-pipeline.md');
+const autoStep0 = autoPipelineMode.indexOf('## Step 0 — Extract JD');
+const autoStep1 = autoPipelineMode.indexOf('## Step 1 — A-G Evaluation');
+const autoGate = autoPipelineMode.indexOf('Liveness gate (first)');
 if (
-  autoPipelineMode.includes('Liveness gate (first)') &&
+  autoStep0 !== -1 &&
+  autoStep1 !== -1 &&
+  autoGate !== -1 &&
+  autoStep0 <= autoGate &&
+  autoGate < autoStep1 &&
   autoPipelineMode.includes('do not run Step 1 evaluation')
 ) {
   pass('auto-pipeline mode gates evaluation on URL liveness');
 } else {
   fail('auto-pipeline mode missing the liveness gate before evaluation');
 }
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test-all.mjs` around lines 518 - 533, The current assertions only check
substring presence; update the tests for ofertaMode and autoPipelineMode to also
verify ordering by using index comparisons: ensure ofertaMode.indexOf('Step 0')
and/or ofertaMode.indexOf('Liveness gate') are >= 0 and less than
ofertaMode.indexOf('Step 1') before calling pass/fail, and similarly ensure
autoPipelineMode.indexOf('Liveness gate (first)') and/or
autoPipelineMode.indexOf('do not run Step 1 evaluation') are found and occur
before the indexOf('Step 1') in autoPipelineMode; use the existing variables
ofertaMode and autoPipelineMode and keep the same pass/fail calls but only pass
when both presence and correct order are satisfied.

} else {
fail('auto-pipeline mode missing the liveness gate before evaluation');
}

// ── 9. LOCAL PARSER CONTRACT ────────────────────────────────────

console.log('\n9. Local parser contract');
Expand Down
Loading