fix(eval): make the harness runnable on Windows and price runs correctly

[akumrazor]

The L1 harness could not run on Windows at all, and the cost preflight was wrong twice over for any model outside the two in the pricing table. Found while pointing the harness at claude-opus-5.

Portability

render-bridge.mjs passed a filesystem path to dynamic import(). On POSIX a bare absolute path happens to work; on Windows an absolute path like E:\source\... parses as the scheme e: and is rejected outright:

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Received protocol 'e:'

Fixed with pathToFileURL().

anthropic-client.mjs hardcoded ~/.claude/local/claude and never passed it down — cci.py reads its own CCI_CLAUDE_BIN, which nothing set. npm/global installs land elsewhere (e.g. ~/.local/bin/claude), so the harness failed with a confusing exited 1 rather than a missing-binary error. Now resolves env → legacy path → PATH, and passes it to the shim.

cci.py drives the interactive TUI through a pty, which pexpect only provides on POSIX — pexpect.spawn does not exist on win32, so the shim can never run there. Added a headless claude -p transport, automatic on win32 and forcable anywhere with CCI_HEADLESS=1. Bonus: -p returns a real server usage block that the TUI panels do not, so accounting improves on that path.

Cost estimator

No entry for claude-opus-5 / claude-fable-5, and MODELS[model] ?? MODELS[DEFAULT_MODEL] silently fell back to another model's rates behind a $?/Mtok label — a confidently wrong dollar figure. Added both, plus derived cache read/write rates, and the unknown-model path now says so instead of guessing.

printCostEstimate assumed 2 calls per block while L1 runs one per variant, understating the bill 5.5x at the current 11. It now takes the real count. eval-l1-ocr.mjs was scaling externally to compensate, so that is removed — otherwise the two would double-apply.

Before / after for a 20-block L1 sweep on claude-opus-5:

- Pricing: $?/Mtok input, $?/Mtok output
- L1 OCR Fidelity (20 blocks x 2 calls)
-    API calls: 40      Estimated cost: $1.4898

+ Pricing: $5/Mtok input, $25/Mtok output
+          $0.50/Mtok cache read, $6.25 (5m) / $10.00 (1h) cache write
+ L1 OCR Fidelity (20 blocks x 11 variants)
+    API calls: 220     Estimated cost: $13.6568

The variant names moved to eval/lib/variant-names.mjs so the estimator can count them without importing (and thereby running) the eval. eval-l1-ocr.mjs asserts its VARIANTS array matches at startup, so a variant added in one place and not the other fails loudly instead of drifting back into exactly the bug this fixes.

run-eval.mjs combined report

Step 3 read baselineAgg / reflowAgg, fields that disappeared when L1 became an N-variant sweep — so the orchestrator crashed with Cannot read properties of undefined even on a dry run. Rebuilt over perVariant, now emitting a row per variant with deltas against baseline.

Verification

Full pipeline runs end to end on Windows (dry run and live). Drift guard verified by injecting a bogus variant name — it fails with a diff of both lists. Estimator cross-checked between the orchestrator and the L1 spend gate for both the full sweep and a --variants subset; the two agree, confirming no double-scaling.

🤖 Generated with Claude Code