fix(eval): agent cost was the wrong model's price, and the budget was the wrong unit

[sunholo-voight-kampff]

Found while confirming OpenAI's 2026-07-30 GPT-5.6 price cut. Three defects, one new tally.

1. Agent cost billed the wrong model

Executors that compute cost from tokens used their own hardcoded table, so the codex harness billed every model it ran at gpt-5-codex's $1.25/$10 per 1M. gpt5-6-luna and gpt5-4-mini banked at identical rates despite really costing $0.20/$1.20 and $0.75/$4.50.

Task.Pricing now carries per-model rates; ResolveCostModel prefers them and falls back to the executor's table. A present-but-zero rate is honoured, so free local models aren't billed at cloud prices.

The cost-kill path was already per-model correct — which meant banked cost and kill threshold came from two different price tables:

luna/graph_bfs (256,987 in / 2,136 out)
banked cost_usd	$0.34259 (codex rates) — above the $0.30 cap
budget saw	$0.26980 (luna rates) — under the cap
outcome	PASS

2. cost_usd never said whether anyone was charged

codex (auth_mode: chatgpt) and claude (OAuth) run on subscriptions, so their dollars are list-price equivalents — yet they summed with metered OpenRouter/Vertex spend under one label, directly beneath a v1.0 KPI whose numerator is attributable metered dollars.

Each executor now classifies its own auth lane. Banked as cost_provenance, stored on chain stages (schema v17), and split by ClassifyStageCost into a subscription status that CostRollup keeps out of TotalKnownCost. Absent labels read as unknown, never metered — nothing is backfilled by guess.

3. The budget was denominated in the wrong unit

A dollar ceiling buys tokens in inverse proportion to price, so nominally-uniform $0.30/$0.50 caps spanned 24× in real work:

model	before	after
opencode-or-deepseek-v4-flash	3.40M tok	3.00M
claude-sonnet-4-6 (anchor)	0.14M	3.00M
claude-haiku-4-5	0.25M	3.00M
gpt5-6-luna	1.20M	3.00M

claude-sonnet-4-6 — the longitudinal anchor everything else is scored against — was the most starved model in the suite. It had no budgets: block at all and fell through to a formula that buys almost nothing at $3/$15. A suite asking "does the agent loop rescue weak models" was giving the weakest the most rope.

budgets.max_tokens_per_bench sets one shared 3.0M ceiling. Subscription lanes gate on tokens alone and now sit identical; their dollar caps were raised only so the token gate binds — not a spend increase, those lanes aren't billed. Metered opencode lanes keep their dollar ceiling as a real spend control with tokens as a second bound.

Task.MaxTokensPerBench had been plumbed to executor.Task since M-EVAL-OS-LONGITUDINAL, but only opencode ever honoured it. codex and claude now enforce it too.

4. End-of-run cost tally

eval-suite now prints per-model cost then totals split by provenance. METERED (actually billed) is the only line that answers "what did this cost us".

Verification

Build, go vet, make check-boundaries, and tests across executor, eval_harness, observatory, eval_analysis, coordinator, server and storage. Tally rendered against the real v0.30.0 baseline — 2,090 rows, all correctly classified unknown (they predate the label).

Reviewer notes

• ⚠️ The claude dollar caps ($10.80, $3.60) are free on this rig's OAuth but become real per-benchmark exposure under AILANG_AUTH_MODE=apikey in cloud dispatch. Flagged inline at both sites.
• Metered lanes still span 15.5× in work — now a stated policy choice, not an accident. Equalizing would cost ~$260/run on glm-5-2.
• Existing v0.30.0 rows keep their wrong cost_usd; correcting them would mean fabricating numbers, so the annotation route was taken (same call as CAVEATS.md).

🤖 Generated with Claude Code

[sonarqubecloud]

Quality Gate failed

Failed conditions
63.4% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

[sunholo-voight-kampff]

Mission-loop triage (iteration 150). This PR is an orphan: it has zero mentions in the V1 mission charter and zero in the mission log (control: #544 has 4 log mentions, so the search sees positives). It was opened 2026-07-31, last touched 11:04 the same day, and nothing has referenced it since — so it was never picked up by an iteration and no one is waiting on it.

Its purpose is still live — I checked rather than assuming. The standing rule here is that an OPEN + long-untouched PR is evidence toward superseded, so I measured whether dev had solved this independently. It has not:

symbol / file	hits at dev
ResolveCostModel	0
CostProvenance / cost_provenance	0
internal/eval_harness/cost_tally.go	absent
internal/executor/codex/cost.go	absent

Control: internal/executor/cost.go (a file this PR only modifies) does exist, so the absences are measurements rather than a mistyped path.

So the two defects it fixes are still present: agent cost banked from the executor's hardcoded table rather than per-model rates, and cost_usd summing subscription list-price-equivalents together with genuinely metered spend under one label.

State: MERGEABLE: CONFLICTING, and the branch is 125 commits behind dev (2 commits ahead). It cannot be merged as-is, and the surface it touches (47 files across eval_harness, executor/*, observatory, storage/firestore) has moved a lot in those 125 commits — notably the v0.32.0 confidence-gating work and the v0.33.0 recorded-stream changes.

Not actioned this iteration — a triage sweep never outranks the queue head by itself, and this is a rebase-and-revalidate job, not a merge. Filed as a queue row in the charter so it stops being invisible to the loop.

🤖 Generated with Claude Code