regression gate 對真實 regression 回報 exit 0（fail-open）：非有限數 + tolerance 從不被渲染

[kiki830621]

Problem

regression gate 會對真實的 regression 回報 exit 0。 這是 fail-open——gate 在該擋的
時候說通過，比任何 log 顯示問題嚴重一個量級。

發現於 #132（#117）的 verify：security lens 找到 NaN 路徑，Devil's Advocate 隨後證明
NaN 只是其中一條，且 security 建議的修法無效。

路徑一：非有限數（security lens 實測）

json.load 接受 NaN / Infinity / 1e999。

golden = NaN   →   diff = actual - golden = NaN
                   NaN > tol  →  False      →  「✓ 通過」

measured 0.99 對 golden 0.10，gate 印出 ✓ all corpora within tolerance 並 exit 0。

DA 實測否決了「三行就能修」的說法：json.load(parse_constant=...) 不能關閉這個
class——{"golden": "NaN"} 作為 JSON 字串根本不觸發 parse_constant，而後續的
float("NaN") / float("1e999") 仍然產生 nan / inf。

路徑二：tolerance 從不被渲染（DA 發現，更難察覺）

pass 行印的是 golden / measured / diff，但不印 tolerance——閾值只在失敗行出現。
也就是說：稽核一個「通過」時，唯一能揭穿它的數字恰好被隱藏。

tolerance: 1e308  （或在 diff 裡夾帶一個看似無害的 0.02 → 0.9）
→ ✓ jfk [en] wer: golden 0.0500 → measured 0.9900 (+0.9400)   ← 與真正通過無法區分
→ exit 0

此值通過所有現有防線：Swift schema test 的 tolerance > 0、JSONSerialization、
parse_constant、math.isfinite()。

Type

bug

Expected

gate 對「真實 regression」不得 exit 0。至少需要：

1. 數值健全性：golden / tolerance / error_rate 必須是有限數且落在合理範圍
   （math.isfinite() + 上界；注意字串形式的 "NaN" 也要擋）
2. tolerance 上界：一個大到能吞下任何 regression 的容差本身就該是 gate error
3. pass 行渲染 tolerance：稽核一個通過時，判準必須可見。
   ⚠️ 此項會改變 happy-path 輸出，與 baseline-compare.py：驗證 metric + render 邊界跳脫控制字元 (#117) #132 的 byte-identical 回歸鎖衝突，需同時更新該測試

Impact

scripts/lib/baseline-compare.py、Tests/BestASRKitTests/RegressionBaselineTests.swift
（含 #132 建立的 byte-identical 鎖）。

為什麼不在 #132 修

DA 裁決：F3 的 code 落在 scope 規則之外（既有缺陷，非 #132 造成或揭露），且
「categorically worse + 三行可修」的例外論證在自己的前提上失敗——它不是三行，
而且那三行沒用。#132 只修它自己弄假的文字（CHANGELOG 原本斷言 float() 在做驗證）。

命名

DA 明確要求本 issue 的 scope 是**「gate 對真實 regression 回報 exit 0」**，
而非「NaN 處理」——NaN 只是其中一條路徑，tolerance 那條不涉及任何非有限數。

Source: /idd-verify PR #132 — security lens F3（NaN）+ Devil's Advocate（否決該修法、
並發現 tolerance 未渲染這條獨立路徑）。

[kiki830621]

Verify (via PR #140)

Result: FAIL — 2 blocking, 5 follow-up. Engine: 4 lens Agents + sequenced Devil's Advocate (opus) + Codex gpt-5.6-sol — 6 independent verifiers, two model families. Diff-freshness gate PASS at 7c84d28.

Full report: #140 (comment)

Every atomic requirement in this issue is FULLY addressed and was independently reproduced, and both mutation-testing claims in the PR body were verified exactly (5 and 7, matching in composition as well as total). The umbrella requirement — 「gate 對『真實 regression』不得 exit 0」 — is PARTIALLY addressed:

• F1 GOLDEN_MAX = 2.0 leaves the algebraic twin of the bounded tolerance open. Reproduced against the real committed baseline with one field changed: a 100%-WER corpus prints ✓ and the run exits 0. golden: 0.5 suffices — no extreme value needed.
• F2 The empty-set fix closes cardinality 0, not 1-of-N. Both sides of the comparison derive from the same baseline.json, so no independent expected-corpus anchor exists: deleting 19 of 20 entries yields all 1 corpora within tolerance, exit 0 — by pure deletion, with no unusual value at all.

This issue's warning about a conflict with #132's byte-identical lock was checked by exhaustion and is wrong; the PR's rebuttal is correct.

The report also records an adjudicated severity dispute (the four lenses split CRITICAL / HIGH / MEDIUM / MEDIUM-non-blocking on F1) and an honest scope limit of the adversarial fuzz that explains why F2 came only from the cross-model leg.

[kiki830621]

Verify Round 2 (via PR #140)

Result at 7a4b2ea: FAIL — 1 blocking (adjudicated) + 1 record-integrity item that could not be repaired after merge. Engine: 4 lens Agents + sequenced Devil's Advocate (opus) + Codex gpt-5.6-sol — 6 verifiers, two model families. The Codex leg needed three attempts; the pattern of network failures on this PR's larger diffs is recorded in the report.

Full report: #140 (comment)

Both round-1 blockers were closed. This issue's three enumerated requirements are FULLY addressed and were independently reproduced.

The round turned on one disagreement, 5–1: four lenses and the Devil's Advocate's first pass ruled F2 CLOSED; Codex ruled PARTIALLY, blocking. The Devil's Advocate re-derived the argument and overturned itself and all four lenses — the requirement lives in regression-gate.sh, the layer that knows which baseline it is running, and that layer had no invariant at all. Its account of the error is the part worth keeping: every one of us verified that the named attack fails, and then treated the surviving configurations as residuals — the same move round 1 had already overturned on F1, applied one round later to F2.

It also overturned Codex's proposed remedy (making the stdin filter fail-closed would break its contract and eleven tests) in favour of Codex's own second branch, and corrected its own boolean-payload conclusion after Codex found a case its probe grid's shape had masked.

Fixes are in 39a9a46: the wrapper invariant, the test split so the permissive half is no longer the whole specification, a wiring test that catches the one-token typo which used to revert the guard with a green suite, both verdict-flipping boolean payloads, and scripts/mutation-check.sh so the non-vacuity counts can be re-run by a reader holding only the diff. 474 tests green; every guard load-bearing.

Still deferred here: coupling goldens to the provenance record. Carry the measured window — jfk, committed golden 0.0000, inflatable to 0.48, admitting a 50 % WER on the canonical clean sample with everything else honest.