Fix terminal-hook session poisoning

.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "gpt-5-6-router",
-  "version": "0.1.0+codex.20260713131040",
+  "version": "0.1.0+codex.20260714074631",
   "description": "Autorouter automatically selects the right GPT-5.6 model, effort, and bounded agent plan for each Codex task.",
   "author": {
     "name": "Alchemist"

.github/workflows/ci.yml

@@ -31,6 +31,8 @@ jobs:
       - name: Run effective-config checker tests
         working-directory: scripts
         run: python -m unittest -v test_check_hooks test_standalonectl
+      - name: Run capability benchmark harness tests
+        run: python -m unittest -v benchmarks.test_autorouter_bench
       - name: Run deterministic routing fuzz
         run: python scripts/fuzz_router.py
       - name: Revalidate source integrity after tests

CHANGELOG.md

@@ -25,3 +25,7 @@
 - Accepted substantive multiline Evidence, Verification, and Uncertainty sections without weakening empty-section rejection.
 - Added five-run native delegation stability acceptance and a final exact-installed-cache replay.
 - Added cross-platform CI bytecode hygiene and a clean-room release-archive lifecycle gate.
+- Made `Stop` and `SubagentStop` strictly advisory: terminal hooks record unmet routing, attestation, and verification debt as `FAILED_OPEN` and can no longer emit a hard-block response that Codex may replay as a poisoned user message.
+- Front-loaded required-agent coverage, lifecycle attestation, final-mutation verification, service-tier inheritance, and no-recursion rules into trusted `UserPromptSubmit` and `SubagentStart` context.
+- Added `routerctl sanitize --session PATH|--all` for conservative offline repair of already-poisoned JSONL sessions, including nested compacted history, exact backups, atomic replacement, idempotency, and fail-closed malformed-input handling.
+- Added a release guard that rejects any reintroduced `{"decision":"block"}` terminal response, plus regression coverage and an upstream Codex replay bug report.

README.md

@@ -119,7 +119,7 @@ Standalone mode deliberately does not claim plugin-hook enforcement. See [standa
 - **Protected floors:** Token Saver never discounts security, payments, privacy, production, destructive actions, or other high-risk work.
 - **No silent speed changes:** Fast/Standard is inherited, not selected by Autorouter.
 - **Post-mutation verification:** the root agent remains responsible for checking the final mutated state; verification debt is recorded without interrupting Stop.
-- **Non-recursive completion:** verification state alone never injects hook feedback; hard routing continuations remain one-shot and duplicate Stop events are idempotent.
+- **Session-safe completion:** Stop and SubagentStop are advisory-only, record unmet gates as `FAILED_OPEN`, and never emit a replayable hard-block message; duplicate terminal events are idempotent.
 - **Upgrade-resilient hooks:** open tasks can locate the newest installed cache after their original immutable cache is replaced.
 - **Conflict-safe activation:** only six namespaced agent files are managed, with checksums, atomic replacement, rollback, crash recovery, and legacy-role migration.
 - **Prompt privacy:** routing ledgers contain no prompt content or prompt-derived hashes.
@@ -145,6 +145,7 @@ Standalone mode deliberately does not claim plugin-hook enforcement. See [standa
 | [Known limitations](docs/LIMITATIONS.md) | Runtime, hooks, Fast, and custom-agent boundaries |
 | [Privacy](docs/PRIVACY.md) | Local preference and ledger contents |
 | [Troubleshooting](docs/TROUBLESHOOTING.md) | Readiness, trust, recovery, and routing-profile issues |
+| [Upstream hook replay report](docs/UPSTREAM_CODEX_HOOK_REPLAY_BUG.md) | Reproduction and required Codex-side fix for poisoned terminal-hook messages |
 | [Real-host acceptance](docs/HOST_ACCEPTANCE.md) | Exact release smoke procedure |
 | [Release checklist](docs/RELEASE_CHECKLIST.md) | Automated and public-launch gates |
 | [Release evidence](docs/RELEASE_EVIDENCE.md) | Candidate-specific validation proof |
@@ -160,9 +161,30 @@ PYTHONDONTWRITEBYTECODE=1 python3 -m unittest -v test_router test_routerctl
 
 cd ../../../scripts
 PYTHONDONTWRITEBYTECODE=1 python3 -m unittest -v test_check_hooks test_standalonectl
+
+cd ..
+PYTHONDONTWRITEBYTECODE=1 python3 -m unittest -v benchmarks.test_autorouter_bench
+```
+
+The current suite covers 189 routing, lifecycle, and benchmark-harness tests plus 5,000 deterministic fuzz cases per CI job, including all three profiles, Extra High selection, stale-cache migration, terminal-hook poison prevention and offline repair, nested-tool verification, unobservable-result fail-open behavior, wrapped Stop-feedback idempotency, lifecycle ownership, concurrency, crash recovery, hook readiness, clean-room standalone installation, frozen capability-fixture validation, profile restoration, and end-to-end blind-panel claim boundaries.
+
+### Capability and efficiency benchmark
+
+Unit tests prove routing policy; they do not prove that routing improves a real
+task. [Autorouter Arena](benchmarks/README.md) runs frozen product-building
+cases against plain Codex, Token Saver, Balanced, and Max Quality from identical
+starters, then reports outcome quality, actual routing/subagent evidence,
+tokens, and elapsed time separately. An optional private GLM 5.2 + DeepSeek V4
+Pro blind panel can provide separated qualitative review during development;
+that reviewer machinery is not part of the runtime plugin.
+
+```bash
+python3 benchmarks/autorouter_bench.py route-audit
+python3 benchmarks/autorouter_bench.py prepare --output /tmp/autorouter-arena
 ```
 
-The current suite covers 171 routing and lifecycle tests plus 5,000 deterministic fuzz cases per CI job, including all three profiles, Extra High selection, stale-cache migration, nested-tool verification, unobservable-result fail-open behavior, wrapped Stop-feedback idempotency, lifecycle ownership, concurrency, crash recovery, hook readiness, and clean-room standalone installation.
+Paid A/B runs require explicit execution and reviewed-hook acknowledgements;
+the benchmark never spends tokens by default.
 
 ## License
 

benchmarks/README.md

@@ -0,0 +1,153 @@
+# Autorouter Arena
+
+Autorouter Arena is a controlled benchmark for the part unit tests cannot
+prove: whether Autorouter changes execution in a useful way.
+
+It measures three layers independently:
+
+1. **Outcome quality** — deterministic graders run against frozen task fixtures.
+2. **Routing behavior** — the router ledger is checked for the expected route,
+   orchestration shape, bounded depth, and worker coverage.
+3. **Efficiency** — Codex JSON events provide wall time and token usage for the
+   baseline, Token Saver, Balanced, and Max Quality runs.
+4. **Outsider review** — the same separated GLM 5.2 + DeepSeek V4 Pro panel used
+   by Spark Loops can privately review randomized, output-only artifacts.
+
+The suite deliberately includes work where different behavior should emerge:
+
+| Case | Capability under test | Expected Autorouter behavior |
+|---|---|---|
+| `locale_batch` | repeated, objectively checked transformation | two Luna leaves |
+| `webhook_race` | ambiguous concurrency debugging | Terra High unless the root already satisfies it |
+| `sqlite_migration` | reversible, expensive-to-get-wrong ownership | Sol ownership and migration/rollback evidence |
+| `parallel_audit` | independent security, accessibility, and data review | Terra evidence followed by one Sol owner |
+
+Route-only probes also exercise ordinary implementation, unclear multi-area
+work, protected security floors, profile changes, and high-volume delegation.
+
+## Start with a zero-cost route audit
+
+```bash
+python3 benchmarks/autorouter_bench.py route-audit
+```
+
+This calls the checked-in classifier directly. It does not call a model or
+spend tokens.
+
+## Prepare inspectable fixtures
+
+```bash
+python3 benchmarks/autorouter_bench.py prepare --output /tmp/autorouter-arena
+```
+
+Every case gets a clean starter repository, a `TASK.md`, and a deterministic
+grader. You can open any fixture in Codex and run the task manually.
+
+## Run the four controlled arms
+
+Real runs spend model tokens and execute Codex. They require both explicit
+flags so the cost and reviewed-hook bypass are never accidental:
+
+```bash
+python3 benchmarks/autorouter_bench.py run \
+  --output /tmp/autorouter-arena \
+  --execute \
+  --trust-reviewed-hooks
+```
+
+The baseline keeps the same root model, effort, and service tier but disables
+the Autorouter plugin and ignores routing rules. The other three arms set Token
+Saver, Balanced, and Max Quality in turn. Each arm starts from a fresh copy of
+the same fixture, run order rotates across repetitions, and the user's original
+Autorouter profile is restored even if the run fails.
+
+Defaults match the release test environment (`gpt-5.6-sol`, `high`,
+`priority`). Override them only when comparing another controlled root:
+
+```bash
+python3 benchmarks/autorouter_bench.py run \
+  --output /tmp/autorouter-arena \
+  --execute --trust-reviewed-hooks \
+  --root-model gpt-5.6-terra --root-effort medium --service-tier default
+```
+
+Run one case while iterating:
+
+```bash
+python3 benchmarks/autorouter_bench.py run \
+  --output /tmp/autorouter-arena \
+  --case locale_batch \
+  --execute --trust-reviewed-hooks
+```
+
+## Run the private outsider panel
+
+This is development tooling only. It is not installed as an Autorouter runtime
+gate and users never need it. After the four arms finish, configure the same
+active outside seats used by Spark Loops and run:
+
+```bash
+export ZAI_API_KEY=...
+export DEEPSEEK_API_KEY=...
+
+python3 benchmarks/autorouter_bench.py review \
+  --output /tmp/autorouter-arena \
+  --execute \
+  --acknowledge-external-review-cost
+```
+
+On the same Mac used by Spark Loops, the benchmark can reuse its existing
+reviewer credentials without printing or copying them into the repository:
+
+```bash
+python3 benchmarks/autorouter_bench.py review \
+  --output /tmp/autorouter-arena \
+  --execute \
+  --acknowledge-external-review-cost \
+  --reuse-spark-provider-keys
+```
+
+That explicit flag checks Spark Loops' macOS Keychain service first and its
+permission-restricted `~/.spark/.env` fallback second. Keys are held only in
+memory for their matching provider request and are never written to reports.
+
+Each reviewer receives opaque artifact ids, the task, deterministic check
+results, the candidate patch, and the candidate's final response. It does not
+receive model names, Autorouter profiles, routes, tokens, latency, or the blind
+mapping. The mapping is applied only after both reviewers return.
+
+A panel is official only at full `2/2` quorum. Partial or malformed panels are
+saved as diagnostic evidence and cannot declare a winner. Reviewer feedback is
+repair input for us; it never automatically mutates or promotes routing policy.
+
+## Read the result
+
+`report.md` and `report.json` are written under the output directory. A useful
+Autorouter result requires more than passing tasks:
+
+- quality is equal to or better than baseline;
+- routing behavior matches the case contract;
+- total tokens or elapsed time improve on cheaper/batch cases; and
+- deeper orchestration improves coverage on consequential cases enough to
+  justify its overhead.
+
+Do not treat one run as a model ranking. Run at least three repetitions before
+making cost or quality claims. The CLI records raw evidence so medians can be
+computed without relying on screenshots or memory.
+
+## Safety and limitations
+
+- `--trust-reviewed-hooks` passes Codex's hook-trust bypass only for the
+  isolated benchmark subprocess. Review the installed Autorouter hooks first.
+- Outsider review sends candidate patches and responses to Z.AI and DeepSeek.
+  Use only benchmark fixtures or code you are authorized to share with those
+  providers. The command refuses to run without explicit cost acknowledgement
+  and both provider keys.
+- Tasks run with `workspace-write` inside generated fixture directories.
+- Token totals depend on what the installed Codex JSON stream exposes. Missing
+  usage is reported as unavailable, never estimated.
+- Dollar cost is intentionally omitted. Token and service-tier evidence remain
+  comparable without baking changing prices into the repository.
+- A skill cannot replace the already-running root model. The benchmark makes
+  this visible: savings on a Sol root must come from useful bounded delegation,
+  not from claiming the root silently became Terra or Luna.