From 3f1e3e7193e9ebd117225be181e8d40257e550dd Mon Sep 17 00:00:00 2001
From: che cheng <kiki830621@gmail.com>
Date: Sun, 5 Jul 2026 23:15:02 +0800
Subject: [PATCH 1/4] =?UTF-8?q?feat:=20snapshot=20reality=20checks=20?=
 =?UTF-8?q?=E2=80=94=20staleness=20budget=20+=20fail-loud=20verify-live=20?=
 =?UTF-8?q?+=20monthly=20cron=20(#49)?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The guard chain locked README ⟷ sample but nothing checked sample ⟷ reality:
--verify-live only PRINTED drift warnings (exit 0 — a log nobody reads is not a
signal) and nothing bounded the snapshot's age, so `captured 2026-07-02` would
silently rot into an asserted number (#27/#29) with everything green.

- `--check-staleness [--max-age-days N=120] [--today ISO]`: pure date math on
  captured_at with an INJECTED today — the tests never read the wall clock (a
  wall-clock assertion would itself be the time bomb). Over budget / future
  captured_at / malformed dates → loud exit 1.
- `--verify-live` now exits 1 when any recorded ground_truth has drifted
  (behavior change: warn → fail-loud, consistent with the whole guard chain).
- `.github/workflows/freshness.yml`: monthly cron + manual dispatch runs both.
  Deliberately NOT PR CI — a time/network failure must never redden unrelated
  PRs; a red scheduled run is the maintainer's refresh signal. No ${{ }}
  expressions (injection-safe by construction).
- evals/look-up/README.md: § Refreshing the snapshot — the human procedure both
  error messages point to (context7's column is a recorded judgment; re-capture
  is deliberately a human act — stated, not hidden).

Tests (TDD, RED→GREEN): +4 staleness + 1 verify-live exit codes → 71 Python.
---
 .github/workflows/freshness.yml     | 37 ++++++++++++++++++++
 evals/look-up/README.md             | 34 ++++++++++++++++---
 evals/look-up/compare_context7.py   | 52 +++++++++++++++++++++++++++++
 evals/look-up/tests/test_compare.py | 51 ++++++++++++++++++++++++++++
 4 files changed, 170 insertions(+), 4 deletions(-)
 create mode 100644 .github/workflows/freshness.yml

diff --git a/.github/workflows/freshness.yml b/.github/workflows/freshness.yml
new file mode 100644
index 0000000..e035c91
--- /dev/null
+++ b/.github/workflows/freshness.yml
@@ -0,0 +1,37 @@
+# vs-context7 snapshot freshness (#49): the PR-gate guards (ci.yml) lock README ⟷ sample,
+# but nothing there checks sample ⟷ REALITY. This scheduled run is that reality check:
+#   1. --verify-live   — re-fetches the registry ground truth (LiveDocs' side; network)
+#                        and exits 1 if any recorded version has drifted;
+#   2. --check-staleness — exits 1 when captured_at exceeds the age budget (default 120d).
+# Deliberately NOT part of PR CI: a time/network-based failure must never redden PRs
+# unrelated to the chart. A red run here is the maintainer's refresh signal — procedure
+# in evals/look-up/README.md § Refreshing the snapshot. context7's side is a recorded
+# human judgment and structurally cannot be auto-re-verified (stated in the README).
+#
+# Injection surface: no ${{ }} expressions — static commands + stock actions only.
+name: vs-context7 freshness
+
+on:
+  schedule:
+    - cron: "17 3 1 * *"   # monthly, 1st @ 03:17 UTC (off-peak, non-round minute)
+  workflow_dispatch:
+
+jobs:
+  freshness:
+    name: snapshot reality check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install eval-harness dependencies
+        run: pip install -r evals/look-up/requirements.txt
+
+      - name: Registry drift check (LiveDocs-side ground truth)
+        run: python3 evals/look-up/compare_context7.py --verify-live
+
+      - name: Staleness budget (captured_at age)
+        run: python3 evals/look-up/compare_context7.py --check-staleness
diff --git a/evals/look-up/README.md b/evals/look-up/README.md
index 976d719..bb1edf1 100644
--- a/evals/look-up/README.md
+++ b/evals/look-up/README.md
@@ -104,7 +104,8 @@ registry as neutral ground truth.
 ```bash
 python3 evals/look-up/compare_context7.py            # print the table + tally
 python3 evals/look-up/compare_context7.py --json     # machine-readable
-python3 evals/look-up/compare_context7.py --verify-live   # warn if the snapshot drifted
+python3 evals/look-up/compare_context7.py --verify-live       # exit 1 if the registry moved past the snapshot
+python3 evals/look-up/compare_context7.py --check-staleness   # exit 1 if captured_at is over the age budget (120d)
 ```
 
 - Corpus: [`compare_corpus.yaml`](compare_corpus.yaml) — fast-moving libraries across
@@ -135,9 +136,9 @@ harness lets the data show it and **concedes** what context7 is good at:
   answer, not "context7 can't find the version." Per library the default is either
   *behind* (react, vite) or *version-less* (fastapi, pydantic, tokio, serde) — the
   sample records which.
-- **`--verify-live` is one-sided:** it re-fetches the registry (LiveDocs' side) and warns
-  on drift, but does not re-query context7. The context7 column is a dated snapshot;
-  refresh it by re-running both MCP tools.
+- **`--verify-live` is one-sided:** it re-fetches the registry (LiveDocs' side) and exits
+  non-zero on drift (#49), but does not re-query context7. The context7 column is a dated
+  snapshot; refresh it by re-running both MCP tools.
 - **D1 (this harness):** a dated recorded head-to-head. **D2 (future, not built):**
   a fully-automated live A/B that shells `claude -p` per tool — stronger but
   costly/flaky, so it stays out of the default path.
@@ -146,6 +147,31 @@ Result at capture (2026-07-02): context7's top-ranked default match reflected th
 release on **0 of 6** (behind for react/vite, version-less for fastapi/pydantic/tokio/serde);
 LiveDocs returned the live registry version each time.
 
+### Refreshing the snapshot (#49)
+
+The front-page claim is only as honest as the snapshot's `captured_at`. Two mechanical
+signals tell you it's time (both run monthly by `.github/workflows/freshness.yml`, and
+either can be run by hand):
+
+- `--verify-live` exits 1 → the registry has moved past a recorded `ground_truth`;
+- `--check-staleness` exits 1 → `captured_at` is over the age budget (default 120 days,
+  tunable via `--max-age-days`).
+
+Refresh procedure (deliberately a **human** act — context7's column is a recorded
+judgment, not queryable):
+
+1. Re-run both MCP tools per library in `compare_corpus.yaml`: LiveDocs `latest_version`,
+   and context7's resolve→query flow; judge whether context7's **top-ranked default**
+   match reflects the current release.
+2. Update `compare_context7_sample.json`: `ground_truth` + `livedocs.answer` verbatim,
+   `context7.answer`/`is_current`/`default_version`/`default_status` per the new
+   judgment, and bump `captured_at` to today.
+3. Regenerate the committed artifacts:
+   `python3 evals/look-up/compare_context7.py --emit-chart --write` then
+   `--emit-readme --write`.
+4. `python3 -m pytest evals/look-up/tests/` — the drift guards confirm README ⟷ sample
+   ⟷ SVG are in sync again; commit sample + SVG + README together.
+
 ## Scope / caveats
 
 - Measures a **sample**, not a proof of universal correctness — the corpus must
diff --git a/evals/look-up/compare_context7.py b/evals/look-up/compare_context7.py
index 1b2bc3b..28889ae 100644
--- a/evals/look-up/compare_context7.py
+++ b/evals/look-up/compare_context7.py
@@ -160,6 +160,15 @@ def _require_iso_date(value):
         )
 
 
+def staleness_days(captured_at, today):
+    """Days elapsed since the snapshot's captured_at (#49). Pure: `today` is INJECTED
+    (a datetime.date) so tests and the CLI's --today flag never read the wall clock —
+    a wall-clock assertion would itself become the time bomb this exists to catch.
+    Validates captured_at with the same ISO-calendar round-trip as the README guard."""
+    _require_iso_date(captured_at)
+    return (today - datetime.date.fromisoformat(captured_at)).days
+
+
 CHART_ASSET = "docs/assets/vs-context7-freshness.svg"
 
 
@@ -245,6 +254,15 @@ def main(argv=None):
                     help="emit the README vs-context7 block (SVG image embed + caption); compose with --write / --check")
     ap.add_argument("--emit-chart", action="store_true",
                     help="emit the per-library SVG chart asset (issue #50); compose with --write / --check")
+    ap.add_argument("--check-staleness", action="store_true",
+                    help="exit 1 if the snapshot's captured_at is older than --max-age-days (#49; "
+                         "run by the scheduled freshness workflow, NOT PR CI — a time-based failure "
+                         "must never redden unrelated PRs)")
+    ap.add_argument("--max-age-days", type=int, default=120,
+                    help="staleness budget in days for --check-staleness (default: 120)")
+    ap.add_argument("--today", default=None,
+                    help="ISO date override for --check-staleness (tests / reproducing a cron run); "
+                         "defaults to the wall clock")
     mode = ap.add_mutually_exclusive_group()
     mode.add_argument("--write", action="store_true",
                       help="with --emit-readme / --emit-chart: rewrite the target file in place")
@@ -263,6 +281,30 @@ def main(argv=None):
     results = sample.get("results", [])
     rows = [judge_row(e) for e in results]
 
+    if args.check_staleness:
+        captured = sample.get("captured_at")
+        try:
+            today = datetime.date.fromisoformat(args.today) if args.today else datetime.date.today()
+        except (ValueError, TypeError):
+            print(f"✗ --today must be an ISO date (YYYY-MM-DD); got {args.today!r}", file=sys.stderr)
+            return 1
+        try:
+            age = staleness_days(captured, today)
+        except ValueError as exc:
+            print(f"✗ {exc}", file=sys.stderr)
+            return 1
+        if age < 0:
+            print(f"✗ captured_at {captured} is in the FUTURE of {today} — clock or snapshot error",
+                  file=sys.stderr)
+            return 1
+        if age > args.max_age_days:
+            print(f"✗ snapshot is {age} days old (captured {captured}; budget {args.max_age_days}d) — "
+                  "the front-page competitive claim is rotting into an ASSERTED number (#27/#29). "
+                  "Refresh per evals/look-up/README.md § Refreshing the snapshot.", file=sys.stderr)
+            return 1
+        print(f"✓ snapshot age {age}d is within the {args.max_age_days}d budget (captured {captured})")
+        return 0
+
     if args.emit_chart:
         from chart import render_svg
         svg = render_svg(sample)
@@ -324,14 +366,24 @@ def main(argv=None):
         print("note: --verify-live re-checks the registry (LiveDocs' ground truth) only; "
               "context7's recorded judgments are a dated snapshot, not re-fetched here.",
               file=sys.stderr)
+        drifted = 0
         for e, r in zip(results, rows):
             lib, eco = e.get("library"), e.get("ecosystem")
             if not lib or not eco:
                 continue
             live = fetch_latest_version(lib, eco)
             if live and live != r["ground_truth"]:
+                drifted += 1
                 print(f"⚠ drift: {lib} recorded {r['ground_truth']} but registry now {live}",
                       file=sys.stderr)
+        # Fail LOUD on drift (#49): a scheduled run that only prints warnings to an
+        # unread log is not a signal. Exit 1 → the freshness workflow goes red → the
+        # maintainer refreshes the snapshot (procedure: evals/look-up/README.md).
+        if drifted:
+            print(f"✗ {drifted} of {len(results)} recorded ground truths have drifted — "
+                  "refresh the snapshot per evals/look-up/README.md § Refreshing the snapshot.",
+                  file=sys.stderr)
+            return 1
 
     t = tally(rows)
     if args.json:
diff --git a/evals/look-up/tests/test_compare.py b/evals/look-up/tests/test_compare.py
index d9a7450..71ae36c 100644
--- a/evals/look-up/tests/test_compare.py
+++ b/evals/look-up/tests/test_compare.py
@@ -223,6 +223,57 @@ def test_replace_readme_block_requires_exactly_one():
         replace_readme_block("no sentinels at all", "NEW")
 
 
+def test_staleness_days_pure():
+    # #49: pure date math with an INJECTED today — a wall-clock test would itself become
+    # the time bomb this feature exists to catch. Validates captured_at via the same
+    # ISO-calendar round-trip as the README guard.
+    import datetime
+    import pytest
+    from compare_context7 import staleness_days
+    assert staleness_days("2026-07-02", datetime.date(2026, 7, 5)) == 3
+    assert staleness_days("2026-07-02", datetime.date(2026, 7, 2)) == 0
+    with pytest.raises(ValueError):
+        staleness_days("not-a-date", datetime.date(2026, 7, 5))
+
+
+def test_check_staleness_fresh_via_main():
+    # Drive the real CLI: within budget → exit 0. `--today` injected so the test never
+    # reads the wall clock (sample captured_at = 2026-07-02).
+    from compare_context7 import main
+    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-07-10"]) == 0
+
+
+def test_check_staleness_over_budget_via_main():
+    # Over budget → exit 1 (fail-loud): the front-page claim has rotted into an
+    # asserted number and the scheduled run must go red.
+    from compare_context7 import main
+    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2027-07-10"]) == 1
+
+
+def test_check_staleness_boundary_future_and_bad_today():
+    # Boundary: age == budget is still WITHIN (fail only when age > budget);
+    # 2026-07-02 + 120d = 2026-10-30. A captured_at in the FUTURE of today is a
+    # clock/snapshot error → loud 1, never silently "fresh". Malformed --today → 1.
+    from compare_context7 import main
+    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-10-30"]) == 0
+    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-10-31"]) == 1
+    assert main(["--check-staleness", "--today", "2026-07-01"]) == 1   # captured_at in the future
+    assert main(["--check-staleness", "--today", "99-99-99"]) == 1     # malformed today
+
+
+def test_verify_live_exit_codes(monkeypatch):
+    # #49: --verify-live must FAIL LOUD (exit 1) when the registry has moved past the
+    # recorded ground truth — a scheduled run that only prints warnings to a log nobody
+    # reads is not a signal. Registry stubbed (no network in unit tests).
+    import oracle
+    from compare_context7 import main, load_sample
+    truths = {r["library"]: r["ground_truth"] for r in load_sample()["results"]}
+    monkeypatch.setattr(oracle, "fetch_latest_version", lambda lib, eco: truths[lib])
+    assert main(["--verify-live"]) == 0                     # registry == recorded → green
+    monkeypatch.setattr(oracle, "fetch_latest_version", lambda lib, eco: "999.0.0")
+    assert main(["--verify-live"]) == 1                     # drifted → red
+
+
 def test_sample_covers_corpus():
     # compare_corpus.yaml is the canonical library list; the recorded sample must
     # cover every one of them (so adding a lib to the corpus without recording its

From d5db1d0539029ee724263b437261c69077078fc7 Mon Sep 17 00:00:00 2001
From: che cheng <kiki830621@gmail.com>
Date: Sun, 5 Jul 2026 23:15:02 +0800
Subject: [PATCH 2/4] docs: sync test counts to 215 (144 Swift + 71 Python)
 (#49)

Via scripts/sync-test-counts.sh after the +5 staleness/verify-live tests.
Milestone prose left frozen.
---
 CLAUDE.md                  | 2 +-
 README.md                  | 4 ++--
 docs/wiki/Home-zh-TW.md    | 2 +-
 docs/wiki/Home.md          | 2 +-
 docs/wiki/Testing-zh-TW.md | 8 ++++----
 docs/wiki/Testing.md       | 8 ++++----
 6 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/CLAUDE.md b/CLAUDE.md
index 7b7858c..d2c9db3 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -19,7 +19,7 @@ Swift MCP server (engine) + `look-up` skill (routing) + Python evals.
 
 ```bash
 swift test                                   # 144 Swift tests (no network)
-python3 -m pytest evals/look-up/tests/   # 66 Python tests (no API calls)
+python3 -m pytest evals/look-up/tests/   # 71 Python tests (no API calls)
 python3 evals/look-up/compare_context7.py --verify-live   # freshness snapshot drift check
 make release-signed VERSION=vX.Y.Z           # build + sign + notarize + mcpb (needs DEVELOPER_ID/NOTARY_PROFILE)
 bash scripts/sync-wiki.sh                    # mirror docs/wiki/ -> GitHub wiki (after merge)
diff --git a/README.md b/README.md
index 19ce39c..57a5613 100644
--- a/README.md
+++ b/README.md
@@ -104,7 +104,7 @@ than hardcoding a fact), and it's statistical (N runs, rate thresholds).
 
 ```bash
 pip install -r evals/look-up/requirements.txt
-python3 -m pytest evals/look-up/tests/     # 66 harness unit tests (no API calls)
+python3 -m pytest evals/look-up/tests/     # 71 harness unit tests (no API calls)
 python3 evals/look-up/run_eval.py --dry-run
 python3 evals/look-up/run_eval.py --runs 5 # live baseline (real `claude -p` calls)
 ```
@@ -119,7 +119,7 @@ python3 evals/look-up/compare_context7.py --verify-live  # warn if the snapshot
 
 See [`evals/look-up/README.md`](evals/look-up/README.md) for the design of both, and
 the [Testing wiki page](https://github.com/PsychQuant/livedocs/wiki/Testing) for the full
-suite breakdown (210 tests: 144 Swift + 66 Python).
+suite breakdown (215 tests: 144 Swift + 71 Python).
 
 Status: shipped (v0.8.0; engine features below shipped v0.7.0, the skill's `look-up` rename +
 explicit invocation landed in v0.8.0). 9-ecosystem registry resolution, version pinning (npm/pypi/crates/go/rubygems), OpenAPI/GraphQL/CLI +
diff --git a/docs/wiki/Home-zh-TW.md b/docs/wiki/Home-zh-TW.md
index 348965a..b780b37 100644
--- a/docs/wiki/Home-zh-TW.md
+++ b/docs/wiki/Home-zh-TW.md
@@ -42,7 +42,7 @@ context7 是以覆蓋率排序、週期性重爬的索引; LiveDocs 直接抓 li
 
 - [版本協調流程](Version-Reconciliation-zh-TW): 自動偵測更新流程。
 - [Primary-Source 光譜](Primary-Source-Spectrum-zh-TW): LiveDocs 是什麼、不是什麼。
-- [測試](Testing-zh-TW): 測試套件（210 個 test）與各自覆蓋範圍。
+- [測試](Testing-zh-TW): 測試套件（215 個 test）與各自覆蓋範圍。
 
 ## 其他
 
diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md
index 9f68473..7071441 100644
--- a/docs/wiki/Home.md
+++ b/docs/wiki/Home.md
@@ -57,7 +57,7 @@ Then use the `look-up` skill; it routes questions to the right tool.
 
 - [Version Reconciliation](Version-Reconciliation): the auto-detect update flow.
 - [Primary-Source Spectrum](Primary-Source-Spectrum): what LiveDocs is and isn't for.
-- [Testing](Testing): the test suites (210 tests) and what each covers.
+- [Testing](Testing): the test suites (215 tests) and what each covers.
 
 ## More
 
diff --git a/docs/wiki/Testing-zh-TW.md b/docs/wiki/Testing-zh-TW.md
index f30dd16..f40a8fc 100644
--- a/docs/wiki/Testing-zh-TW.md
+++ b/docs/wiki/Testing-zh-TW.md
@@ -2,13 +2,13 @@
 
 # 測試
 
-LiveDocs 有兩套獨立 test suite —— **210 個 test,全綠**。Swift suite 驗證引擎;Python suite
+LiveDocs 有兩套獨立 test suite —— **215 個 test,全綠**。Swift suite 驗證引擎;Python suite
 驗證 `look-up` *skill* 是否真的會激起 LiveDocs 查詢並給出現行答案, 外加 vs-context7 比較
 harness。以下計數是快照;真正的 source of truth 是實際跑一次。
 
 ```bash
 swift test                                  # 144 個 Swift test
-python3 -m pytest evals/look-up/tests/  # 66 個 Python eval test
+python3 -m pytest evals/look-up/tests/  # 71 個 Python eval test
 ```
 
 ## Swift — 144 個 test（`swift test`）
@@ -43,7 +43,7 @@ python3 -m pytest evals/look-up/tests/  # 66 個 Python eval test
 | `ProcessRunnerTests` | 4 | 大輸出不 deadlock(並發讀 pipe)、exit code 浮現、timeout 回報、SIGTERM→SIGKILL escalation。 |
 | `LatestVersionEncodingTests` | 14 | `latest_version` 工具邊界:pin_honored true/false/nil surfacing(含 nothing-resolved 路徑的 false、無 pin 時缺席)、Go `v`-prefix 已確認 pin(不可誤判成 not-applied)、v-prefix 輸入建出正確 confirm URL、npm 找不到→latest+false、npm dist-tag/exact-pin 區分(移動 tag 絕不假裝成 pin)、RubyGems 端到端 pin(honored + not-found)。 |
 
-## Python — 66 個 test（`pytest evals/look-up/`）
+## Python — 71 個 test（`pytest evals/look-up/`）
 
 `look-up` **skill eval harness** —— 不是測 Swift 引擎,而是測 skill 對多種 prompt 會不會
 激起 LiveDocs 查詢並答對現行 —— 外加 vs-context7 比較 harness。見
@@ -52,7 +52,7 @@ python3 -m pytest evals/look-up/tests/  # 66 個 Python eval test
 | 檔案 | 數 | 測什麼 |
 |------|---:|--------|
 | `test_run_eval.py` | 17 | rate 門檻判定、N=3 門檻塌縮 guard、failed-run / inconclusive 處理、explicit route 比對、`args_match`——對的工具但錯的參數(mis-split `name@version` / 掉 ecosystem pin)判為不正確。 |
-| `test_compare.py` | 20 | vs-context7 新鮮度 harness:版本 token 比對(邊界形式)、對稱計分、表格 render、README 圖片嵌入區塊漂移守衛(唯一*完整*區塊含 orphaned sentinel、ISO 日曆日期驗證、`main()` `--check`/`--write` exit code + write round-trip)、corpus↔sample 同步。 |
+| `test_compare.py` | 25 | vs-context7 新鮮度 harness:版本 token 比對(邊界形式)、對稱計分、表格 render、README 圖片嵌入區塊漂移守衛(唯一*完整*區塊含 orphaned sentinel、ISO 日曆日期驗證、`main()` `--check`/`--write` exit code + write round-trip)、corpus↔sample 同步、snapshot 時效預算(`--check-staleness` 注入 `--today` —— 純數學 / fresh / 超額 / 邊界 / 未來 / 格式錯)、`--verify-live` fail-loud exit code(registry stub)。 |
 | `test_chart.py` | 8 | 逐庫 vs-context7 SVG 圖:`chart_rows` 是 snapshot 的 verbatim 投影、committed SVG **語意**忠實(逐庫 cell 以 `version_matches` 綁對欄位、headline == `tally()`、證明能抓欄位對調)——matplotlib 版本無關的漂移守衛、render 確定性、`--emit-chart` `--check`/`--write` CLI(drift / write round-trip / flag 互斥)。 |
 | `test_oracle.py` | 8 | `self_check`(eval 時打 registry —— 防腐爛)/ `structural` / `golden` oracle。 |
 | `test_detect.py` | 9 | `claude -p` stream-json 解析、`is_error` 偵測、LiveDocs 觸發訊號辨識、逐 call 工具輸入捕捉(short name + input args)。 |
diff --git a/docs/wiki/Testing.md b/docs/wiki/Testing.md
index 7ab8521..f0c48ac 100644
--- a/docs/wiki/Testing.md
+++ b/docs/wiki/Testing.md
@@ -2,14 +2,14 @@
 
 # Testing
 
-LiveDocs has two independent test suites — **210 tests, all green**. The Swift suite verifies
+LiveDocs has two independent test suites — **215 tests, all green**. The Swift suite verifies
 the engine; the Python suite verifies that the `look-up` *skill* actually triggers a
 LiveDocs query and answers currently, plus the vs-context7 comparison harness. Counts below
 are a snapshot; the source of truth is running the suites.
 
 ```bash
 swift test                                  # 144 Swift tests
-python3 -m pytest evals/look-up/tests/  # 66 Python eval tests
+python3 -m pytest evals/look-up/tests/  # 71 Python eval tests
 ```
 
 ## Swift — 144 tests (`swift test`)
@@ -45,7 +45,7 @@ and file-system layer.
 | `ProcessRunnerTests` | 4 | Large output doesn't deadlock (concurrent pipe drain), exit code surfaces, timeout reported, SIGTERM→SIGKILL escalation. |
 | `LatestVersionEncodingTests` | 14 | `latest_version` tool boundary: `pin_honored` true/false/nil surfacing (incl. `false` in the nothing-resolved path, and absent when no pin asked), the Go `v`-prefix confirmed-pin case (must not read as not-applied), v-prefixed input building a valid confirm URL, npm not-found → latest+false, npm dist-tag / exact-pin discrimination (a moving tag is never a faked pin), RubyGems end-to-end pin (honored + not-found). |
 
-## Python — 66 tests (`pytest evals/look-up/`)
+## Python — 71 tests (`pytest evals/look-up/`)
 
 The `look-up` **skill eval harness** — not a test of the Swift engine, but of whether
 the skill fires a LiveDocs query for varied prompts and answers currently — plus the
@@ -55,7 +55,7 @@ vs-context7 comparison harness. See
 | File | Tests | Covers |
 |------|------:|--------|
 | `test_run_eval.py` | 17 | Rate-threshold judging, the N=3 threshold-collapse guard, failed-run / inconclusive handling, explicit-invocation route matching, and `args_match` — the right tool with the wrong args (a mis-split `name@version` / dropped ecosystem pin) scores incorrect. |
-| `test_compare.py` | 20 | vs-context7 freshness harness: version-token matching (boundary forms), symmetric scoring, table render, README image-embed block drift guard (exactly-one *well-formed* block incl. orphaned sentinel, ISO calendar-date validation, `main()` `--check`/`--write` exit codes + write round-trip), corpus↔sample sync. |
+| `test_compare.py` | 25 | vs-context7 freshness harness: version-token matching (boundary forms), symmetric scoring, table render, README image-embed block drift guard (exactly-one *well-formed* block incl. orphaned sentinel, ISO calendar-date validation, `main()` `--check`/`--write` exit codes + write round-trip), corpus↔sample sync, snapshot-staleness budget (`--check-staleness` with injected `--today` — pure math / fresh / over-budget / boundary / future / malformed), and `--verify-live` fail-loud exit codes (registry stubbed). |
 | `test_chart.py` | 8 | Per-library vs-context7 SVG chart: `chart_rows` is a verbatim snapshot projection; the committed SVG is **semantically** faithful (each per-library cell bound to the right column via `version_matches`, headline == `tally()`, proven to catch a column swap) — a matplotlib-version-independent drift guard; deterministic render; `--emit-chart` `--check`/`--write` CLI (drift / write round-trip / flag mutual-exclusion). |
 | `test_oracle.py` | 8 | `self_check` (fetch registry at eval time — rot-proof) / `structural` / `golden` oracles. |
 | `test_detect.py` | 9 | `claude -p` stream-json parsing, `is_error` detection, LiveDocs trigger-signal recognition, and per-call tool-input capture (short name + input args). |

From a4363b8bfecfce5619a50066dac5004f8d878115 Mon Sep 17 00:00:00 2001
From: che cheng <kiki830621@gmail.com>
Date: Mon, 6 Jul 2026 01:26:30 +0800
Subject: [PATCH 3/4] fix: harden freshness machinery per #48/#49 verify round
 1 (#49)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Closes the R1 findings from BOTH ensembles (the two #48 HIGHs land here by design
— same guard family, avoids cross-PR count/file conflicts; cross-referenced):

- HIGH (#48): snapshot-SOURCE tampering merged green — the PR-gate guards only lock
  rendered artifacts ⟷ sample. freshness.yml gains a `pull_request: paths:` trigger
  on the sample/corpus: any PR touching them runs the LIVE registry check as a hard
  gate; a fabricated ground_truth reddens that PR. Network cost only on the rare
  sample-touching PR.
- HIGH (#48): wiki Home pages' hand-written competitive numbers (0 of 6, 6/6|0/6
  table, react v18 vs 19.2.7) had ZERO guard yet publish to the public wiki — new
  test_home_pages_match_sample derives every token from the sample (exact table row
  binds both scores in order; prose score; react pair; capture date; both languages).
- HIGH (#49): GitHub auto-disables scheduled workflows after ~60d of repo inactivity
  — the cron dies quietly exactly when the repo goes dormant. Acknowledged in
  freshness.yml + README with the recovery path (Actions tab / manual run); the
  paths trigger + manual dispatch are the activity-driven backstops.
- MED: --verify-live treated fetch-failure (None) as "no drift" → false GREEN on
  outage + permanent silent-skip for unsupported ecosystems. Now: per-lib unverified
  warning, "verified X of N" summary, exit 1 on drift OR unverified (distinct
  messages). Test extended (None stub → 1).
- MED (alarm fatigue): drift on a deliberately fast-moving corpus is EXPECTED, so a
  monthly hard-red would train the maintainer to ignore it. Scheduled runs now mark
  the drift step continue-on-error (informational); the staleness budget is the hard
  red (one deliberate nag per budget). PR runs keep both hard.
- MED: staleness tests hardcoded dates against the committed sample's captured_at —
  the documented refresh procedure would break them. Now pinned via a temp fixture;
  refresh-proof (README notes it).
- MED: root README still said --verify-live "warns" → exit-1 semantics + staleness
  documented; module docstring likewise (the 2nd stale spot the DA found).
- LOW: --today now strict (same ISO round-trip as captured_at; lenient "20261030"
  rejected, tested); permissions: contents: read + timeout on freshness.yml; 120d
  rationale documented; "date is an attestation, not a proof" honesty note added;
  matplotlib capped <4 (the #48 serialization-drift MEDIUM).

Tests: +1 (Home guard) → 72 Python / 216 total.
---
 .github/workflows/freshness.yml     | 41 +++++++++++---
 README.md                           |  9 ++--
 evals/look-up/README.md             | 34 +++++++++---
 evals/look-up/compare_context7.py   | 39 +++++++++++---
 evals/look-up/requirements.txt      |  5 +-
 evals/look-up/tests/test_compare.py | 83 ++++++++++++++++++++++++-----
 6 files changed, 172 insertions(+), 39 deletions(-)

diff --git a/.github/workflows/freshness.yml b/.github/workflows/freshness.yml
index e035c91..0404d2d 100644
--- a/.github/workflows/freshness.yml
+++ b/.github/workflows/freshness.yml
@@ -1,25 +1,47 @@
 # vs-context7 snapshot freshness (#49): the PR-gate guards (ci.yml) lock README ⟷ sample,
-# but nothing there checks sample ⟷ REALITY. This scheduled run is that reality check:
+# but nothing there checks sample ⟷ REALITY. This workflow is that reality check:
 #   1. --verify-live   — re-fetches the registry ground truth (LiveDocs' side; network)
-#                        and exits 1 if any recorded version has drifted;
+#                        and exits 1 on drift OR when any library couldn't be verified;
 #   2. --check-staleness — exits 1 when captured_at exceeds the age budget (default 120d).
-# Deliberately NOT part of PR CI: a time/network-based failure must never redden PRs
-# unrelated to the chart. A red run here is the maintainer's refresh signal — procedure
-# in evals/look-up/README.md § Refreshing the snapshot. context7's side is a recorded
-# human judgment and structurally cannot be auto-re-verified (stated in the README).
 #
-# Injection surface: no ${{ }} expressions — static commands + stock actions only.
+# Trigger design (#48/#49 verify round 1):
+# - schedule: monthly reality check. Drift on the fast-mover corpus is EXPECTED over time,
+#   so on scheduled runs the drift step is informational (continue-on-error) and the HARD
+#   red is the staleness budget — one deliberate nag per budget, not a permanent alarm
+#   that trains the maintainer to ignore it.
+# - pull_request (paths): any PR touching the snapshot/corpus runs the LIVE check as a
+#   hard PR gate — a tampered or mis-refreshed sample (fabricated ground_truth) goes red
+#   on the PR itself, closing the "edit the snapshot source and regenerate" hole (#48).
+#   Network cost only on the rare sample-touching PR, never on unrelated PRs.
+# - workflow_dispatch: manual re-run / reproduction.
+#
+# ⚠ KNOWN PLATFORM LIMIT (#49 verify): GitHub auto-disables scheduled workflows after
+# ~60 days of repository inactivity — i.e. the cron dies quietly exactly when the repo
+# goes dormant (which is also when the snapshot rots). If returning to a dormant repo:
+# check the Actions tab (re-enable if disabled) or run --check-staleness by hand.
+# The refresh procedure lives in evals/look-up/README.md § Refreshing the snapshot.
+#
+# Injection surface: the only ${{ }} is `github.event_name` (a trusted context value in
+# a YAML field, not event-payload text in a shell). No secrets; least-privilege token.
 name: vs-context7 freshness
 
 on:
   schedule:
     - cron: "17 3 1 * *"   # monthly, 1st @ 03:17 UTC (off-peak, non-round minute)
+  pull_request:
+    paths:
+      - "evals/look-up/compare_context7_sample.json"
+      - "evals/look-up/compare_corpus.yaml"
   workflow_dispatch:
 
+permissions:
+  contents: read
+
 jobs:
   freshness:
     name: snapshot reality check
     runs-on: ubuntu-latest
+    timeout-minutes: 10
     steps:
       - uses: actions/checkout@v4
 
@@ -31,7 +53,12 @@ jobs:
         run: pip install -r evals/look-up/requirements.txt
 
       - name: Registry drift check (LiveDocs-side ground truth)
+        # Informational on SCHEDULED runs (drift on fast-movers is expected over time;
+        # the staleness budget below is the hard signal). Hard-fails on PR runs — a
+        # sample-touching PR must match live reality at merge time.
+        continue-on-error: ${{ github.event_name == 'schedule' }}
         run: python3 evals/look-up/compare_context7.py --verify-live
 
       - name: Staleness budget (captured_at age)
+        if: always()   # surface both axes in one run, even if the drift step failed
         run: python3 evals/look-up/compare_context7.py --check-staleness
diff --git a/README.md b/README.md
index 57a5613..2764b9b 100644
--- a/README.md
+++ b/README.md
@@ -104,7 +104,7 @@ than hardcoding a fact), and it's statistical (N runs, rate thresholds).
 
 ```bash
 pip install -r evals/look-up/requirements.txt
-python3 -m pytest evals/look-up/tests/     # 71 harness unit tests (no API calls)
+python3 -m pytest evals/look-up/tests/     # 72 harness unit tests (no API calls)
 python3 evals/look-up/run_eval.py --dry-run
 python3 evals/look-up/run_eval.py --runs 5 # live baseline (real `claude -p` calls)
 ```
@@ -113,13 +113,14 @@ The same directory carries the **vs-context7 freshness comparison** — a dated,
 head-to-head (registry = neutral ground truth) whose numbers the wiki homepage cites:
 
 ```bash
-python3 evals/look-up/compare_context7.py                # table + tally
-python3 evals/look-up/compare_context7.py --verify-live  # warn if the snapshot drifted
+python3 evals/look-up/compare_context7.py                    # table + tally
+python3 evals/look-up/compare_context7.py --verify-live      # exit 1 if the registry drifted past the snapshot
+python3 evals/look-up/compare_context7.py --check-staleness  # exit 1 if the snapshot is over its age budget
 ```
 
 See [`evals/look-up/README.md`](evals/look-up/README.md) for the design of both, and
 the [Testing wiki page](https://github.com/PsychQuant/livedocs/wiki/Testing) for the full
-suite breakdown (215 tests: 144 Swift + 71 Python).
+suite breakdown (216 tests: 144 Swift + 72 Python).
 
 Status: shipped (v0.8.0; engine features below shipped v0.7.0, the skill's `look-up` rename +
 explicit invocation landed in v0.8.0). 9-ecosystem registry resolution, version pinning (npm/pypi/crates/go/rubygems), OpenAPI/GraphQL/CLI +
diff --git a/evals/look-up/README.md b/evals/look-up/README.md
index bb1edf1..caad84b 100644
--- a/evals/look-up/README.md
+++ b/evals/look-up/README.md
@@ -150,12 +150,30 @@ LiveDocs returned the live registry version each time.
 ### Refreshing the snapshot (#49)
 
 The front-page claim is only as honest as the snapshot's `captured_at`. Two mechanical
-signals tell you it's time (both run monthly by `.github/workflows/freshness.yml`, and
-either can be run by hand):
-
-- `--verify-live` exits 1 → the registry has moved past a recorded `ground_truth`;
-- `--check-staleness` exits 1 → `captured_at` is over the age budget (default 120 days,
-  tunable via `--max-age-days`).
+signals tell you it's time (run monthly by `.github/workflows/freshness.yml`, and either
+can be run by hand):
+
+- `--verify-live` exits 1 → the registry has moved past a recorded `ground_truth`, **or**
+  some library could not be verified (registry unreachable / unsupported ecosystem —
+  never silently counted as "no drift"). On the *scheduled* run this step is
+  **informational** (drift on a deliberately fast-moving corpus is expected over time);
+  on a PR that touches the sample it is a **hard gate** — a tampered or mis-refreshed
+  `ground_truth` reddens that PR against live reality.
+- `--check-staleness` exits 1 → `captured_at` is over the age budget — the **hard**
+  monthly signal. Default 120 days ≈ one typical release cycle for this project — long
+  enough that refreshes aren't busywork, short enough that a dated front-page claim
+  can't quietly become ancient; tunable via `--max-age-days`.
+
+Two honesty notes on the machinery itself:
+
+- **GitHub platform limit**: scheduled workflows are auto-disabled after ~60 days of
+  repository inactivity — the cron dies quietly exactly when the repo goes dormant. If
+  returning after a quiet stretch, check the Actions tab (re-enable if disabled) or run
+  `--check-staleness` by hand.
+- **The date is an attestation, not a proof.** Bumping `captured_at` asserts you actually
+  re-judged context7's side (step 1 below). Bumping the date without re-judging is
+  fabricating a capture — no tool can catch it (context7's column is a recorded human
+  judgment); the procedure is trust-based by design.
 
 Refresh procedure (deliberately a **human** act — context7's column is a recorded
 judgment, not queryable):
@@ -170,7 +188,9 @@ judgment, not queryable):
    `python3 evals/look-up/compare_context7.py --emit-chart --write` then
    `--emit-readme --write`.
 4. `python3 -m pytest evals/look-up/tests/` — the drift guards confirm README ⟷ sample
-   ⟷ SVG are in sync again; commit sample + SVG + README together.
+   ⟷ SVG (and the wiki Home numbers) are in sync again; commit sample + SVG + README
+   together. The staleness tests pin their own sample fixture, so a refresh never
+   breaks them.
 
 ## Scope / caveats
 
diff --git a/evals/look-up/compare_context7.py b/evals/look-up/compare_context7.py
index 28889ae..205697f 100644
--- a/evals/look-up/compare_context7.py
+++ b/evals/look-up/compare_context7.py
@@ -21,7 +21,8 @@
 
 The comparison data is a dated snapshot in compare_context7_sample.json,
 reproducible by re-running both MCP tools by hand (procedure in README). Pass
---verify-live to re-fetch registry ground truth and warn on drift.
+--verify-live to re-fetch registry ground truth — it exits 1 on drift or when any
+library could not be verified (#49); --check-staleness bounds the snapshot's age.
 """
 import argparse
 import datetime
@@ -284,7 +285,14 @@ def main(argv=None):
     if args.check_staleness:
         captured = sample.get("captured_at")
         try:
-            today = datetime.date.fromisoformat(args.today) if args.today else datetime.date.today()
+            if args.today:
+                # Same strict ISO-calendar round-trip as captured_at — Python 3.11+'s bare
+                # fromisoformat is lenient (accepts e.g. "20260702"), which would let a
+                # mistyped override silently parse (#49 verify round 1).
+                _require_iso_date(args.today)
+                today = datetime.date.fromisoformat(args.today)
+            else:
+                today = datetime.date.today()
         except (ValueError, TypeError):
             print(f"✗ --today must be an ISO date (YYYY-MM-DD); got {args.today!r}", file=sys.stderr)
             return 1
@@ -366,24 +374,41 @@ def main(argv=None):
         print("note: --verify-live re-checks the registry (LiveDocs' ground truth) only; "
               "context7's recorded judgments are a dated snapshot, not re-fetched here.",
               file=sys.stderr)
-        drifted = 0
+        drifted, verified, unverified = 0, 0, 0
         for e, r in zip(results, rows):
             lib, eco = e.get("library"), e.get("ecosystem")
             if not lib or not eco:
                 continue
             live = fetch_latest_version(lib, eco)
-            if live and live != r["ground_truth"]:
+            if not live:
+                # fetch_latest_version returns None on network failure OR an ecosystem it
+                # doesn't support — either way this library was NOT verified. Silently
+                # treating that as "no drift" made a registry outage a false GREEN
+                # (#49 verify round 1): green must provably mean "all checked, none drifted".
+                unverified += 1
+                print(f"⚠ unverified: could not fetch a live version for {lib} ({eco}) — "
+                      "registry unreachable or unsupported ecosystem", file=sys.stderr)
+                continue
+            verified += 1
+            if live != r["ground_truth"]:
                 drifted += 1
                 print(f"⚠ drift: {lib} recorded {r['ground_truth']} but registry now {live}",
                       file=sys.stderr)
-        # Fail LOUD on drift (#49): a scheduled run that only prints warnings to an
-        # unread log is not a signal. Exit 1 → the freshness workflow goes red → the
-        # maintainer refreshes the snapshot (procedure: evals/look-up/README.md).
+        # Fail LOUD (#49): a run that only prints warnings to an unread log is not a
+        # signal. Green must mean "verified all N, none drifted" — drift AND
+        # couldn't-verify both exit 1, with distinct messages.
+        print(f"verified {verified} of {len(results)} recorded ground truths "
+              f"({drifted} drifted, {unverified} unverified)", file=sys.stderr)
         if drifted:
             print(f"✗ {drifted} of {len(results)} recorded ground truths have drifted — "
                   "refresh the snapshot per evals/look-up/README.md § Refreshing the snapshot.",
                   file=sys.stderr)
             return 1
+        if unverified:
+            print(f"✗ {unverified} of {len(results)} libraries could not be verified — "
+                  "nothing was proven about them; re-run when the registry is reachable "
+                  "(or add oracle support for the ecosystem).", file=sys.stderr)
+            return 1
 
     t = tally(rows)
     if args.json:
diff --git a/evals/look-up/requirements.txt b/evals/look-up/requirements.txt
index 0cae737..7969d33 100644
--- a/evals/look-up/requirements.txt
+++ b/evals/look-up/requirements.txt
@@ -2,4 +2,7 @@
 #   pip install -r evals/look-up/requirements.txt
 PyYAML>=6.0      # corpus.yaml parsing (run_eval.py)
 pytest>=7.0      # unit tests (tests/)
-matplotlib>=3.9  # per-library vs-context7 SVG chart (chart.py, #50)
+matplotlib>=3.9,<4  # per-library vs-context7 SVG chart (chart.py, #50); <4 cap: CI's
+                    # semantic guard compares a fresh render's <text> strings against the
+                    # committed asset — a major-version serialization change would
+                    # false-FAIL the PR gate (#48 verify), so majors are adopted deliberately.
diff --git a/evals/look-up/tests/test_compare.py b/evals/look-up/tests/test_compare.py
index 71ae36c..e85030d 100644
--- a/evals/look-up/tests/test_compare.py
+++ b/evals/look-up/tests/test_compare.py
@@ -236,42 +236,99 @@ def test_staleness_days_pure():
         staleness_days("not-a-date", datetime.date(2026, 7, 5))
 
 
-def test_check_staleness_fresh_via_main():
+def _pinned_sample(tmp_path, captured_at="2026-07-02"):
+    # #49 verify round 1: the main()-driven staleness tests must NOT depend on the
+    # committed sample's captured_at — the documented refresh procedure bumps that date,
+    # and tests hardcoding "today" against it would break on every legitimate refresh.
+    import json as _json
+    from compare_context7 import load_sample, SAMPLE_PATH
+    s = load_sample(SAMPLE_PATH)
+    s["captured_at"] = captured_at
+    p = tmp_path / "pinned_sample.json"
+    p.write_text(_json.dumps(s), encoding="utf-8")
+    return str(p)
+
+
+def test_check_staleness_fresh_via_main(tmp_path):
     # Drive the real CLI: within budget → exit 0. `--today` injected so the test never
-    # reads the wall clock (sample captured_at = 2026-07-02).
+    # reads the wall clock; `--sample` pinned so a snapshot refresh never breaks it.
     from compare_context7 import main
-    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-07-10"]) == 0
+    s = _pinned_sample(tmp_path)                                   # captured_at = 2026-07-02
+    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-07-10",
+                 "--sample", s]) == 0
 
 
-def test_check_staleness_over_budget_via_main():
+def test_check_staleness_over_budget_via_main(tmp_path):
     # Over budget → exit 1 (fail-loud): the front-page claim has rotted into an
     # asserted number and the scheduled run must go red.
     from compare_context7 import main
-    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2027-07-10"]) == 1
+    s = _pinned_sample(tmp_path)
+    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2027-07-10",
+                 "--sample", s]) == 1
 
 
-def test_check_staleness_boundary_future_and_bad_today():
+def test_check_staleness_boundary_future_and_bad_today(tmp_path):
     # Boundary: age == budget is still WITHIN (fail only when age > budget);
     # 2026-07-02 + 120d = 2026-10-30. A captured_at in the FUTURE of today is a
-    # clock/snapshot error → loud 1, never silently "fresh". Malformed --today → 1.
+    # clock/snapshot error → loud 1, never silently "fresh". Malformed --today → 1,
+    # and lenient-but-nonstandard ISO ("20261030") is rejected by the strict
+    # calendar round-trip (#49 verify round 1).
     from compare_context7 import main
-    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-10-30"]) == 0
-    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-10-31"]) == 1
-    assert main(["--check-staleness", "--today", "2026-07-01"]) == 1   # captured_at in the future
-    assert main(["--check-staleness", "--today", "99-99-99"]) == 1     # malformed today
+    s = _pinned_sample(tmp_path)
+    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-10-30", "--sample", s]) == 0
+    assert main(["--check-staleness", "--max-age-days", "120", "--today", "2026-10-31", "--sample", s]) == 1
+    assert main(["--check-staleness", "--today", "2026-07-01", "--sample", s]) == 1   # captured_at in the future
+    assert main(["--check-staleness", "--today", "99-99-99", "--sample", s]) == 1     # malformed today
+    assert main(["--check-staleness", "--today", "20261030", "--sample", s]) == 1     # lenient ISO rejected
 
 
 def test_verify_live_exit_codes(monkeypatch):
     # #49: --verify-live must FAIL LOUD (exit 1) when the registry has moved past the
     # recorded ground truth — a scheduled run that only prints warnings to a log nobody
-    # reads is not a signal. Registry stubbed (no network in unit tests).
+    # reads is not a signal. Registry stubbed (no network in unit tests). A fetch that
+    # returns None (outage / unsupported ecosystem) is UNVERIFIED, not "no drift" —
+    # green must provably mean "all checked, none drifted" (#49 verify round 1).
     import oracle
     from compare_context7 import main, load_sample
     truths = {r["library"]: r["ground_truth"] for r in load_sample()["results"]}
     monkeypatch.setattr(oracle, "fetch_latest_version", lambda lib, eco: truths[lib])
-    assert main(["--verify-live"]) == 0                     # registry == recorded → green
+    assert main(["--verify-live"]) == 0                     # all verified, none drifted → green
     monkeypatch.setattr(oracle, "fetch_latest_version", lambda lib, eco: "999.0.0")
     assert main(["--verify-live"]) == 1                     # drifted → red
+    monkeypatch.setattr(oracle, "fetch_latest_version", lambda lib, eco: None)
+    assert main(["--verify-live"]) == 1                     # nothing verified → red, not false-green
+
+
+def test_home_pages_match_sample():
+    # #48 verify (HIGH): the wiki Home pages restate the competitive numbers in
+    # hand-written prose + a table, are mirrored verbatim to the PUBLIC GitHub wiki by
+    # sync-wiki.sh, and had zero guard — an edit inflating context7 (0/6 → 6/6) merged
+    # green. Derive every expected token from the SAMPLE (no hardcoding) and assert
+    # both pages carry them: the exact table row (binds both scores in order — a cell
+    # swap fails), the prose score phrase, react's version pair, and the capture date.
+    # Residual (honest): prose nuances beyond these tokens remain unguarded.
+    from compare_context7 import load_sample, judge_row, tally
+    sample = load_sample()
+    rows = [judge_row(e) for e in sample.get("results", [])]
+    t = tally(rows)
+    ld_tok = f"{t['livedocs_current']}/{t['n']}"
+    c7_tok = f"{t['context7_current']}/{t['n']}"
+    react = next(e for e in sample["results"] if e["library"] == "react")
+    pages = {
+        "Home.md": (f"| {ld_tok} (live registry, by construction) | {c7_tok} |",
+                    f"**{t['context7_current']} of {t['n']}**"),
+        "Home-zh-TW.md": (f"| {ld_tok}(live registry, by construction) | {c7_tok} |",
+                          f"**{c7_tok}**"),
+    }
+    for page, (table_row, prose_score) in pages.items():
+        path = os.path.abspath(os.path.join(EVAL_DIR, "..", "..", "docs", "wiki", page))
+        with open(path, encoding="utf-8") as f:
+            text = f.read()
+        assert table_row in text, f"{page}: head-to-head table row diverged from the sample tally"
+        assert prose_score in text, f"{page}: prose score diverged from the sample tally"
+        assert react["ground_truth"] in text, f"{page}: react ground truth missing"
+        assert react["context7"]["default_version"] in text, f"{page}: react context7 default missing"
+        assert sample["captured_at"] in text, f"{page}: capture date missing"
 
 
 def test_sample_covers_corpus():

From 4be92d9df5752525dc1504aa2d62a8fcab20e837 Mon Sep 17 00:00:00 2001
From: che cheng <kiki830621@gmail.com>
Date: Mon, 6 Jul 2026 01:26:30 +0800
Subject: [PATCH 4/4] docs: sync test counts to 216 (144 Swift + 72 Python)
 (#49)

---
 CLAUDE.md                  | 2 +-
 docs/wiki/Home-zh-TW.md    | 2 +-
 docs/wiki/Home.md          | 2 +-
 docs/wiki/Testing-zh-TW.md | 8 ++++----
 docs/wiki/Testing.md       | 8 ++++----
 5 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/CLAUDE.md b/CLAUDE.md
index d2c9db3..382d4cc 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -19,7 +19,7 @@ Swift MCP server (engine) + `look-up` skill (routing) + Python evals.
 
 ```bash
 swift test                                   # 144 Swift tests (no network)
-python3 -m pytest evals/look-up/tests/   # 71 Python tests (no API calls)
+python3 -m pytest evals/look-up/tests/   # 72 Python tests (no API calls)
 python3 evals/look-up/compare_context7.py --verify-live   # freshness snapshot drift check
 make release-signed VERSION=vX.Y.Z           # build + sign + notarize + mcpb (needs DEVELOPER_ID/NOTARY_PROFILE)
 bash scripts/sync-wiki.sh                    # mirror docs/wiki/ -> GitHub wiki (after merge)
diff --git a/docs/wiki/Home-zh-TW.md b/docs/wiki/Home-zh-TW.md
index b780b37..51bc76b 100644
--- a/docs/wiki/Home-zh-TW.md
+++ b/docs/wiki/Home-zh-TW.md
@@ -42,7 +42,7 @@ context7 是以覆蓋率排序、週期性重爬的索引; LiveDocs 直接抓 li
 
 - [版本協調流程](Version-Reconciliation-zh-TW): 自動偵測更新流程。
 - [Primary-Source 光譜](Primary-Source-Spectrum-zh-TW): LiveDocs 是什麼、不是什麼。
-- [測試](Testing-zh-TW): 測試套件（215 個 test）與各自覆蓋範圍。
+- [測試](Testing-zh-TW): 測試套件（216 個 test）與各自覆蓋範圍。
 
 ## 其他
 
diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md
index 7071441..80af413 100644
--- a/docs/wiki/Home.md
+++ b/docs/wiki/Home.md
@@ -57,7 +57,7 @@ Then use the `look-up` skill; it routes questions to the right tool.
 
 - [Version Reconciliation](Version-Reconciliation): the auto-detect update flow.
 - [Primary-Source Spectrum](Primary-Source-Spectrum): what LiveDocs is and isn't for.
-- [Testing](Testing): the test suites (215 tests) and what each covers.
+- [Testing](Testing): the test suites (216 tests) and what each covers.
 
 ## More
 
diff --git a/docs/wiki/Testing-zh-TW.md b/docs/wiki/Testing-zh-TW.md
index f40a8fc..cfe54dc 100644
--- a/docs/wiki/Testing-zh-TW.md
+++ b/docs/wiki/Testing-zh-TW.md
@@ -2,13 +2,13 @@
 
 # 測試
 
-LiveDocs 有兩套獨立 test suite —— **215 個 test,全綠**。Swift suite 驗證引擎;Python suite
+LiveDocs 有兩套獨立 test suite —— **216 個 test,全綠**。Swift suite 驗證引擎;Python suite
 驗證 `look-up` *skill* 是否真的會激起 LiveDocs 查詢並給出現行答案, 外加 vs-context7 比較
 harness。以下計數是快照;真正的 source of truth 是實際跑一次。
 
 ```bash
 swift test                                  # 144 個 Swift test
-python3 -m pytest evals/look-up/tests/  # 71 個 Python eval test
+python3 -m pytest evals/look-up/tests/  # 72 個 Python eval test
 ```
 
 ## Swift — 144 個 test（`swift test`）
@@ -43,7 +43,7 @@ python3 -m pytest evals/look-up/tests/  # 71 個 Python eval test
 | `ProcessRunnerTests` | 4 | 大輸出不 deadlock(並發讀 pipe)、exit code 浮現、timeout 回報、SIGTERM→SIGKILL escalation。 |
 | `LatestVersionEncodingTests` | 14 | `latest_version` 工具邊界:pin_honored true/false/nil surfacing(含 nothing-resolved 路徑的 false、無 pin 時缺席)、Go `v`-prefix 已確認 pin(不可誤判成 not-applied)、v-prefix 輸入建出正確 confirm URL、npm 找不到→latest+false、npm dist-tag/exact-pin 區分(移動 tag 絕不假裝成 pin)、RubyGems 端到端 pin(honored + not-found)。 |
 
-## Python — 71 個 test（`pytest evals/look-up/`）
+## Python — 72 個 test（`pytest evals/look-up/`）
 
 `look-up` **skill eval harness** —— 不是測 Swift 引擎,而是測 skill 對多種 prompt 會不會
 激起 LiveDocs 查詢並答對現行 —— 外加 vs-context7 比較 harness。見
@@ -52,7 +52,7 @@ python3 -m pytest evals/look-up/tests/  # 71 個 Python eval test
 | 檔案 | 數 | 測什麼 |
 |------|---:|--------|
 | `test_run_eval.py` | 17 | rate 門檻判定、N=3 門檻塌縮 guard、failed-run / inconclusive 處理、explicit route 比對、`args_match`——對的工具但錯的參數(mis-split `name@version` / 掉 ecosystem pin)判為不正確。 |
-| `test_compare.py` | 25 | vs-context7 新鮮度 harness:版本 token 比對(邊界形式)、對稱計分、表格 render、README 圖片嵌入區塊漂移守衛(唯一*完整*區塊含 orphaned sentinel、ISO 日曆日期驗證、`main()` `--check`/`--write` exit code + write round-trip)、corpus↔sample 同步、snapshot 時效預算(`--check-staleness` 注入 `--today` —— 純數學 / fresh / 超額 / 邊界 / 未來 / 格式錯)、`--verify-live` fail-loud exit code(registry stub)。 |
+| `test_compare.py` | 26 | vs-context7 新鮮度 harness:版本 token 比對(邊界形式)、對稱計分、表格 render、README 圖片嵌入區塊漂移守衛(唯一*完整*區塊含 orphaned sentinel、ISO 日曆日期驗證、`main()` `--check`/`--write` exit code + write round-trip)、corpus↔sample 同步、snapshot 時效預算(`--check-staleness` 注入 `--today` + pinned sample fixture —— 純數學 / fresh / 超額 / 邊界 / 未來 / 格式錯 / 寬鬆 ISO 拒絕)、`--verify-live` fail-loud exit code(drift 與 unverified 都紅;registry stub)、wiki Home 手寫競品數字對 sample 守衛(表格列 + 散文比分 + react 版本對 + 擷取日期,雙語)。 |
 | `test_chart.py` | 8 | 逐庫 vs-context7 SVG 圖:`chart_rows` 是 snapshot 的 verbatim 投影、committed SVG **語意**忠實(逐庫 cell 以 `version_matches` 綁對欄位、headline == `tally()`、證明能抓欄位對調)——matplotlib 版本無關的漂移守衛、render 確定性、`--emit-chart` `--check`/`--write` CLI(drift / write round-trip / flag 互斥)。 |
 | `test_oracle.py` | 8 | `self_check`(eval 時打 registry —— 防腐爛)/ `structural` / `golden` oracle。 |
 | `test_detect.py` | 9 | `claude -p` stream-json 解析、`is_error` 偵測、LiveDocs 觸發訊號辨識、逐 call 工具輸入捕捉(short name + input args)。 |
diff --git a/docs/wiki/Testing.md b/docs/wiki/Testing.md
index f0c48ac..24bb174 100644
--- a/docs/wiki/Testing.md
+++ b/docs/wiki/Testing.md
@@ -2,14 +2,14 @@
 
 # Testing
 
-LiveDocs has two independent test suites — **215 tests, all green**. The Swift suite verifies
+LiveDocs has two independent test suites — **216 tests, all green**. The Swift suite verifies
 the engine; the Python suite verifies that the `look-up` *skill* actually triggers a
 LiveDocs query and answers currently, plus the vs-context7 comparison harness. Counts below
 are a snapshot; the source of truth is running the suites.
 
 ```bash
 swift test                                  # 144 Swift tests
-python3 -m pytest evals/look-up/tests/  # 71 Python eval tests
+python3 -m pytest evals/look-up/tests/  # 72 Python eval tests
 ```
 
 ## Swift — 144 tests (`swift test`)
@@ -45,7 +45,7 @@ and file-system layer.
 | `ProcessRunnerTests` | 4 | Large output doesn't deadlock (concurrent pipe drain), exit code surfaces, timeout reported, SIGTERM→SIGKILL escalation. |
 | `LatestVersionEncodingTests` | 14 | `latest_version` tool boundary: `pin_honored` true/false/nil surfacing (incl. `false` in the nothing-resolved path, and absent when no pin asked), the Go `v`-prefix confirmed-pin case (must not read as not-applied), v-prefixed input building a valid confirm URL, npm not-found → latest+false, npm dist-tag / exact-pin discrimination (a moving tag is never a faked pin), RubyGems end-to-end pin (honored + not-found). |
 
-## Python — 71 tests (`pytest evals/look-up/`)
+## Python — 72 tests (`pytest evals/look-up/`)
 
 The `look-up` **skill eval harness** — not a test of the Swift engine, but of whether
 the skill fires a LiveDocs query for varied prompts and answers currently — plus the
@@ -55,7 +55,7 @@ vs-context7 comparison harness. See
 | File | Tests | Covers |
 |------|------:|--------|
 | `test_run_eval.py` | 17 | Rate-threshold judging, the N=3 threshold-collapse guard, failed-run / inconclusive handling, explicit-invocation route matching, and `args_match` — the right tool with the wrong args (a mis-split `name@version` / dropped ecosystem pin) scores incorrect. |
-| `test_compare.py` | 25 | vs-context7 freshness harness: version-token matching (boundary forms), symmetric scoring, table render, README image-embed block drift guard (exactly-one *well-formed* block incl. orphaned sentinel, ISO calendar-date validation, `main()` `--check`/`--write` exit codes + write round-trip), corpus↔sample sync, snapshot-staleness budget (`--check-staleness` with injected `--today` — pure math / fresh / over-budget / boundary / future / malformed), and `--verify-live` fail-loud exit codes (registry stubbed). |
+| `test_compare.py` | 26 | vs-context7 freshness harness: version-token matching (boundary forms), symmetric scoring, table render, README image-embed block drift guard (exactly-one *well-formed* block incl. orphaned sentinel, ISO calendar-date validation, `main()` `--check`/`--write` exit codes + write round-trip), corpus↔sample sync, snapshot-staleness budget (`--check-staleness` with injected `--today` + pinned sample fixture — pure math / fresh / over-budget / boundary / future / malformed / lenient-ISO), `--verify-live` fail-loud exit codes (drift AND unverified both red; registry stubbed), and the wiki Home pages' hand-written competitive numbers vs the sample (table row + prose score + react pair + capture date, both languages). |
 | `test_chart.py` | 8 | Per-library vs-context7 SVG chart: `chart_rows` is a verbatim snapshot projection; the committed SVG is **semantically** faithful (each per-library cell bound to the right column via `version_matches`, headline == `tally()`, proven to catch a column swap) — a matplotlib-version-independent drift guard; deterministic render; `--emit-chart` `--check`/`--write` CLI (drift / write round-trip / flag mutual-exclusion). |
 | `test_oracle.py` | 8 | `self_check` (fetch registry at eval time — rot-proof) / `structural` / `golden` oracles. |
 | `test_detect.py` | 9 | `claude -p` stream-json parsing, `is_error` detection, LiveDocs trigger-signal recognition, and per-call tool-input capture (short name + input args). |