Skip to content

The record-anchor ratchet's own public figures are gated by nothing: restoring the wrong pre-#851 numbers into docs/BENCHMARKS.md leaves check-public-doc-tables AND check-agent-record both green #1324

Description

@localai-bot

ENG-RECORD-ANCHOR-RATCHET (#632) landed in #851 as 678fc672c. The row's own figures are published in two public projections, and no gate reads either of them. The ratchet reads only the code and tests cells of matrix rows; docs/ prose is outside RECORD_ANCHOR_FIELDS by construction, and check-public-doc-tables.py measures shape (sections, paragraph length, cell length) and never a value.

Measured, not predicted

At af87251c5, restoring the WRONG pre-#851 figures into docs/BENCHMARKS.md:14:

-Rot **38** (32 stale, 6 broken) over **844** OK, **32 in range**
+Rot **39** (32 stale, 7 broken) over **828** OK, **32 in range**
$ python3 scripts/check-public-doc-tables.py ; echo $?
OK: docs/BENCHMARKS.md and docs/FEATURES.md are human-readable keyed tables, ...
0
$ python3 scripts/check-agent-record.py ; echo $?
agent record OK: ENGINE=164 MODEL=377 QUANT=84 KERNEL=52 BACKEND=85 ANCHOR-ROT=38
0

Both green. The rot the checker actually derived in that same run was stale=32, broken=6, and the page said 7 broken beside it. The tree was restored byte-for-byte afterwards (sha256 7f63ca97141e9bffc025d728c95c2bf8254e7fbf71679fa8ab24d94d23a6fad0 before and after).

Three instances, none caught by a gate

  1. Round four of feat(ENG-RECORD-ANCHOR-RATCHET): the record checker range-checks its own citations and reports nothing (#632) #851's fresh review: wrong figures in the public docs, caught by a human reading.
  2. Round five: the same class again, caught the same way.
  3. On main. feat(ENG-RECORD-ANCHOR-RATCHET): the record checker range-checks its own citations and reports nothing (#632) #851 landed over **844** OK in docs/BENCHMARKS.md:14 and over 844 correct in docs/STATUS.md:100. At its own merge commit 678fc672c the tree reads ok=847 — three good citations landed on main between verification and merge. The figure was false the moment it landed, and every gate stayed green.

Instance 3 also shows why chasing the number is not the fix: ok moves on every merge that adds or removes a citation, so any pinned value is false again after the next one. record-anchor-baseline.json already refuses to store the OK count for exactly this reason, in its own _comment. The follow-up branch row/RECORD-ANCHOR-DOC-DRIFT drops the count from all four surfaces that carried it and leaves only the ratcheted rot and the SHA-anchored population figures at 8daa67b39. That repairs the three instances. It does not close the gap, because nothing stops the fourth.

What this is not

Why this is filed rather than fixed

Any repair is a semantic checker change and owes a spec, a red-before mutation, and green-after evidence per AGENTS.md §Changing the rules or a checker. It also has a design decision to make first: whether the derived figures are asserted against the page (a new lock on docs/, which #364 is the standing argument against) or whether the page stops stating derived values at all and the derivation stays in --report. The second shape is what the baseline JSON already chose for itself.

Owning row: ENG-RECORD-ANCHOR-RATCHET, listed under ## Owed in .agents/specs/record-anchor-ratchet.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions