Skip to content

check-doc-checkpoint is per-COMMIT, so a branch that records evidence and projects it in different commits cannot go green without a rewrite #1585

Description

@localai-bot

scripts/check-doc-checkpoint.py classifies a change to .agents/benchmark-record.md as a measurement and then requires the same commit to also edit docs/STATUS.md and docs/BENCHMARKS.md. It walks per commit, and it does so in every flag form, including --base/--head, which is the form CI uses:

python3 scripts/check-doc-checkpoint.py --base "$PR_BASE" --head "$PR_HEAD"   # ci.yml:540

So on a branch whose forensic record and whose public projection landed in different commits, the gate is red and cannot be made green without rewriting history. Adding the missing surface in a later commit does not help: the earlier commit is still in the range and is still reported.

Reproduced on row/LTX25-DIT-ATTN-FLASH (#1549, PR #1557):

ERROR: commit 2aa78c69b: changed .agents/benchmark-record.md: measurement recorded
  but did not update docs/STATUS.md.
ERROR: commit 2f39a9426: changed .agents/benchmark-record.md: measurement recorded
  but did not update docs/BENCHMARKS.md, docs/STATUS.md.

--staged on the repairing commit reports OK, and the whole-branch diff carries all three surfaces, so the branch satisfies the RULE while failing the GATE.

Why this is the shape AGENTS.md names rather than a user error. ## Records says: "A gate often creates the lock. If a checker requires every change to edit one shared file, the checker is defective. Move the obligation to a per-row surface. Do not delete the obligation." Here the obligation is real -- a measurement is a claim and the projections are how it reaches a reader -- but it is levied per COMMIT against two files that every concurrent branch also writes. The three ways to satisfy it are all bad:

  1. put every forensic append in the same commit as both projection edits, which forbids the ordinary shape of recording evidence as it arrives and then projecting it once;
  2. rewrite the branch, which needs force-push authority for a records edit;
  3. never append to benchmark-record.md except in a projection commit, which is the surface AGENTS.md explicitly designates as the append-only detail file and which check-public-doc-tables.py actively routes prose into ("move the detail to .agents/benchmark-record.md").

Rule (3) and the table checker are in direct tension: one checker pushes detail INTO benchmark-record.md, the other penalises the commit that puts it there.

Not fixed in flow. A semantic checker change needs its own spec, a red-before test and green-after evidence, and this one is about what "a commit" means to a range gate -- not a small, clear fix. Candidate directions, none chosen here: classify over the whole base..head diff rather than per commit for the PR form; or exempt .agents/benchmark-record.md from being a measurement TRIGGER while keeping docs/BENCHMARKS.md as one, since the forensic file is by construction the place detail is moved TO.

Consequence today: PR #1557 carries a documentation-checkpoint red that its own branch content does not deserve, and the squash-merged commit on main would pass the same gate because it carries all three surfaces at once.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions