A merge that deletes another PR's test is invisible to CI — add a deleted-test guard

[macanderson]

Problem

Three separate times now, a PR has landed on main that silently deleted code
another PR added to the same file earlier the same day, and CI could not see
it. The most recent (fixed in #1975): #1951 rewrote
crates/stella-pipeline/src/pipeline/tests/verification_hardening.rs from a
pre-#1945 base, deleting the PassingShell double, shell_call_result, and
the witness a_revision_halts_at_the_step_where_the_tracked_test_flips that
#1945 had added hours earlier.

main went red. It was caught only because #1945 had also added a
mod flip_halt_arming; line that survived the rewrite and referenced two of the
deleted symbols. Had #1945 added only the test and the doubles — no new
module — the deletion would have compiled clean and silently removed a witness
from the tree. A witness that no longer exists cannot fail, so nothing
downstream would ever have reported it.

This is the same shape recorded for #1860 silently reverting #1836's forwarding
in four crates.

Why CI cannot see it

Both PRs are green against the main they branched from. The merge is textually
clean — git has no conflict to report, because one side simply does not contain
the other's lines. Branch protection's "require branches to be up to date"
would catch it, but it is off (and turning it on serializes every merge, which
is its own cost at this repo's merge rate).

Fix direction

The cheap, high-value version — a deleted-test guard, in the spirit of
check-left-behind.sh:

For a PR's merge base against main, enumerate #[test] / #[tokio::test]
function names present in the base and absent in the head. A non-empty set is
not automatically wrong — tests are legitimately renamed, moved between
modules, or deliberately removed — so the guard should fail asking for an
acknowledgement rather than fail outright: the PR body (or a commit trailer)
must name each removed test. That converts an invisible deletion into a
sentence a reviewer reads.

Cheaper still, and worth doing regardless: this only needs to run in CI on the
merge result, not in make gate locally, since it is inherently a
two-branch question.

Deliberately NOT proposed: requiring up-to-date branches, or forbidding test
file rewrites. Both cost more than the defect.

Files

• scripts/ — new guard, modelled on scripts/check-left-behind.sh (the
  closest existing shape: a mechanical check whose output is "name the thing or
  fix it")
• Makefile — GATE_STEPS, if it is to run locally too (note: adding a step
  there is five edits, per the gate-parity guard — Makefile, AGENTS.md,
  CONTRIBUTING.md, and both spelled-out lists)
• .github/workflows/ci.yml — the merge-result job

Verify

Reconstruct the real case: branch A adds a test to
verification_hardening.rs; branch B rewrites the same file from before A;
merge both. The guard must name the deleted test. Confirm a genuine rename
(same body, new name) is also reported, and that acknowledging it in the PR
body passes.

Definition of done

• A test deleted by a merge cannot reach main without a human having named it
  in the PR body.
• The guard is quiet on ordinary PRs (no false positives on a branch that adds
  tests only).
• Documented wherever the gate is described, if it joins GATE_STEPS.

Related

• fix(stella-pipeline): unbreak main — three parallel-merge collisions in one round #1975 (this round's unbreak — the concrete instance)
• fix(stella-pipeline): arm FlipHalt on the authored-witness path and in revise turns (#1793) #1945, fix(stella-pipeline,stella-protocol): record verifier degradation per candidate in a fan-out (#1787) #1951 (the colliding PRs)
• scripts/check-left-behind.sh (the exemplar to copy)

[macanderson]

Second occurrence, different half of the same problem — worth folding into this issue's scope.

The first round (above) was a merge deleting another PR's test. Hours later, the recovery from that round produced the mirror-image defect: two sessions independently unbroke the same red main, and the merges kept both halves of each fix. main ended up compiling and passing its tests while still failing the gate, with five clippy warnings under -D warnings:

• pipeline/scope_stage.rs — a Spend built before the re-plan loop and a fresh one inline per iteration (unused_variables + unused_mut)
• management_prompt/tests.rs — ModelCallRole::Research listed twice in one match arm (unreachable_pattern)
• tests/verification_hardening.rs and its flip_halt_arming child — PassingShell and shell_call_result defined in both, the child shadowing the parent and leaving it dead (3 × dead_code)

Fixed in #2009. Note the failure mode is quieter than the first round: a duplicated fix compiles and its tests pass, so only the lint catches it — and a session that verified with cargo test alone would have reported green.

Suggested scope addition for the guard proposed here: the same merge-result CI job that reports deleted tests should also report added-in-both hunks — two PRs introducing the same symbol name at the same seam. Both directions are the same underlying gap (CI never evaluates the merge of two independently-green branches), so one job can carry both checks rather than two issues carrying one each.

Also worth capturing separately if anyone picks this up: while fixing the above, check-file-size.sh was also failing on two god files no in-flight PR touched — driver.rs (2572 vs 2571) and pipeline/tests.rs (2537 vs 2536) — because merged PRs grew each by one line without regenerating the baseline. That is a third instance of the same class and is the repo's most frequent red-main cause. Regenerated in #2009.