feat(eval): mergeability verdict for external-repo PRs without a benchmark delta

[matedev01]

Problem (current state, with file refs)

The merge decision on our OWN repo is self-referential: "should I merge this PR?" reduces to "did this change to OUR agent raise OUR benchmark composite?" scripts/score_pr_delta.py is a reporter (docstring line 27) that consumes two pre-existing run_eval artifacts (baseline-agent vs. candidate-agent) and bands the composite_mean delta behind a Pareto floor over PARETO_AXES = ("judge_mean", "objective_mean") (line 52), doubled to a public/private target by combine_dual_target() (line 264).

That delta only exists when the change is to the agent itself. scripts/benchmark_pr_policy.py is_agent_submission() (line 50) restricts the scored surface to agent.py / agent/**. For an arbitrary PR on an external product repo ("fix validator lease handling") there is no baseline-agent/candidate-agent artifact pair, so score_pr_delta has nothing to consume — no delta, no band. The verdict falls back on the LLM judge, which currently saturates at judge_mean=1.0 (#2379) and, even once fixed via challenger-vs-king (#2382), grades maintainer foresight rather than the correctness/safety of a submitted diff.

So there is no defined signal today for "is THIS external PR worth merging?" — this is the structural gap the whole roadmap is built around.

Why it blocks autonomous external maintenance

Every downstream capability — attestation, the run loop, the safety envelope, the eventual execution gateway — needs a value verdict to carry. Without a non-self-referential mergeability signal there is nothing meaningful for the maintainer loop to act on for an external repo, and "autonomous external maintainer" cannot be truthfully claimed.

Proposed direction (concrete, staged; reuse existing seams, invent no new APIs)

Define a mergeability verdict object for external-repo PRs, computed without a benchmark delta, that combines three independent signals with an anti-Goodhart floor rather than an average:

• Correctness gate (hard, fail-closed): the green/red correctness-evidence bundle from the sibling feat(eval): diff-correctness harness. No green bundle → no merge verdict, full stop.
• Domain veto (absolute): the classifier from the sibling feat(guardrail): permanent, fail-closed domain gate for subnet-critical paths #2384. A critical match forces the verdict away from any write recommendation, and no score can override it.
• Qualitative axes (judge): scope fit, readability, and safety-of-approach via the pairwise judge — used only once it discriminates (post-benchmark: judge opponent should be a peer agent (challenger-vs-king), not a deterministic baseline #2382); never trusted while it saturates (benchmark: judge_mean saturates to 1.0 (judge not discriminating) — surfaced by the v0.9.0 anchor #2379).

Stage 1 — verdict schema. A serializable verdict keyed to {repo, base_sha, head_sha, diff_digest} recording each signal, the floor decision, and an advisory value_label + rationale. Keep the anti-Goodhart shape of score_pr_delta: a Pareto-style floor that blocks on ANY regressed/uncovered/critical axis, never averaging a bad axis away.

Stage 2 — wire into the recommendation. Replace agent/review.py's ungrounded value_label with the composed verdict; write it to the owner-only local sink only. No posting, no write.

Stage 3 — publish the verdict object so the attestation sibling can bind it ({repo, pr_number, base_sha, head_sha, diff_digest, decision}), and so it is the object the (future, ROADMAP §4) owner-action gateway would consume in place of the absent score_pr_delta band.

Acceptance criteria

• A documented mergeability-verdict schema exists for a PR that produces no run_eval artifact, keyed to {repo, base_sha, head_sha, diff_digest}.
• The verdict is fail-closed on the correctness bundle: no green bundle → no merge recommendation.
• A domain-critical match is an absolute veto no score can override (test).
• The judge contributes only qualitative axes and is gated off while it saturates; a test asserts a saturated judge cannot by itself produce a merge verdict.
• The floor blocks on any single regressed/uncovered/critical axis rather than averaging (anti-Goodhart test mirroring score_pr_delta's Pareto floor).
• Output is advisory only — written to the owner-only sink, never posted; can_auto_execute() stays False.

Dependencies

• benchmark: judge_mean saturates to 1.0 (judge not discriminating) — surfaced by the v0.9.0 anchor #2379, benchmark: judge opponent should be a peer agent (challenger-vs-king), not a deterministic baseline #2382 — the judge must discriminate before its qualitative axes may be trusted.
• Sibling feat(eval): diff-correctness harness — supplies the hard correctness gate.
• Sibling feat(guardrail): permanent, fail-closed domain gate for subnet-critical paths #2384 — supplies the absolute veto.
• Sibling feat(attest): per-decision evidence bundle and receipt for live review decisions #2386 — binds and publishes this verdict.

Out of scope / stays human-gated

• No GitHub write. This issue produces a verdict, not an action. Execution belongs to the owner-action execution gateway (external signer, approval binding, idempotency, audit; ROADMAP §4) — a separate, permanently human-gated track.
• Incentive / consensus / emission / weights / payment / wallet code paths stay PERMANENTLY human-gated. A positive verdict never earns autonomy on those surfaces — the domain veto guarantees they never become mergeable by this signal, regardless of any score or judge outcome. Autonomy is earned only on the low-risk surface.
• No change to the foresight benchmark, score_pr_delta, or the Pareto floor for our own agent PRs; internal repo-set selection stays anonymized as it is today (combine_dual_target).