ci(e2e): gate final-main fanout on exact applicable base-image publication

[apurvvkumaria]

Parent epic: #7140

Summary

Final-main E2E currently tries source-SHA base-image tags and can then fall back to a mutable or locally rebuilt image. It does not first prove that every required base-image publisher succeeded for the most recent applicable base-image input commit in the exact E2E checkout. As a result, final validation can fan out against stale or mixed base images after publication failed.

Current evidence

On 2026-07-22:

• Base-image run 29891942278 failed for commit 6b29277260b056b7bb18cf1e0490ad82f4d55c45: OpenClaw and Deep Agents Code published successfully, but Hermes failed.
• Final-main E2E run 29897237525 subsequently started for descendant commit 6d0bc582ab85b11ab688c35aeb4d615151d6ebde.
• The two preceding base-image push runs, 29886835260 and 29889945761, also concluded with partial publication failures.

The immediate publisher failures were the same code-ordering regression: curl was invoked before it existed in the slim base image. #7280 already owns the remaining Hermes fix. This issue owns the independent sequencing defect: failed or incomplete publication did not prevent E2E fanout.

Proposed behavior

Add a standalone base-image-publication preflight job before E2E matrix generation that:

1. Runs only for trusted NVIDIA/NemoClaw main scheduled or manual E2E with an empty controller checkout_sha.
2. Reads the exact checked-out .github/workflows/base-image.yaml push paths and derives the newest applicable first-parent main commit.
3. Bounded-polls the Images / Base Images workflow for that exact source SHA.
4. Requires one trusted main-push run and one successful completed publisher job for each required image: OpenClaw, Hermes, and Deep Agents Code.
5. Fails closed on missing, failed, cancelled, ambiguous, malformed, or partial publication and reports the expected SHA plus authoritative run URL.
6. Completes successfully without publication checks for controller-selected PR E2E, preserving its separate trust and image-resolution path.
7. Gates generate-matrix through an explicit dependency so every downstream fanout is transitively blocked.

Keep the publication wait outside generate-matrix: #7262 gives matrix generation a 10-minute budget, while a base-image publication can legitimately take up to 45 minutes.

Non-overlap and sequencing

• Do not duplicate test(e2e): add semantic live progress phases #7100 phase telemetry, progress artifacts, stall snapshots, or its active local continuation. It can land independently.
• perf(e2e): accelerate Hermes rebuild validation #7213 is merged; do not reopen Hermes fixture, provenance, or current-base reuse work here.
• perf(ci): shorten main and E2E feedback loops #7262 owns PR-gate topology and main/sandbox feedback-loop changes and is the integration blocker for this task. Do not publish or wire this gate until perf(ci): shorten main and E2E feedback loops #7262 is reconciled.
• fix(images): lock OpenClaw production graph #6830 owns current base-image publisher and OpenClaw lock-input changes. Read publisher paths dynamically from the checked-out workflow; do not modify publisher or resolver files here.
• fix(security): update OpenClaw to 2026.7.1 #7280 owns the current Hermes spawnSync curl ENOENT publisher failure. Do not reproduce that fix here.
• The intended implementation slice is limited to .github/workflows/e2e.yaml, one focused verifier under tools/e2e/, and focused verifier/workflow contract tests. Start from fresh main after perf(ci): shorten main and E2E feedback loops #7262 settles.

Acceptance criteria

• Final-main scheduled and manual E2E fanout cannot start until all required base images for the exact applicable source revision are published successfully.
• A partial matrix publication blocks fanout; it cannot silently select latest.
• Missing, queued, in-progress, failed, cancelled, duplicate, and malformed workflow evidence have deterministic bounded outcomes.
• The failure message identifies the expected SHA and links the authoritative base-image run when one exists.
• Controller-selected PR E2E behavior remains unchanged.
• Focused verifier tests and workflow-boundary tests cover success, bounded waiting, and every fail-closed state.
• The implementation is delivered in small complete commits after perf(ci): shorten main and E2E feedback loops #7262 is reconciled.

[apurvvkumaria]

Implementation is now in #7373.

This proceeds as a standalone preflight from current main. #7262 remains open and conflicted, but its overlap is limited to the generate-matrix timeout; it does not verify base-image publication. #7100's progress artifacts are also independent. If either lands first, the small workflow adjacency will be reconciled mechanically without absorbing their work.

The PR adds a fail-closed trusted-main publication verifier and gates matrix fanout on successful OpenClaw, Hermes, and Deep Agents Code publisher evidence. Controller-selected PR E2E bypasses the check before any PR checkout. The branch has two signed, GitHub-Verified commits; 95 focused tests, CLI typecheck, repository checks, and the changed-range hooks pass.

This fixes publication sequencing. Strict digest pinning and removal of the downstream mutable latest fallback remain separate follow-up scope.