Skip to content

ci: split E2E feedback-loop work into focused PRs #7451

Description

@cv

Summary

PR #7262 combines several independent CI and E2E changes. Its final diff has 38 files, 1,179 additions, and 1,420 deletions.

Replace #7262 with focused PRs from fresh origin/main. Each PR must change one behavior and include its tests and operator documentation.

This work changes internal CI and E2E infrastructure. It does not create a supported integration, solution recipe, or product surface.

Source state

Focused PRs

Platform test prerequisites

  • Complete test(platform): remove macOS watcher path mismatch and Shields timer race #7259 in a test-only PR.
    • Change only src/lib/shields/flow.test.ts.
    • Replace the timer oracle with the exact child PID and process-start identity assertion.
    • Preserve owner-stop ordering, transition cleanup, and policy restoration assertions.
  • Make the Hermes root-topology test fail closed outside WSL.
    • Change only test/hermes-runtime-config-guard-topology.test.ts.
    • WSL without Docker may skip the Docker-dependent contract.
    • Native Linux must attempt the contract and fail when Docker is unavailable.

Platform sharding

  • Shard the full macOS and WSL Vitest suites.
    • Change .github/workflows/platform-vitest-main.yaml.
    • Run four shards per platform with fail-fast: false.
    • Keep 30-minute macOS and 90-minute WSL job budgets.
    • Run the root-required WSL contracts only on shard 1.
    • Update test/platform-vitest-main-workflow.test.ts.
    • Update only the platform paragraphs in the internal E2E documentation.
    • Merge the platform test prerequisite PRs first.

Hermes image production and reuse

  • Build the Hermes production image once and reuse it.
    • Change only the Hermes producer and consumer jobs in .github/workflows/sandbox-images-and-e2e.yaml.
    • Use a pinned local-load Buildx action and a runner-scoped GitHub Actions cache.
    • Keep the bounded export swap, completed-image node-tar scan, and no-registry-write contract.
    • Upload one image artifact from the producer.
    • Load that artifact in the secret-boundary and root-entrypoint consumer.
    • Update the sandbox workflow boundary validator and focused tests.
    • Do not change .github/workflows/main.yaml in this PR.

Main workflow scheduling

  • Start sandbox image validation after static and build/type-check preflight.
    • Change .github/workflows/main.yaml.
    • Keep sandbox-images-and-e2e in the final checks dependency list.
    • Fail the final check when the sandbox workflow does not pass.
    • Update only the main-caller hunks in the sandbox boundary validator and tests.
    • Merge the Hermes image reuse PR first.

Brev provisioning

  • Retry Brev provisioning from authoritative state.
    • Add tools/e2e/brev-provisioning.mts and its focused state tests.
    • Reject malformed entries in recognized JSON inventory.
    • Stop an attempt on authoritative terminal state.
    • Stop after three consecutive authoritative queries omit the instance.
    • Do not count failed or text-only inventory queries as authoritative absence.
    • Allow two bounded CPU and GPU provisioning attempts by default.
    • Clean up a failed attempt before another create.
    • Do not retry after live test execution starts.
    • Keep unrelated formatting changes out of test/e2e/brev-e2e.test.ts.

Direct PR E2E required check

  • Publish E2E / PR Gate directly as the exact-diff required check.
    • Remove the polling required job.
    • Delete tools/e2e/pr-e2e-required.mts and its tests.
    • Make the controller own the required check lifecycle.
    • Preserve exact PR SHA, base SHA, GitHub App, retry-history, and final-state validation.
    • Keep rollout fallback to E2E / PR Gate Coordination only when the current check is absent.
    • Update maintainer gate inspection, risk planning, watch triggers, tests, and direct-check documentation together.
    • Obtain a sensitive-path security review.
    • Merge this PR after the lower-risk work.

Independent follow-ups

  • Add explicit timeouts to the E2E matrix, PR report, and scorecard jobs.
    • Add focused contract assertions.
    • Do not include indentation-only workflow changes.
  • Document protected internal E2E authorization.
    • Document the protected-environment path before the manual run-control-plane fallback.
    • Keep this documentation correction separate from the direct-check architecture change.

Repository configuration blocker

The environment approve-credentialed-e2e-for-internal-pr exists. The GitHub API currently reports:

  • protection_rules: []
  • deployment_branch_policy: null

Before a PR relies on protected internal authorization, a repository administrator must:

  • Configure at least one required reviewer.
  • Enable prevent self-review.
  • Disable administrator bypass.
  • Restrict the environment to protected main.
  • Keep environment secrets, variables, and custom protection apps empty.
  • Verify the configuration through the GitHub API.

Validation

Each implementation PR must:

  • Run focused tests for its changed behavior.
  • Run npm run test:changed.
  • Rerun targeted tests after behavior-affecting edits or hook fixes.
  • Include real macOS and WSL evidence for platform sharding.
  • Include sandbox image workflow evidence for Hermes image reuse.
  • Include staging Brev evidence for provisioning changes.
  • Include exact-SHA required-check evidence for the direct PR E2E check.
  • Complete the documentation-writer review required by the repository workflow.
  • Avoid unrelated formatting and documentation changes.

Close this issue when all work items merge or a maintainer records an explicit waiver for each remaining item.

Metadata

Metadata

Assignees

Labels

v0.0.96Release target

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions