perf(e2e): parallelize plugin EXDEV coverage

[apurvvkumaria]

Summary

Runs the exact v0.0.71 OpenClaw plugin baseline and the current restart/recreate/rebuild EXDEV lifecycle in independent jobs, reducing the critical path without dropping regression assertions. This clean current-main branch supersedes draft #7243 because repository rules disallow rewriting its stale, pre-squash history.

Related Issue

Part of #7140. Supersedes #7243.

Changes

• Split the combined live test into independently selectable release-baseline and current-lifecycle cases.
• Add independent scheduled and regression-workflow jobs with distinct sandbox names, target IDs, artifact roots, and bounded timeouts.
• Preserve the historical builder digest for the release fixture while rewriting the cloned fixture to the current reviewed builder digest for current-lifecycle validation.
• Preserve the existing openclaw-plugin-runtime-exdev target for current lifecycle coverage and the existing regression dispatch selector.
• Extend workflow boundary tests to protect both jobs, selectors, credential revocation, immutable action pins, artifact uploads, and result reporting.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification: This restructures internal E2E coverage; plugin behavior, public commands, and the external regression selector are unchanged. Existing fixture security and plugin-install documentation remains accurate.
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Workflow boundary tests require secret-free runtime steps, immediate Docker credential revocation, immutable action pins, read-only permissions, independent sandbox/artifact identities, and reviewed artifact upload actions for both lanes.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

• Documentation writer subagent reviewed the completed changes
• Result: no-docs-needed
• Evidence: Existing docs/security/e2e-weather-plugin-fixture-dependency-review.md, docs/deployment/install-openclaw-plugins.mdx, and test/e2e/README.md remain behaviorally accurate; this revision changes only internal E2E lane scheduling, selector wiring, workflow contracts, and test-fixture cleanup.
• Agent: Codex documentation writer subagent

DGX Station Hardware Evidence

• Tested on DGX Station
• Tested commit: Not applicable
• Station profile/scenario: Not applicable
• Result: Not applicable
• Supporting evidence: Not applicable

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — The current-head EXDEV workflow boundary passed 2 tests, the regression workflow boundary passed 5 tests, semantic validation passed 124 tests across 82 files, and CLI build/typecheck plus the pre-push gate passed.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: Targeted workflow, semantic, type, and push gates passed; GitHub CI is authoritative for the repository-wide gate.
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

---

Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

• Tests
  • Expanded end-to-end and regression coverage for OpenClaw custom-plugin EXDEV scenarios across release-baseline and current-lifecycle runs.
  • Strengthened runtime assertions for pinned plugin loading, restart/recreate/rebuild, and cross-device dependency replacement.
  • Updated secret-free workflow boundary checks, including multi-job contract validation and the revised runtime proof budget.
• Chores
  • Added a scheduled release-baseline E2E run with release-pinned fixtures, dedicated artifacts, Docker auth handling/cleanup, and PR reporting integration.
  • Reduced EXDEV job timeouts and narrowed execution using explicit lifecycle selectors for faster feedback.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR separates OpenClaw EXDEV coverage into release-baseline and current-lifecycle workflow jobs, refactors the live test around selectors and artifact assertions, and validates both jobs through contract-driven workflow checks.

Changes

OpenClaw EXDEV validation

Layer / File(s)	Summary
Live test variants and artifact reporting test/e2e/live/openclaw-plugin-runtime-exdev.test.ts	Pinned builder images, shared setup helpers, selector-specific tests, cleanup handling, and release/lifecycle artifact assertions are added.
Workflow job wiring .github/workflows/e2e.yaml, .github/workflows/regression-e2e.yaml	Release-baseline jobs are added, current-lifecycle selectors and timeout changes are applied, artifacts are uploaded, and release results are added to reporting.
Multi-job workflow contracts tools/e2e/openclaw-plugin-runtime-exdev-workflow-boundary.mts	Validation becomes contract-driven for both jobs, covering permissions, environment, pinned actions, Docker preparation, selectors, credentials, ordering, and uploads.
Workflow regression assertions test/regression-e2e-workflow.test.ts, test/e2e/support/openclaw-plugin-runtime-exdev-workflow-boundary.test.ts	Contract tests validate release-job wiring, secret-free commands, selectors, permissions, and updated timeout budgets.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Scheduler
  participant WorkflowJob
  participant Vitest
  participant ArtifactUpload
  Scheduler->>WorkflowJob: start release-baseline or current-lifecycle job
  WorkflowJob->>Vitest: run selected EXDEV test variant
  Vitest->>ArtifactUpload: upload e2e-artifacts
  WorkflowJob->>Scheduler: report job result

Loading

Possibly related PRs

• NVIDIA/NemoClaw#7280: Updates the pinned release builder image digest validated by the same workflow-boundary logic.

Suggested labels: area: performance, feature

Suggested reviewers: cv, prekshivyas, cjagwani

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and accurately captures the main change: parallelizing EXDEV e2e coverage.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/e2e-exdev-parallel-lanes-refresh

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 52abe0c in the codex/e2e-exdev-para... branch remains at 96%, unchanged from commit b51168d in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 52abe0c in the codex/e2e-exdev-para... branch remains at 80%, unchanged from commit b51168d in the main branch.

Show a code coverage summary of the most impacted files.

File	main b51168d	codex/e2e-exdev-para... 52abe0c	+/-
src/lib/onboard...ndbox-create.ts	83%	33%	-50%
src/lib/onboard...-create-plan.ts	88%	75%	-13%
src/lib/onboard...ndbox-create.ts	91%	83%	-8%
src/lib/platform.ts	89%	84%	-5%
src/lib/state/m...-acquisition.ts	86%	84%	-2%
src/lib/state/g...way-registry.ts	95%	94%	-1%
src/lib/messagi...nnels/policy.ts	100%	100%	0%
src/lib/sandbox...rce-identity.ts	87%	87%	0%
src/lib/securit...ntial-filter.ts	93%	93%	0%
src/lib/shields/index.ts	71%	72%	+1%

---

_{Updated July 24, 2026 04:08 UTC}

[github-actions]

PR Review Advisor — Informational

Advisor assessment: Informational / high confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

• GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 3 warnings · 0 suggestions
• Model comparison: normalized findings differ; normalized E2E selections match; Nemotron reported the same number of blockers, 3 more warnings, the same number of suggestions.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture, openclaw-plugin-runtime-exdev

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[coderabbitai]

🧹 Nitpick comments (1)

test/e2e/live/openclaw-plugin-runtime-exdev.test.ts (1)

924-954: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Guard fake.close() so the mock server is always torn down.

If fake.requests() or artifacts.writeJson(...) throws inside this cleanup entry, fake.close() is never awaited, leaking the spawned server child process and its temp dir. Wrap the artifact write in try/finally so termination always runs.

♻️ Proposed fix

   cleanup.add("close EXDEV compatible endpoint mock", async () => {
-    await artifacts.writeJson("fake-openai-compatible-requests.json", fake.requests());
-    await fake.close();
+    try {
+      await artifacts.writeJson("fake-openai-compatible-requests.json", fake.requests());
+    } finally {
+      await fake.close();
+    }
   });

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/live/openclaw-plugin-runtime-exdev.test.ts` around lines 924 - 954,
Update the cleanup callback registered in startDeploymentFixture so fake.close()
is always awaited. Wrap the fake.requests() artifact write in a try/finally
block, placing fake.close() in the finally section while preserving the existing
request artifact and cleanup labels.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/e2e/live/openclaw-plugin-runtime-exdev.test.ts`:
- Around line 924-954: Update the cleanup callback registered in
startDeploymentFixture so fake.close() is always awaited. Wrap the
fake.requests() artifact write in a try/finally block, placing fake.close() in
the finally section while preserving the existing request artifact and cleanup
labels.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2b86b260-8687-4f07-96bf-8d7a2a3efa7d

📥 Commits

Reviewing files that changed from the base of the PR and between 0670d20 and 59b3ed4.

📒 Files selected for processing (6)

• .github/workflows/e2e.yaml
• .github/workflows/regression-e2e.yaml
• test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
• test/e2e/support/openclaw-plugin-runtime-exdev-workflow-boundary.test.ts
• test/regression-e2e-workflow.test.ts
• tools/e2e/openclaw-plugin-runtime-exdev-workflow-boundary.mts

[prekshivyas]

Blocking approval at exact head 46934df7dd7ecb7ab06a323a964a56e25ee20677:

1. The documentation-writer receipt is stale. The PR still records reviewed head 93e9fe18e; repository policy requires rerunning the documentation-writer review after every head-changing commit and refreshing the hidden receipt metadata. The prior exact-head receipt explicitly reported headShaMatches: false and status: invalid.
2. Current-head CI and E2E coordination are still in progress. Please let the exact-head gates finish green before approval.

The implementation/security pass found no blocking code issue. Commit 59c0c1b79 correctly makes fake.close() unconditional with try/finally, resolving the cleanup observation. The current merge result passed 21 focused tests and git diff --check; all four commits are GitHub-verified.

[coderabbitai]

🧹 Nitpick comments (1)

.github/workflows/e2e.yaml (1)

658-671: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicated env: block across 8 identical steps.

The run: body is shared via the &trusted-hermes-e2e-swap YAML anchor/alias, but the 11-entry env: block for the trusted_hermes_swap step is copy-pasted verbatim into all 8 jobs. Any future change to these env vars (e.g. adding a new trusted-dispatch check) requires updating all 8 sites, risking drift.

Consider promoting the shared env: map to its own top-level anchor (e.g. under a .github/workflows/e2e.yaml YAML anchors section or a reusable block) and aliasing it the same way the run: body already is:

♻️ Example refactor

# Define once, near the first occurrence:
env: &trusted-hermes-e2e-swap-env
  BASH_ENV: /dev/null
  CHECKOUT_SHA: ${{ inputs.checkout_sha }}
  DISPATCH_SHA: ${{ github.sha }}
  ENV: /dev/null
  EVENT_NAME: ${{ github.event_name }}
  EXPECTED_WORKFLOW_SHA: ${{ inputs.workflow_sha }}
  LC_ALL: C
  REF: ${{ github.ref }}
  REPOSITORY: ${{ github.repository }}
  RUNNER_ARCH_KIND: ${{ runner.arch }}
  RUNNER_ENVIRONMENT_KIND: ${{ runner.environment }}
  RUNNER_OS_KIND: ${{ runner.os }}
  WORKFLOW_SHA: ${{ github.workflow_sha }}

# Reuse elsewhere:
        env: *trusted-hermes-e2e-swap-env

Also applies to: 1337-1350, 1479-1492, 1986-1999, 2326-2339, 2692-2705, 3660-3673, 4943-4956

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e.yaml around lines 658 - 671, Define the shared
trusted_hermes_swap environment map once as a YAML anchor near its first use,
then replace the duplicated env mappings in all eight trusted_hermes_swap steps
with aliases to that anchor. Preserve every existing variable and expression
while ensuring all listed occurrences reuse the same definition.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/e2e.yaml:
- Around line 658-671: Define the shared trusted_hermes_swap environment map
once as a YAML anchor near its first use, then replace the duplicated env
mappings in all eight trusted_hermes_swap steps with aliases to that anchor.
Preserve every existing variable and expression while ensuring all listed
occurrences reuse the same definition.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 89ba900e-fa34-4381-a512-bf35b557712a

📥 Commits

Reviewing files that changed from the base of the PR and between 46934df and 52abe0c.

📒 Files selected for processing (1)

• .github/workflows/e2e.yaml

[prekshivyas]

Reviewed CodeRabbit’s current-head YAML-anchor nit. No PR change is needed: the repeated trusted_hermes_swap environment blocks are identical in base b51168d190c285d9038a3a345215258de03da135 and head 52abe0c9345332242282a3e16e917d3ae3fe70ee; they arrived from the main synchronization and are outside this PR’s EXDEV-lane patch. Refactoring that shared trusted-workflow code here would broaden scope and invalidate otherwise clean exact-head evidence, so it should be handled separately if maintainers want the cleanup.

[prekshivyas]

Approved at exact head 52abe0c after code, security, focused-test, CI, CodeRabbit, and advisor review. The final selected E2E job and required gate are still running and remain merge requirements.