perf(e2e): parallelize plugin EXDEV coverage by cjagwani · Pull Request #7243 · NVIDIA/NemoClaw

cjagwani · 2026-07-20T15:14:37Z

Summary

Runs the exact v0.0.71 OpenClaw plugin baseline and the current restart/recreate/rebuild EXDEV lifecycle in independent jobs. The original sequential job took about 24m45s; parallel execution is expected to reduce the wall-clock critical path to roughly 17 minutes without dropping regression assertions.

Related Issue

Part of #7140.

Changes

Split the combined live test into release-baseline and current-lifecycle cases selected from the same source file.
Add independent scheduled and regression-workflow jobs with distinct sandbox names, target IDs, artifact roots, and bounded timeouts.
Preserve the existing openclaw-plugin-runtime-exdev target for current lifecycle coverage and the existing regression dispatch selector.
Extend workflow boundary tests to protect both jobs, selectors, credential revocation, immutable action pins, artifact uploads, and result reporting.

Type of Change

Code change (feature, bug fix, or refactor)
Code change with doc updates
Doc only (prose changes, no code sample modifications)
Doc only (includes code sample changes)

Quality Gates

Tests added or updated for changed behavior
Existing tests cover changed behavior — justification:
Tests not applicable — justification:
Docs updated for user-facing behavior changes
Docs not applicable — justification: Documentation review confirmed that contributor commands and the external regression selector are unchanged; existing docs describe artifacts generically per target.
Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Workflow boundary tests require secret-free runtime steps, immediate Docker credential revocation, immutable action pins, read-only permissions, and reviewed artifact upload actions for both lanes.
Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: 63 E2E-support workflow/registry tests, 5 regression-workflow tests, 25 live-boundary/mock-parity tests, npm run test:projects:check, CLI type-check, Biome, YAML validation, and live-test collection passed.
Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
Quality Gates section completed with required justifications or waivers
No secrets, API keys, or credentials committed
npm run docs builds without warnings (doc changes only)
Doc pages follow the style guide (doc changes only)
New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Charan Jagwani cjagwani@nvidia.com

…y-hang' into codex/diagnose-agent-turn-latency-hang

copy-pr-bot · 2026-07-20T15:14:40Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

coderabbitai · 2026-07-20T15:14:46Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 195a8440-6886-45e2-804b-f02e4be35509

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch codex/e2e-exdev-parallel-lanes

_{Comment @coderabbitai help to get the list of available commands.}

github-code-quality · 2026-07-20T15:15:21Z

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit e8f07c9 in the codex/e2e-exdev-para... branch remains at 96%, unchanged from commit a5ccdca in the codex/diagnose-agent... branch.

TypeScript / code-coverage/cli

The overall coverage in commit e8f07c9 in the codex/e2e-exdev-para... branch remains at 80%, unchanged from commit a5ccdca in the codex/diagnose-agent... branch.

Show a code coverage summary of the most impacted files.

File	`codex/diagnose-agent...` a5ccdca	`codex/e2e-exdev-para...` e8f07c9	+/-
`src/lib/onboard...box-prebuild.ts`	88%	69%	-19%
`src/lib/actions...ess-recovery.ts`	78%	77%	-1%
`src/lib/messagi...flow-planner.ts`	93%	93%	0%
`src/lib/onboard...-patch-clone.ts`	94%	94%	0%
`src/lib/onboard...erfile-patch.ts`	93%	93%	0%
`src/lib/onboard...es-dashboard.ts`	62%	62%	0%
`src/lib/onboard...st-proxy-env.ts`	95%	95%	0%
`src/lib/actions...eway-restart.ts`	90%	93%	+3%
`src/lib/onboard...host-anchors.ts`	90%	94%	+4%
`src/lib/onboard...corporate-ca.ts`	92%	100%	+8%

_{Updated July 20, 2026 15:55 UTC}

github-actions · 2026-07-20T15:21:36Z

PR Review Advisor — Informational

Advisor assessment: Informational / medium confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

GPT-5.6 Terra (primary): Completed · medium confidence · 0 blockers · 0 warnings · 0 suggestions
Nemotron 3 Ultra (second opinion): Failed

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: cloud-onboard, credential-sanitization, security-posture, agent-turn-latency, openclaw-plugin-runtime-exdev, rebuild-hermes, rebuild-hermes-stale-base

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

cjagwani · 2026-07-20T15:41:31Z

/nvskills-ci

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

apurvvkumaria · 2026-07-23T18:28:28Z

Superseded by #7449. The replacement preserves the EXDEV lane improvements on current main with one signed, GitHub-verified commit. Repository branch protection prevented rewriting this PR’s stale pre-squash history, so keeping both open would duplicate the same work.

## Summary Runs the exact v0.0.71 OpenClaw plugin baseline and the current restart/recreate/rebuild EXDEV lifecycle in independent jobs, reducing the critical path without dropping regression assertions. This clean current-`main` branch supersedes draft #7243 because repository rules disallow rewriting its stale, pre-squash history. ## Related Issue Part of #7140. Supersedes #7243. ## Changes - Split the combined live test into independently selectable release-baseline and current-lifecycle cases. - Add independent scheduled and regression-workflow jobs with distinct sandbox names, target IDs, artifact roots, and bounded timeouts. - Preserve the historical builder digest for the release fixture while rewriting the cloned fixture to the current reviewed builder digest for current-lifecycle validation. - Preserve the existing `openclaw-plugin-runtime-exdev` target for current lifecycle coverage and the existing regression dispatch selector. - Extend workflow boundary tests to protect both jobs, selectors, credential revocation, immutable action pins, artifact uploads, and result reporting. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: This restructures internal E2E coverage; plugin behavior, public commands, and the external regression selector are unchanged. Existing fixture security and plugin-install documentation remains accurate. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Workflow boundary tests require secret-free runtime steps, immediate Docker credential revocation, immutable action pins, read-only permissions, independent sandbox/artifact identities, and reviewed artifact upload actions for both lanes. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `no-docs-needed` - Evidence: Existing `docs/security/e2e-weather-plugin-fixture-dependency-review.md`, `docs/deployment/install-openclaw-plugins.mdx`, and `test/e2e/README.md` remain behaviorally accurate; this revision changes only internal E2E lane scheduling, selector wiring, workflow contracts, and test-fixture cleanup. - Agent: Codex documentation writer subagent   ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable - Station profile/scenario: Not applicable - Result: Not applicable - Supporting evidence: Not applicable ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — The current-head EXDEV workflow boundary passed 2 tests, the regression workflow boundary passed 5 tests, semantic validation passed 124 tests across 82 files, and CLI build/typecheck plus the pre-push gate passed. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Targeted workflow, semantic, type, and push gates passed; GitHub CI is authoritative for the repository-wide gate. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>  ## Summary by CodeRabbit * **Tests** * Expanded end-to-end and regression coverage for OpenClaw custom-plugin EXDEV scenarios across **release-baseline** and **current-lifecycle** runs. * Strengthened runtime assertions for pinned plugin loading, restart/recreate/rebuild, and cross-device dependency replacement. * Updated secret-free workflow boundary checks, including multi-job contract validation and the revised runtime proof budget. * **Chores** * Added a scheduled **release-baseline** E2E run with release-pinned fixtures, dedicated artifacts, Docker auth handling/cleanup, and PR reporting integration. * Reduced EXDEV job timeouts and narrowed execution using explicit lifecycle selectors for faster feedback.  --------- Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com>

cjagwani added 9 commits July 17, 2026 03:57

test(e2e): expose agent turn latency progress

e24e5a1

test(e2e): expand progress diagnostics and runtime audit

3502581

merge: synchronize with main

e743391

test(e2e): upload shared progress summaries

3071592

fix(e2e): upload matrix progress summaries

9a9057f

fix(e2e): upload matrix progress summaries

3429e4b

Merge remote-tracking branch 'origin/codex/diagnose-agent-turn-latenc…

a9220a9

…y-hang' into codex/diagnose-agent-turn-latency-hang

fix(e2e): preserve cleanup progress evidence

f293e65

perf(e2e): parallelize plugin EXDEV coverage

ad1fd95

cjagwani self-assigned this Jul 20, 2026

cjagwani added 4 commits July 20, 2026 08:22

test(e2e): verify automatic progress artifacts

aecce15

test(e2e): share EXDEV Docker prerequisite

e7be529

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

test(e2e): keep progress fixture regression linear

950fd9c

merge: restack EXDEV lanes on progress diagnostics

7232143

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

cjagwani added 2 commits July 20, 2026 08:44

merge: synchronize E2E diagnostics with main

f4c0926

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

merge: restack EXDEV lanes on current main

e8f07c9

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance integration: openclaw OpenClaw integration behavior labels Jul 20, 2026

Base automatically changed from codex/diagnose-agent-turn-latency-hang to main July 22, 2026 15:10

apurvvkumaria mentioned this pull request Jul 23, 2026

perf(e2e): parallelize plugin EXDEV coverage #7449

Merged

23 tasks

apurvvkumaria closed this Jul 23, 2026

apurvvkumaria mentioned this pull request Jul 24, 2026

perf(e2e): skip throwaway Hermes onboarding #7244

Closed

21 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf(e2e): parallelize plugin EXDEV coverage - #7243

perf(e2e): parallelize plugin EXDEV coverage#7243
cjagwani wants to merge 15 commits into
mainfrom
codex/e2e-exdev-parallel-lanes

cjagwani commented Jul 20, 2026

Uh oh!

copy-pr-bot Bot commented Jul 20, 2026

Uh oh!

coderabbitai Bot commented Jul 20, 2026 •

edited

Loading

Review skipped

Uh oh!

github-code-quality Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

cjagwani commented Jul 20, 2026

Uh oh!

apurvvkumaria commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

cjagwani commented Jul 20, 2026

Summary

Related Issue

Changes

Type of Change

Quality Gates

Verification

Uh oh!

copy-pr-bot Bot commented Jul 20, 2026

Uh oh!

coderabbitai Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-code-quality Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

TypeScript / code-coverage/plugin

TypeScript / code-coverage/cli

Uh oh!

github-actions Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor — Informational

Model lanes

E2E guidance

Uh oh!

cjagwani commented Jul 20, 2026

Uh oh!

apurvvkumaria commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coderabbitai Bot commented Jul 20, 2026 •

edited

Loading

github-code-quality Bot commented Jul 20, 2026 •

edited

Loading

github-actions Bot commented Jul 20, 2026 •

edited

Loading