Skip to content

fix(e2e): replace brittle source-string assertion in skill CLI test#5376

Draft
hunglp6d wants to merge 1 commit into
NVIDIA:mainfrom
hunglp6d:fix/nightly-e2e-skill-cli-source-assertion-eb1f00d
Draft

fix(e2e): replace brittle source-string assertion in skill CLI test#5376
hunglp6d wants to merge 1 commit into
NVIDIA:mainfrom
hunglp6d:fix/nightly-e2e-skill-cli-source-assertion-eb1f00d

Conversation

@hunglp6d

@hunglp6d hunglp6d commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

✨ [AI-generated PR]

The openclaw-skill-cli-e2e nightly job fails because OpenClaw >= 2026.5.27 classifies all installed skills as openclaw-bundled regardless of install origin. The test's hard-coded grep -Fq openclaw-workspace assertion always fails.

This PR replaces the raw grep with a structured Python JSON check that confirms the fixture skill is present in the list output by name. The workspace install path is already validated on disk in Phase 4, so source-field classification is no longer a correctness gate.

Root Cause

OpenClaw's skills list --json output changed its source field taxonomy. Previously workspace-installed skills reported source: openclaw-workspace; now they report source: openclaw-bundled. The E2E test was coupled to the old string value.

Changes

  • test/e2e/test-openclaw-skill-cli-e2e.sh: Replace grep -Fq openclaw-workspace with a Python-based structured JSON check that locates the fixture skill by name in the parsed list output.

Nightly Run

  • Failing run: https://github.com/NVIDIA/NemoClaw/actions/runs/27450816965
  • Failing job: openclaw-skill-cli-e2e / run
  • Error: FAIL: Expected at least one entry with source openclaw-workspace in openclaw skills list --json output
  • Classification: test_failure — assertion coupled to internal OpenClaw source classification string

Test Plan

  • Re-run openclaw-skill-cli-e2e E2E job with this fix
  • Verify the structured check passes when skill is present regardless of source field value
  • Confirm all other assertions in the test still pass

Signed-off-by: Hung Le hple@nvidia.com

Fixes #5378

OpenClaw >= 2026.5.27 classifies all installed skills as
"openclaw-bundled" regardless of install origin, so the hard-coded
`grep -Fq "openclaw-workspace"` assertion always fails.

Replace the raw grep with a structured Python JSON check that
confirms the fixture skill is present in the list output by name.
The workspace install path is already validated on disk in Phase 4,
so source-field classification is no longer a gate.

Signed-off-by: Hung Le <hple@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jun 13, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3d0d60e1-0f2d-4968-bfcd-4f9406d9698b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nightly-e2e: openclaw-skill-cli-e2e fails on changed source classification string

1 participant