Skip to content

nightly-e2e: openclaw-skill-cli-e2e fails on changed source classification string #5378

@hunglp6d

Description

@hunglp6d

Problem

✨ [AI-generated issue]

The openclaw-skill-cli-e2e nightly E2E job fails because OpenClaw >= 2026.5.27 classifies all installed skills as openclaw-bundled regardless of install origin. The test's hard-coded grep -Fq "openclaw-workspace" assertion always fails even though the skill IS correctly installed and enumerated.

Evidence

  • Failing run: https://github.com/NVIDIA/NemoClaw/actions/runs/27450816965
  • Job: openclaw-skill-cli-e2e / run (log)
  • Error line: FAIL: Expected at least one entry with source 'openclaw-workspace' in 'openclaw skills list --json' output
  • Preceding PASS: Installed skill 'openclaw-skill-cli-fixture' is enumerated by 'openclaw skills list --json' (the skill IS found by name)
  • Classification: test_failure
  • Confidence: high
  • Last green nightly (same job): run 27386272836 (June 12)

Root Cause

OpenClaw's skills list --json changed its source field taxonomy. Previously, workspace-installed skills reported "source": "openclaw-workspace"; OpenClaw v2026.5.27 now reports "source": "openclaw-bundled" for all skills. The E2E test was tightly coupled to the old string value. The OPENCLAW_VERSION in the Dockerfile is pinned at 2026.5.27 (unchanged), but the runtime image the sandbox pulls may have been rebuilt with the new classification.

Fix

Fixes #5376

Replace the raw grep -Fq "openclaw-workspace" with a structured Python JSON check that confirms the fixture skill is present in the list output by name. The workspace install path is already validated on disk in Phase 4 of the test, so source-field classification is not a correctness gate.

Suggested Labels

bug, e2e, nightly

Metadata

Metadata

Assignees

No one assigned

    Labels

    v0.0.66Release target

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions