Skip to content

Skill validator false-positives runtime/cache paths mentioned in SKILL.md prose #1926

@taariq

Description

@taariq

Problem

After installing prophet-arb-bot (and other skills that document runtime artifacts in prose), the Skills Explorer renders a misleading warning banner:

Incomplete install: prophet-arb-bot
Missing files referenced in SKILL.md:
state/session_cache.json

The install is not incomplete. state/session_cache.json is a runtime artifact lazily created by agent.py on first JWT cache write — it is intentionally not shipped in the bundle. The validator is producing a false positive.

Root Cause

It's a false positive in the desktop install validator — not a bug in the publisher, the bundle, or the skill source.

The chain:

  1. prophet-arb-bot/SKILL.md line 133 mentions state/session_cache.json in prose as a manual-recovery hint when cold-start auth fails:

    "...the operator must pre-supply a JWT or seed state/session_cache.json by hand."

  2. state/session_cache.json is a runtime artifact, not a shipped payload. The directory state/ doesn't exist in the source repo at seren-skills/prophet/prophet-arb-bot/agent.py creates it lazily on the first JWT cache write. The publisher ships nothing under state/, which is correct.
  3. The desktop validator regex is too permissive. seren-desktop/src-tauri/src/skills.rs:1015-1042, function extract_referenced_files, has two passes:
    • Markdown-link pass [text](path) — fine.
    • Backtick pass: `([a-zA-Z0-9_./-]+\.(py|sh|json|txt|toml|yaml|yml|js|ts))` (line 1033). This matches any backticked filename in prose — including runtime/cache paths, illustrative file names, and seed-it-yourself instructions.
  4. validate_skill_payload (skills.rs:808) walks the regex output, calls has_template_sibling (which only knows about *.example.* / *.template.* / *.sample.* siblings — not runtime dirs), and returns state/session_cache.json as missing.
  5. SkillsExplorer.tsx:666-672 calls the validator after install and renders the warning banner at SkillsExplorer.tsx:1167-1170.

The publisher repo (seren-skills-publisher) doesn't contain prophet-arb-bot files — confirmed not a publishing-side bundling miss.

Why This Matters

This regex catches every backticked runtime/cache path mentioned in prose across the catalog. Any skill that documents a runtime artifact (state/*.json, logs/*.txt, cache/*.json, ~/.config/.../wallet.local.json, etc.) will trip it. prophet-arb-bot is the visible victim, but it's a class of bug.

Fix

Add a runtime-path allow-list to extract_referenced_files so paths under state/, logs/, cache/, .cache/, output/, tmp/, ~/, /Applications/, and similar host/runtime locations are skipped during validation. Keeps the existing strict behavior for genuine payload files (scripts, schema, configs) but stops flagging documented runtime artifacts.

Impact

  • Fixes false-positive "Incomplete install" banner for prophet-arb-bot and any future skill that documents runtime artifacts.
  • No change to genuine missing-file detection for payload files (scripts/agent.py, requirements.txt, serendb_schema.sql, etc.).
  • No publisher or skill-side changes required.

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions