Skip to content

skills: add agent-run-forensics (experimental) - #12

Open
xizhuomengcontin wants to merge 1 commit into
pproenca:masterfrom
xizhuomengcontin:add-agent-run-forensics
Open

xizhuomengcontin wants to merge 1 commit into
pproenca:masterfrom
xizhuomengcontin:add-agent-run-forensics

Conversation

@xizhuomengcontin

Copy link
Copy Markdown

Adds one skill at skills/.experimental/agent-run-forensics/SKILL.md. Nothing else is touched.

Placement question: I put it under .experimental/ rather than .curated/, reading the layout as staged intake for outside contributions. CONTRIBUTING says skills/your-skill-name/, which is ambiguous now that the real skills live under .curated/. Say the word and I will move it.

PR checklist

  • Name follows conventions — agent-run-forensics, lowercase, single hyphens, matches the directory
  • Required frontmatter present and valid YAML: name, description (446 chars, within 1–1024)
  • Description includes trigger keywords ("investigating what a previous agent session actually did", "reproducing a failure someone else reported", "turning a failed session into a regression test")
  • Optional fields used where they help: license: Apache-2.0, compatibility (states the Node 20+ / CLI requirement and that it is inert without recordings), metadata.author / metadata.version
  • No credentials or sensitive data
  • npx add-skill . --list — could not run it, see below

The documented validation command is broken

npx add-skill . --list no longer works:

npm warn deprecated add-skill@2.0.0: This package has been renamed to 'skills'. Use 'npx skills add' instead.
DEPRECATED: 'add-skill' has been renamed to 'skills'
  Please use: npx skills add <package>
Forwarding to 'npx skills add'...
Failed to run 'npx skills add': spawn npx ENOENT

Two separate problems: the package was renamed, and its forwarding shim calls npx in a way that fails on Windows with spawn npx ENOENT (needs npx.cmd / shell: true there). The replacement npx skills list is not equivalent — it lists installed project skills, not the skills in a repo, so it does not validate a contribution either.

So CONTRIBUTING's "Testing Your Skill" section points at something that cannot pass. Worth updating; I validated the frontmatter by hand instead (parsed the YAML, checked the name/length rules) and am flagging the gap rather than ticking a box I did not actually check.

What the skill does

Answers questions about a run that already happened from its recording rather than from the agent's memory of it — which step changed a file, why a command ran, where the build broke.

It is a deliberate complement to the existing debug skill: debug is a methodology for investigating a bug in code; this is for investigating one recorded agent run, where the evidence already exists and the failure mode is different — asked "why did you change this?", an agent answers from a summary of its own context window, where the tool results, exit codes and quietly-changed files are gone. Fluent, confident, occasionally wrong.

So it enforces one rule — read the trace before answering — and requires keeping recorded claims apart from inferred ones.

Safety and disclosure

Replay re-executes the recorded shell commands for real; the skill requires reading them first and replaying into a scratch worktree, and states that blocked egress is model-provider egress, not network isolation. Model comparison spends real money and discloses the run's files to each provider named.

The skill drives an MCP server exposed by orcareplay (Apache-2.0, npm, Node 20+), which I maintain — vendor-authored, weigh accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants