skills: add agent-run-forensics (experimental) - #12
Open
xizhuomengcontin wants to merge 1 commit into
Open
xizhuomengcontin wants to merge 1 commit into
xizhuomengcontin wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds one skill at
skills/.experimental/agent-run-forensics/SKILL.md. Nothing else is touched.Placement question: I put it under
.experimental/rather than.curated/, reading the layout as staged intake for outside contributions. CONTRIBUTING saysskills/your-skill-name/, which is ambiguous now that the real skills live under.curated/. Say the word and I will move it.PR checklist
agent-run-forensics, lowercase, single hyphens, matches the directoryname,description(446 chars, within 1–1024)license: Apache-2.0,compatibility(states the Node 20+ / CLI requirement and that it is inert without recordings),metadata.author/metadata.versionnpx add-skill . --list— could not run it, see belowThe documented validation command is broken
npx add-skill . --listno longer works:Two separate problems: the package was renamed, and its forwarding shim calls
npxin a way that fails on Windows withspawn npx ENOENT(needsnpx.cmd/shell: truethere). The replacementnpx skills listis not equivalent — it lists installed project skills, not the skills in a repo, so it does not validate a contribution either.So CONTRIBUTING's "Testing Your Skill" section points at something that cannot pass. Worth updating; I validated the frontmatter by hand instead (parsed the YAML, checked the name/length rules) and am flagging the gap rather than ticking a box I did not actually check.
What the skill does
Answers questions about a run that already happened from its recording rather than from the agent's memory of it — which step changed a file, why a command ran, where the build broke.
It is a deliberate complement to the existing
debugskill:debugis a methodology for investigating a bug in code; this is for investigating one recorded agent run, where the evidence already exists and the failure mode is different — asked "why did you change this?", an agent answers from a summary of its own context window, where the tool results, exit codes and quietly-changed files are gone. Fluent, confident, occasionally wrong.So it enforces one rule — read the trace before answering — and requires keeping
recordedclaims apart frominferredones.Safety and disclosure
Replay re-executes the recorded shell commands for real; the skill requires reading them first and replaying into a scratch worktree, and states that blocked egress is model-provider egress, not network isolation. Model comparison spends real money and discloses the run's files to each provider named.
The skill drives an MCP server exposed by
orcareplay(Apache-2.0, npm, Node 20+), which I maintain — vendor-authored, weigh accordingly.