[rig-tasks] Add 10 rig samples — 2026-07-25 - #156
Conversation
Samples added: - 150-git-hotspot-analyzer-v3: git churn analysis with defineTool + steering - 151-loc-statistics-v2: LOC per extension with async defineTool - 152-import-cycle-detector-v3: madge circular detection with repair addon - 153-coverage-badge-updater-v2: coverage JSON to shields.io badges via p.write - 154-dep-license-auditor-v2: npm license classification (permissive/copyleft/unknown) - 155-test-coverage-mapper-v2: source-to-test heuristic mapping with repair - 156-regex-pattern-tester: regex test runner with input schema + defineTool - 157-commit-churn-classifier: git churn to risk level with steering addon - 158-npm-package-size: npm pack dry-run size estimation with defineTool - 159-ts-branch-coverage: subagent branch analysis (nano branchAnalyzer) All 10 samples pass typecheck. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ |
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /grill-with-docs — requesting changes on two API misuses and flagging a misleading schema.
📋 Key Themes & Highlights
Issues Found
p.json("input")misuse (156):p.jsonserialises its argument withJSON.stringify— passing the string"input"just embeds that literal. Usep.inputField("patterns")to reference a runtime input field in prompt prose.p.writewith placeholder content (153):p.write(path, placeholder)contributes a write instruction to the prompt but cannot write dynamically generated badge content. The correct pattern (used in samples 123, 133, 143) isp.writeOutput(outputField, path)plus an output field.- Misleading
coveredfield in 159:covered: falseis always hardcoded — static analysis can't determine runtime coverage. The schema implies capability the agent doesn't have. - Unvalidated shell interpolation in 151: The
extensionparameter is interpolated directly into a shell command; samples should model safe practices with an allowlist check.
Positive Highlights
- ✅ Excellent variety of patterns across the 10 samples (steering, repair, subagent delegation, input schema)
- ✅ All 10 pass typecheck — clean additions
- ✅ 159 is a nice demonstration of the two-agent delegation pattern with clear separation
- ✅
runRegexTestin 156 correctly wrapsnew RegExp()in try/catch
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 57.8 AIC · ⌖ 4.56 AIC · ⊞ 6.3K
Comment /matt to run again
| instructions: p`Run each regex pattern against its test cases and report results. | ||
|
|
||
| Input patterns and test cases: | ||
| ${p.json("input")} |
There was a problem hiding this comment.
[/grill-with-docs] p.json("input") passes the string "input" to JSON.stringify — the literal string ends up in the prompt, not the caller-supplied input object. Use p.inputField("patterns") to reference the runtime input field in prompt prose.
💡 Suggested fix
Replace:
${p.json("input")}With:
Input patterns and test cases: ${p.inputField("patterns")}p.json(value) is a JSON.stringify helper for concrete values already in scope — not a reference to the runtime input bag. The current code serialises the string "input" into the prompt, so the model never sees the caller-supplied patterns array.
| Return coverageByCategory (record of category to percentage), overallPct, rating, and | ||
| badgesWritten (true if README was updated).`, | ||
| output: s.object({ | ||
| coverageByCategory: s.record(s.number), |
There was a problem hiding this comment.
[/grill-with-docs] p.write("README.md", "<!-- badges will be written by agent -->") uses a hardcoded placeholder string — the agent can't write the real badge content because the content is only known after the LLM produces it. Use p.writeOutput("badgesMarkdown", "README.md") instead, and add a badgesMarkdown: s.string field to the output schema so the harness writes the generated content post-completion.
💡 Suggested fix
The invariant (documented in INV:p-write-no-path) is that p.write contributes a write instruction to the prompt but does not execute it with dynamic content. For dynamic content, use p.writeOutput(outputField, path):
instructions: p`...Generate badge markdown.
${p.writeOutput("badgesMarkdown", "README.md")}`,
output: s.object({
coverageByCategory: s.record(s.number),
overallPct: s.number,
rating: s.enum("green", "yellow", "red"),
badgesMarkdown: s.string, // harness writes this to README.md
badgesWritten: s.boolean,
}),Existing samples 123, 133, 143 all use p.writeOutput for this pattern.
| `find . -name '*${extension}' -not -path '*/node_modules/*' | xargs wc -l 2>/dev/null | tail -1 | awk '{print $1}'`, | ||
| { encoding: "utf-8" } | ||
| ).trim(); | ||
| const files = execSync( |
There was a problem hiding this comment.
[/grill-with-docs] The extension parameter in the shell command is interpolated without any sanitisation — passing extension = '.ts; rm -rf .') would execute arbitrary commands. Since this is a sample demonstrating defineTool, the parameter should be validated or escaped before being interpolated into a shell string.
💡 Suggested fix
Add an allowlist check before the shell call:
async handler({ extension, sampleFile }) {
if (!/^\.\w+$/.test(extension)) {
return { extension, lineCount: 0, fileCount: 0, sampleFile };
}
// ... rest unchanged
}Samples are teaching material — they should model safe shell-interpolation practices.
| // Agent role: identify branch statements in a TypeScript file and estimate coverage. | ||
| const branchAnalyzer = agent({ | ||
| name: "branchAnalyzer", | ||
| model: "nano", |
There was a problem hiding this comment.
[/grill-with-docs] The branchAnalyzer subagent uses covered: false as a hardcoded default with a comment "assume uncovered unless the code contains obvious test guards" — the subagent cannot actually determine coverage from static analysis. This misleads readers into thinking the output field has meaningful values; consider either removing covered from the schema or documenting clearly in the sample title/instructions that this is static structural analysis (not runtime coverage).
💡 Suggestion
Either rename the agent to tsBranchStructureAnalyzer and drop covered, or add a note in the instructions:
instructions: p`Analyze the TypeScript source code and identify all branch points.
NOTE: This is static structural analysis only — runtime coverage is not available.
For each branch: ...`,And remove covered from the schema to avoid false impressions.
Summary
Added 10 new rig sample files to
skills/rig/samples/.Typecheck failures
No failures — all 10 samples passed typecheck on the first attempt.
Tasks run