Skip to content

[rig-tasks] Add 10 rig samples — 2026-07-25 - #156

Merged
pelikhan merged 1 commit into
mainfrom
rig-tasks/2026-07-25-5388cc481b9af41f
Jul 25, 2026
Merged

[rig-tasks] Add 10 rig samples — 2026-07-25#156
pelikhan merged 1 commit into
mainfrom
rig-tasks/2026-07-25-5388cc481b9af41f

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Summary

Added 10 new rig sample files to skills/rig/samples/.

# File Description Typecheck
1 150-git-hotspot-analyzer-v3.md Git churn analysis with defineTool + steering addon pass
2 151-loc-statistics-v2.md LOC per extension with async defineTool + execSync pass
3 152-import-cycle-detector-v3.md madge circular detection with repair() addon pass
4 153-coverage-badge-updater-v2.md Coverage JSON to shields.io badges via p.write pass
5 154-dep-license-auditor-v2.md npm license classification (permissive/copyleft/unknown) pass
6 155-test-coverage-mapper-v2.md Source-to-test heuristic mapping with repair pass
7 156-regex-pattern-tester.md Regex test runner with input schema + defineTool pass
8 157-commit-churn-classifier.md Git churn to risk levels with steering addon pass
9 158-npm-package-size.md npm pack dry-run size estimation with defineTool pass
10 159-ts-branch-coverage.md Subagent branch analysis (nano branchAnalyzer) pass

Typecheck failures

No failures — all 10 samples passed typecheck on the first attempt.

Tasks run

  • (reused) Hot-spot file analyzer using p.bash git log + shortlog, defineTool, steering
  • (reused) Lines-of-code statistics gatherer using p.bash wc -l + find, async defineTool
  • (reused) Import cycle detector using p.bash madge --circular, repair addon maxTurns:3
  • (reused) Coverage badge updater with p.readOptional + p.write to README
  • (reused) Dependency license auditor using p.bash npm ls --json, s.enum classification
  • (reused) Test coverage mapper with p.bash find, defineTool heuristics, repair addon
  • (new) Regex pattern tester with input schema, defineTool runRegexTest, repair
  • (new) Commit churn classifier with p.bash git log pipe chain, steering addon
  • (new) NPM package size estimator with p.bash npm pack --dry-run, defineTool classifySize
  • (new) TypeScript branch coverage analyzer with p.readInput + nano subagent delegation

Generated by Daily Rig Task Generator · sonnet46 137.3 AIC · ⌖ 10.2 AIC · ⊞ 6.7K ·

Samples added:
- 150-git-hotspot-analyzer-v3: git churn analysis with defineTool + steering
- 151-loc-statistics-v2: LOC per extension with async defineTool
- 152-import-cycle-detector-v3: madge circular detection with repair addon
- 153-coverage-badge-updater-v2: coverage JSON to shields.io badges via p.write
- 154-dep-license-auditor-v2: npm license classification (permissive/copyleft/unknown)
- 155-test-coverage-mapper-v2: source-to-test heuristic mapping with repair
- 156-regex-pattern-tester: regex test runner with input schema + defineTool
- 157-commit-churn-classifier: git churn to risk level with steering addon
- 158-npm-package-size: npm pack dry-run size estimation with defineTool
- 159-ts-branch-coverage: subagent branch analysis (nano branchAnalyzer)

All 10 samples pass typecheck.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review July 25, 2026 14:36
@pelikhan
pelikhan merged commit fbc275d into main Jul 25, 2026
1 check passed
@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /grill-with-docs — requesting changes on two API misuses and flagging a misleading schema.

📋 Key Themes & Highlights

Issues Found

  • p.json("input") misuse (156): p.json serialises its argument with JSON.stringify — passing the string "input" just embeds that literal. Use p.inputField("patterns") to reference a runtime input field in prompt prose.
  • p.write with placeholder content (153): p.write(path, placeholder) contributes a write instruction to the prompt but cannot write dynamically generated badge content. The correct pattern (used in samples 123, 133, 143) is p.writeOutput(outputField, path) plus an output field.
  • Misleading covered field in 159: covered: false is always hardcoded — static analysis can't determine runtime coverage. The schema implies capability the agent doesn't have.
  • Unvalidated shell interpolation in 151: The extension parameter is interpolated directly into a shell command; samples should model safe practices with an allowlist check.

Positive Highlights

  • ✅ Excellent variety of patterns across the 10 samples (steering, repair, subagent delegation, input schema)
  • ✅ All 10 pass typecheck — clean additions
  • ✅ 159 is a nice demonstration of the two-agent delegation pattern with clear separation
  • runRegexTest in 156 correctly wraps new RegExp() in try/catch

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 57.8 AIC · ⌖ 4.56 AIC · ⊞ 6.3K
Comment /matt to run again

instructions: p`Run each regex pattern against its test cases and report results.

Input patterns and test cases:
${p.json("input")}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] p.json("input") passes the string "input" to JSON.stringify — the literal string ends up in the prompt, not the caller-supplied input object. Use p.inputField("patterns") to reference the runtime input field in prompt prose.

💡 Suggested fix

Replace:

${p.json("input")}

With:

Input patterns and test cases: ${p.inputField("patterns")}

p.json(value) is a JSON.stringify helper for concrete values already in scope — not a reference to the runtime input bag. The current code serialises the string "input" into the prompt, so the model never sees the caller-supplied patterns array.

Return coverageByCategory (record of category to percentage), overallPct, rating, and
badgesWritten (true if README was updated).`,
output: s.object({
coverageByCategory: s.record(s.number),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] p.write("README.md", "<!-- badges will be written by agent -->") uses a hardcoded placeholder string — the agent can't write the real badge content because the content is only known after the LLM produces it. Use p.writeOutput("badgesMarkdown", "README.md") instead, and add a badgesMarkdown: s.string field to the output schema so the harness writes the generated content post-completion.

💡 Suggested fix

The invariant (documented in INV:p-write-no-path) is that p.write contributes a write instruction to the prompt but does not execute it with dynamic content. For dynamic content, use p.writeOutput(outputField, path):

instructions: p`...Generate badge markdown.
${p.writeOutput("badgesMarkdown", "README.md")}`,
output: s.object({
  coverageByCategory: s.record(s.number),
  overallPct: s.number,
  rating: s.enum("green", "yellow", "red"),
  badgesMarkdown: s.string,   // harness writes this to README.md
  badgesWritten: s.boolean,
}),

Existing samples 123, 133, 143 all use p.writeOutput for this pattern.

`find . -name '*${extension}' -not -path '*/node_modules/*' | xargs wc -l 2>/dev/null | tail -1 | awk '{print $1}'`,
{ encoding: "utf-8" }
).trim();
const files = execSync(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] The extension parameter in the shell command is interpolated without any sanitisation — passing extension = '.ts; rm -rf .') would execute arbitrary commands. Since this is a sample demonstrating defineTool, the parameter should be validated or escaped before being interpolated into a shell string.

💡 Suggested fix

Add an allowlist check before the shell call:

async handler({ extension, sampleFile }) {
  if (!/^\.\w+$/.test(extension)) {
    return { extension, lineCount: 0, fileCount: 0, sampleFile };
  }
  // ... rest unchanged
}

Samples are teaching material — they should model safe shell-interpolation practices.

// Agent role: identify branch statements in a TypeScript file and estimate coverage.
const branchAnalyzer = agent({
name: "branchAnalyzer",
model: "nano",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/grill-with-docs] The branchAnalyzer subagent uses covered: false as a hardcoded default with a comment "assume uncovered unless the code contains obvious test guards" — the subagent cannot actually determine coverage from static analysis. This misleads readers into thinking the output field has meaningful values; consider either removing covered from the schema or documenting clearly in the sample title/instructions that this is static structural analysis (not runtime coverage).

💡 Suggestion

Either rename the agent to tsBranchStructureAnalyzer and drop covered, or add a note in the instructions:

instructions: p`Analyze the TypeScript source code and identify all branch points.
NOTE: This is static structural analysis only — runtime coverage is not available.
For each branch: ...`,

And remove covered from the schema to avoid false impressions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant