[docs] Self-healing documentation fixes from issue analysis - 2026-07-15

[github-actions]

Summary

Documents the evals artifact in the artifacts reference page.

Changes

docs/src/content/docs/reference/artifacts.md

• Added evals entry to the artifact summary table (constants.EvalsArtifactName, Single-file, evals.jsonl).
• Added evals row to the --artifacts filter alias table.
• Added a new ## evals section: conditional upload (only when evals declared in frontmatter), contents (evals.jsonl BinEval YES/NO records), access commands, and gh aw audit --evals flag behaviour.

Type

docs — documentation only, no code changes.

Affected components

Component	Change type
docs/src/content/docs/reference/artifacts.md	Documentation added

Notes

No code or test changes. Adds missing reference documentation for the evals artifact.

Generated by PR Description Updater for #45578 · 29.1 AIC · ⌖ 4.33 AIC · ⊞ 4.7K · ◷

[github-actions]

Thanks for improving the docs around the evals artifact and related workflow outputs. This is clearly relevant and nicely scoped. The main issue is process: this repository's contributing guidelines say non-core contributors should not open PRs directly; instead, they should open an issue or agentic plan for a core team member to implement. Also, this change does not include tests, which is acceptable for docs-only work but still worth calling out for review context.

Suggested next step: close this PR and open an issue describing the documentation gap and proposed edits so a core team member can land it through the project's agentic workflow.

Review this contribution against github/gh-aw contributing rules. Convert the PR into a concise issue/agentic plan for the core team. Include: problem statement, exact files to update, proposed documentation text summary, validation steps, and why the change is on-topic. Do not open a PR directly.

Generated by ✅ Contribution Check · 82.2 AIC · ⌖ 8.96 AIC · ⊞ 6.2K · ◷

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #45578 does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100).

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

No test files were added or modified in this PR. PR #45578 contains only documentation changes (docs/src/content/docs/reference/artifacts.md). Test Quality Sentinel analysis skipped.

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

[Copilot]

Pull request overview

Documents the evals artifact and its CLI access paths.

Changes:

• Adds evals to artifact reference tables.
• Documents contents, download commands, and --evals.

Show a summary per file

File	Description
docs/src/content/docs/reference/artifacts.md	Adds evals artifact reference documentation.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 1/1 changed files
• Comments generated: 2
• Review effort level: Medium

[github-actions]

Documentation additions are accurate and consistent with the codebase: constants.EvalsArtifactName, EvalsResultFilename, and the --evals flag on gh aw audit/gh aw logs all match the documented behavior.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · 14.1 AIC · ⌖ 4.92 AIC · ⊞ 5K

[github-actions]

Documentation additions are accurate and consistent with the source code. The constants.EvalsArtifactName, evals.jsonl filename, and --evals flag behavior all match the implementation in pkg/constants/job_constants.go and pkg/cli/audit.go. New section follows existing artifact documentation patterns correctly.

🔎 Code quality review by PR Code Quality Reviewer · 33.8 AIC · ⌖ 4.27 AIC · ⊞ 5.6K
_{Comment /review to run again}

[github-actions]

Skills-Based Review 🧠

Applied /grill-with-docs — one minor observation on the --evals usage example.

📋 Key Themes & Highlights

Positive Highlights

• ✅ Constants and filenames verified accurate (EvalsArtifactName, EvalsResultFilename)
• ✅ Correct placement in Quick Reference table, Artifact Sets table, and dedicated section
• ✅ --evals flag behaviour accurately described (skips runs without evals results)
• ✅ BinEval terminology consistent with codebase usage

Finding

• The gh aw audit <run-id> --evals example on line 233 implies a run-id is required, but --evals is primarily a multi-run list filter. See inline comment.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 28.1 AIC · ⌖ 4.44 AIC · ⊞ 6.7K
_{Comment /matt to run again}

[github-actions]

[/grill-with-docs] The example gh aw audit <run-id> --evals implies --evals requires a specific run ID, but the flag is primarily a multi-run filter (skip runs without evals results). With a single run-id it still works, but the usage pattern shown here is the less common case.

💡 Suggested revision

Consider showing the multi-run (list) form first, which is the primary use case:

# Audit all runs, skipping those without evals results
gh aw audit --evals

# Or narrow to a specific run
gh aw audit <run-id> --evals

Also worth noting: gh aw logs supports the same --evals flag (gh aw logs --evals) — adding a cross-reference or example here would make the docs consistent.

@copilot please address this.