Skip to content

fix(safe-outputs): bundle safe-outputs-items manifest into usage artifact to restore SafeItemsCount#43056

Merged
pelikhan merged 11 commits into
mainfrom
copilot/deep-report-fix-safe-output-aggregator-undercount
Jul 3, 2026
Merged

fix(safe-outputs): bundle safe-outputs-items manifest into usage artifact to restore SafeItemsCount#43056
pelikhan merged 11 commits into
mainfrom
copilot/deep-report-fix-safe-output-aggregator-undercount

Conversation

Copilot AI commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Since 2026-05-31, SafeItemsCount=0 and actuation_style=read_only on every run because the safe-output-items.jsonl manifest (uploaded as safe-outputs-items artifact by the safe_outputs job) was never downloaded in the conclusion job — so activity/summary.json contained no safe-output data, and gh aw logs --artifacts usage always saw zero items.

Changes

Conclusion job — download manifest before summary generation (notify_comment.go)

  • Prepend a actions/download-artifact step in buildUsageArtifactUploadSteps() that fetches {prefix}safe-outputs-items/tmp/gh-aw/ before "Collect usage artifact files" runs
  • continue-on-error: true so workflows with no safe outputs don't fail

Usage summary generator — parse manifest (generate_usage_activity_summary.cjs)

  • Add parseSafeOutputsManifest(): reads /tmp/gh-aw/safe-output-items.jsonl, returns { total_items, items_by_type } keyed by handler type (create_issue, add_comment, …)
  • Include result as safe_outputs key in activity/summary.json

Go backfill (logs_usage_activity.go)

  • Add usageActivitySafeOutputs struct; when SafeItemsCount == 0 and summary.SafeOutputs.TotalItems > 0, backfill result.Run.SafeItemsCount from the summary — avoids masking genuine zero-item runs
// activity/summary.json — new safe_outputs section
{
  "safe_outputs": {
    "total_items": 4,
    "items_by_type": {
      "create_issue": 2,
      "add_comment": 2
    }
  }
}


✨ PR Review Safe Output Test - Run 28636821292

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · 100.4 AIC · ⌖ 31.5 AIC · ⊞ 8.7K ·
Comment /smoke-claude to run again


Generated by 👨‍🍳 PR Sous Chef · 6.07 AIC · ⌖ 7.49 AIC · ⊞ 6.4K ·

…temsCount backfill

- Add parseSafeOutputsManifest() to generate_usage_activity_summary.cjs to read
  /tmp/gh-aw/safe-output-items.jsonl and include safe_outputs counts in activity/summary.json
- Add "Download safe outputs items manifest" step in conclusion job (buildUsageArtifactUploadSteps)
  so the safe-outputs-items artifact is available before usage summary generation
- Add usageActivitySafeOutputs struct and SafeItemsCount backfill logic in logs_usage_activity.go
- Add tests for parseSafeOutputsManifest in generate_usage_activity_summary.test.cjs
- Add tests for SafeItemsCount backfill in logs_usage_activity_test.go
- Update TestConclusionJobIncludesUsageArtifactSteps to assert new download step

Fixes SafeItemsCount=0 / actuation_style=read_only undercount since 2026-05-31.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix safe-output aggregator undercount for SafeItemsCount fix(safe-outputs): bundle safe-outputs-items manifest into usage artifact to restore SafeItemsCount Jul 3, 2026
Copilot AI requested a review from pelikhan July 3, 2026 00:26
@pelikhan pelikhan marked this pull request as ready for review July 3, 2026 00:27
Copilot AI review requested due to automatic review settings July 3, 2026 00:27
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

⚠️ PR Code Quality Reviewer failed during code quality review.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ completed the design decision gate check.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Test Quality Sentinel completed test quality analysis.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer failed during the skills-based review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR restores safe-output item accounting in gh aw logs --artifacts usage by ensuring the safe-outputs items manifest is present when the usage activity summary is generated, then surfacing those counts in activity/summary.json and backfilling SafeItemsCount from that summary when needed.

Changes:

  • Download {prefix}safe-outputs-items into /tmp/gh-aw/ in the conclusion job before generating the usage activity summary.
  • Extend generate_usage_activity_summary.cjs to parse /tmp/gh-aw/safe-output-items.jsonl and emit a safe_outputs section in activity/summary.json.
  • Backfill SafeItemsCount from summary.safe_outputs.total_items when SafeItemsCount is zero but the usage summary reports items.
Show a summary per file
File Description
pkg/workflow/notify_comment.go Adds a download-artifact step so the safe-outputs manifest is available during usage summary generation.
pkg/workflow/notify_comment_test.go Extends coverage to assert the manifest download step exists and precedes usage artifact collection.
pkg/cli/logs_usage_activity.go Parses safe_outputs from the usage activity summary and backfills SafeItemsCount when appropriate.
pkg/cli/logs_usage_activity_test.go Adds tests for safe-outputs parsing and SafeItemsCount backfill behavior.
actions/setup/js/generate_usage_activity_summary.cjs Adds parseSafeOutputsManifest() and includes safe_outputs in the generated summary JSON.
actions/setup/js/generate_usage_activity_summary.test.cjs Adds unit tests for parseSafeOutputsManifest().
.github/workflows/ab-testing-advisor.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/ace-editor.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/agent-performance-analyzer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/agent-persona-explorer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/agentic-token-audit.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/agentic-token-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/agentic-token-trend-audit.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/ai-moderator.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/api-consumption-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/approach-validator.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/archie.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/architecture-guardian.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/artifacts-summary.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/audit-workflows.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/auto-triage-issues.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/avenger.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/aw-failure-investigator.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/bot-detection.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/brave.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/breaking-change-checker.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/chaos-pr-bundle-fuzzer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/changeset.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/ci-coach.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/ci-doctor.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/claude-code-user-docs-review.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/cli-consistency-checker.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/cli-version-checker.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/cloclo.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/codex-github-remote-mcp-test.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/code-scanning-fixer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/code-simplifier.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/commit-changes-analyzer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/contribution-check.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/constraint-solving-potd.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-agent-analysis.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-centralization-drilldown.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-centralization-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-cli-deep-research.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-opt.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-pr-merged-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-pr-nlp-analysis.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-pr-prompt-analysis.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/copilot-session-insights.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/craft.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-agentrx-trace-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-ambient-context-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-architecture-diagram.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-assign-issue-to-user.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-astrostylelite-markdown-spellcheck.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-aw-cross-repo-compile-check.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-awf-spec-compiler-surfacing.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-byok-ollama-test.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-cache-strategy-analyzer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-caveman-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-choice-test.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-cli-performance.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-cli-tools-tester.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-code-metrics.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-community-attribution.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-compiler-quality.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-compiler-threat-spec-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-credit-limit-test.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-doc-healer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-doc-updater.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-experiment-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-fact.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-file-diet.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-firewall-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-formal-spec-verifier.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-function-namer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-geo-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-hippo-learn.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-issues-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-malicious-code-scan.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-max-ai-credits-test.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-mcp-concurrency-analysis.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-model-inventory.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-model-resolution.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-multi-device-docs-tester.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-news.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-observability-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-performance-summary.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-regulatory.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-rendering-scripts-verifier.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-reliability-review.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-repo-chronicle.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-safe-output-integrator.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-safe-output-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-safe-outputs-conformance.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-safeoutputs-git-simulator.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-security-observability.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-security-red-team.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-semgrep-scan.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-sentrux-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-secrets-analysis.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-skill-optimizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-spdd-spec-planner.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-syntax-error-quality.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-team-evolution-insights.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-team-status.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-testify-uber-super-expert.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-token-consumption-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-workflow-updater.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-windows-terminal-integration-builder.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/daily-yamllint-fixer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/dead-code-remover.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/dependabot-burner.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/dependabot-go-checker.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/dependabot-repair.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/deployment-incident-monitor.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/designer-drift-audit.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/design-decision-gate.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/detection-analysis-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/dev-hawk.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/dev.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/dictation-prompt.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/discussion-task-miner.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/docs-noob-tester.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/draft-pr-cleanup.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/delight.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/duplicate-code-detector.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/eslint-miner.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/eslint-monster.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/eslint-refiner.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/example-failure-category-filter.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/example-permissions-warning.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/example-workflow-analyzer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/firewall-escape.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/firewall.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/functional-pragmatist.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/github-mcp-structural-analysis.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/github-mcp-tools-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/github-remote-mcp-auth-test.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/glossary-maintainer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/go-fan.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/go-logger.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/go-pattern-detector.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/gpclean.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/grumpy-reviewer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/hippo-embed.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/hourly-ci-cleaner.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/impeccable-skills-reviewer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/instructions-janitor.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/issue-arborist.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/issue-monster.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/issue-triage-agent.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/jsweep.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/layout-spec-maintainer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/lint-monster.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/linter-miner.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/lockfile-stats.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/mattpocock-skills-reviewer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/mcp-inspector.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/mergefest.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/metrics-collector.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/necromancer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/notion-issue-summary.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/objective-impact-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/org-health-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/outcome-collector.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/pdf-summary.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/plan.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/poem-bot.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/portfolio-analyst.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/pr-code-quality-reviewer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/pr-description-caveman.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/pr-nitpick-reviewer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/pr-sous-chef.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/pr-triage-agent.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/prompt-clustering-analysis.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/python-data-charts.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/q.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/refactoring-cadence.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/refiner.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/release.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/repo-audit-analyzer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/repo-tree-map.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/repository-quality-improver.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/research.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/ruflo-backed-task.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/safe-output-health.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/scout.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/schema-consistency-checker.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/schema-feature-coverage.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/security-compliance.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/security-review.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/semantic-function-refactor.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/sergo.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/skillet.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/slide-deck-maintainer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-agent-all-merged.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-agent-all-none.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-agent-public-approved.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-agent-public-none.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-agent-scoped-approved.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-antigravity.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-call-workflow.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-ci.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-claude-on-copilot.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-claude.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-codex.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-copilot-arm.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-copilot-aoai-apikey.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-copilot-aoai-entra.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-copilot-sdk.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-copilot-sub-agents.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-copilot.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-create-cross-repo-pr.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-crush.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-gemini.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-multi-pr.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-opencode.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-otel-backends.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-pi.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-project.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-service-ports.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-temporary-id.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-test-tools.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-update-cross-repo-pr.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/smoke-workflow-call-with-inputs.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job (with artifact prefixing for workflow_call).
.github/workflows/smoke-workflow-call.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job (with artifact prefixing for workflow_call).
.github/workflows/spec-enforcer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/spec-extractor.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/spec-librarian.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/static-analysis-report.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/stale-pr-cleanup.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/stale-repo-identifier.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/step-name-alignment.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/sub-issue-closer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/super-linter.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/technical-doc-writer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/terminal-stylist.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/test-create-pr-error-handling.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/test-dispatcher.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/test-project-url-default.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/test-quality-sentinel.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/test-workflow.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/tidy.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/typist.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/ubuntu-image-analyzer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/uk-ai-operational-resilience.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/unbloat-docs.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/update-astro.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/video-analyzer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/visual-regression-checker.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/weekly-blog-post-writer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/weekly-editors-health-check.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/weekly-issue-summary.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/weekly-safe-outputs-spec-review.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/workflow-generator.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/workflow-health-manager.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/workflow-normalizer.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.
.github/workflows/workflow-skill-extractor.lock.yml Regenerates compiled workflow to include the new manifest download step in the conclusion job.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 264/264 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment thread pkg/workflow/notify_comment_test.go
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (>100 new lines) but does not have a linked Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/43056-download-safe-outputs-manifest-in-conclusion-job.md — review and complete it before merging.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next
  1. Review the draft ADR committed to your branch — it was generated from the PR diff
  2. Complete the missing sections — add context the AI could not infer, refine the decision rationale, and list real alternatives you considered
  3. Commit the finalized ADR to docs/adr/ on your branch
  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-43056: Download safe-outputs-items manifest in conclusion job

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

❓ Why ADRs Matter

"AI made me procrastinate on key design decisions. Because refactoring was cheap, I could always say 'I’ll deal with this later.' Deferring decisions corroded my ability to think clearly."

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context — What is the problem? What forces are at play?
  • Decision — What did you decide? Why?
  • Alternatives Considered — What else could have been done?
  • Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number (e.g., 43056-download-safe-outputs-manifest-in-conclusion-job.md for PR #43056).

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · 53.6 AIC · ⌖ 10 AIC · ⊞ 8.5K ·
Comment /review to run again

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

⚠️ Test Quality Score: 78/100 — Acceptable

Analyzed 10 test(s): 10 design, 0 implementation, 0 violation(s).

📊 Metrics (10 tests)
Metric Value
Analyzed 10 (Go: 5, JS: 5)
✅ Design 10 (100%)
⚠️ Implementation 0 (0%)
Edge/error coverage 6 (60%)
Duplicate clusters 0
Inflation YES (logs_usage_activity_test.go: 3.94:1)
🚨 Violations 0
Test File Classification Issues
it("returns null when manifest does not exist") generate_usage_activity_summary.test.cjs design_test none
it("returns null when manifest file is empty") generate_usage_activity_summary.test.cjs design_test none
it("returns null when manifest contains only blank lines") generate_usage_activity_summary.test.cjs design_test none
it("counts items by type from a valid manifest") generate_usage_activity_summary.test.cjs design_test none
it("skips lines with missing or empty type field") generate_usage_activity_summary.test.cjs design_test none
TestApplyUsageActivitySummaryBackfillsSafeItemsCount logs_usage_activity_test.go design_test none
TestApplyUsageActivitySummaryDoesNotOverwriteExistingSafeItemsCount logs_usage_activity_test.go design_test none
TestApplyUsageActivitySummarySafeItemsCountZeroNotBackfilled logs_usage_activity_test.go design_test none
TestLoadUsageActivitySummaryWithSafeOutputs logs_usage_activity_test.go design_test none
TestConclusionJobIncludesUsageArtifactSteps (additions) notify_comment_test.go design_test none
⚠️ Flagged Tests (1 — test inflation)

logs_usage_activity_test.go — test inflation: 71 new lines added vs 18 in production (logs_usage_activity.go), a 3.94:1 ratio (threshold: 2:1). This is above the threshold but justified: the 4 new tests cover distinct behavioral branches of the new SafeItemsCount backfill logic (backfill-when-absent, no-overwrite-when-present, zero-not-backfilled, JSON deserialization). No guideline violations.

Verdict

Passed. 0% implementation tests (threshold: 30%). Test inflation flagged for logs_usage_activity_test.go (3.94:1) but coverage quality is high with no violations.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🧪 Test quality analysis by Test Quality Sentinel · 50.4 AIC · ⌖ 10.2 AIC · ⊞ 6.8K ·
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 78/100 (Acceptable). 0% implementation tests (threshold: 30%). No guideline violations. Test inflation flagged for logs_usage_activity_test.go (3.94:1) but justified by thorough behavioral coverage of the new SafeItemsCount backfill logic.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: fix(safe-outputs): bundle safe-outputs-items manifest into restore SafeItemsCount

Overall assessment: Correct approach, one minor maintainability note.

The fix correctly addresses the root cause: the safe-output-items.jsonl manifest was never available when generate_usage_activity_summary.cjs ran in the conclusion job, causing SafeItemsCount=0 on every run since 2026-05-31.

What the PR does well

  • Correct placement: the new Download safe outputs items manifest step is prepended before "Collect usage artifact files" (verified by the ordering test in notify_comment_test.go), so the manifest is present when the summary generator runs.
  • Safe failure mode: continue-on-error: true ensures workflows that emit no safe outputs don't fail the conclusion job.
  • Zero-item protection in Go backfill: the backfill in logs_usage_activity.go only fires when summary.SafeOutputs.TotalItems > 0, correctly avoiding masking genuine zero-item runs.
  • Well-tested: new tests cover the JS manifest parser (null on missing/empty/blank file, counts by type, skips bad entries) and the Go backfill (backfill fires, not overwritten if already set, no backfill on zero).
  • Clean struct design: usageActivitySafeOutputs follows the existing pattern of the other activity structs.

One inline note

Hardcoded path in parseSafeOutputsManifest() — see inline comment. Non-blocking, but worth aligning with the MANIFEST_FILE_PATH constant that already exists in constants.cjs.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · 89.4 AIC · ⌖ 6.59 AIC · ⊞ 4.9K

* @returns {{ total_items: number, items_by_type: Record<string, number> } | null}
*/
function parseSafeOutputsManifest() {
const manifestPath = "/tmp/gh-aw/safe-output-items.jsonl";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The manifest path is hardcoded as /tmp/gh-aw/safe-output-items.jsonl directly here, but MANIFEST_FILE_PATH from constants.cjs already defines this exact path — and sibling modules like safe_output_manifest.cjs import it from there.\n\nConsider reusing the constant to avoid silent drift if the path ever changes:\njs\nconst { MANIFEST_FILE_PATH } = require('./constants.cjs');\n// ...\nconst manifestPath = MANIFEST_FILE_PATH;\n\n\n@copilot please address this.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Skills-Based Review Summary 🧠

Applied /diagnosing-bugs, /tdd, and /codebase-design to PR #43056.

The approach is correct — downloading the safe-outputs-items artifact before the summary script runs, parsing it in JS, and backfilling SafeItemsCount in Go are all the right moves. However, there is one blocking correctness bug that will prevent the fix from working at all:

🔴 Ordering bug in logs_run_processor.go (not modified in this PR)

// Line 346 — backfill runs first ...
applyUsageActivitySummaryToResult(usageActivitySummary, &result, !hasFirewallArtifact)

// Line 349 — ... but then unconditionally resets SafeItemsCount to 0 (when manifest absent)
result.Run.SafeItemsCount = len(extractCreatedItemsFromManifest(runOutputDir))

The backfill condition result.Run.SafeItemsCount == 0 is always true at call time (line 346), but the result is always overwritten by line 349. Swapping these two lines is the minimal fix required for this PR to actually restore SafeItemsCount.

🟡 Supporting improvements

Issue File Comment
Missing integration test for the combined pipeline logs_usage_activity_test.go See inline comment
Shared global manifest path blocks test isolation generate_usage_activity_summary.test.cjs See inline comment
Hardcoded path in parseSafeOutputsManifest generate_usage_activity_summary.cjs See inline comment
No id: on download step (masks silent failures) notify_comment.go See inline comment

The implementation quality of the new code is solid — good error handling, correct null returns, and thorough unit tests. The primary ask is to fix the call-site ordering and make the path injectable.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 162.1 AIC · ⌖ 8.27 AIC · ⊞ 6.6K ·
Comment /matt to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs, /tdd, and /codebase-design — requesting changes on one correctness bug and three supporting improvements.

📋 Key Themes & Highlights

Key Themes

  • Ordering bug (blocks the fix): applyUsageActivitySummaryToResult in logs_run_processor.go runs before extractCreatedItemsFromManifest unconditionally resets SafeItemsCount to 0. The backfill guard SafeItemsCount == 0 is therefore always satisfied at call time, but the result is always overwritten by the subsequent line — making the entire backfill logic dead code. This file was not changed in the PR, but the call-site ordering must be fixed for the bug to actually be resolved.
  • Test isolation: parseSafeOutputsManifest tests write to a shared global path /tmp/gh-aw/safe-output-items.jsonl; the hardcoded path in the implementation forces this. Accepting the path as an optional parameter would close both issues at once.
  • Observability gap: The new download-artifact step has no id:, so continue-on-error failures are invisible to downstream steps and harder to diagnose.
  • Missing integration test: Unit tests for applyUsageActivitySummaryToResult are thorough, but no test exercises the combined processor pipeline to catch the ordering regression.

Positive Highlights

  • parseSafeOutputsManifest is well-structured: consistent error handling, correct null-returns for all empty/missing cases
  • ✅ Good use of continue-on-error: true on the download step — workflows without safe outputs won't fail
  • ✅ The usageActivitySafeOutputs struct mirrors the JSON schema cleanly and is exported from the module for testing
  • ✅ Test coverage for the new Go and JS code is comprehensive — 4 new Go tests and 5 new JS tests covering all boundary conditions

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 162.1 AIC · ⌖ 8.27 AIC · ⊞ 6.6K
Comment /matt to run again


applyUsageActivitySummaryToResult(summary, &result, true)

assert.Equal(t, 3, result.Run.SafeItemsCount, "SafeItemsCount should be backfilled from usage summary safe_outputs.total_items")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] This test calls applyUsageActivitySummaryToResult in isolation and correctly validates the backfill — but there is no integration test in logs_run_processor.go that verifies the overall call order: currently applyUsageActivitySummaryToResult runs before extractCreatedItemsFromManifest in the processor, so the backfill is always overwritten. A test that exercises the full downloadRunArtifactsConcurrent path with a missing manifest artifact (so extractCreatedItemsFromManifest returns 0) would catch this ordering bug.

💡 What to add

Add a test to logs_run_processor_test.go (or equivalent) that:

  1. Sets up a run directory without a safe-output-items.jsonl manifest file
  2. Provides a usage/activity/summary.json with safe_outputs.total_items > 0
  3. Invokes the processor pipeline
  4. Asserts that result.Run.SafeItemsCount equals the summary value (not 0)

This would have caught the ordering bug before merge.

@copilot please address this.

});

describe("parseSafeOutputsManifest", () => {
const manifestPath = "/tmp/gh-aw/safe-output-items.jsonl";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The manifest path /tmp/gh-aw/safe-output-items.jsonl is shared across all test runs in the suite. If another test or a parallel vitest worker writes to the same path, tests can interfere with each other — the afterEach cleanup only helps within the same describe block.

💡 Suggested fix — use a unique tmp path per test

Follow the same pattern used in the parseFirewallLogs suite: create a unique directory per test and pass it as a parameter, or stub fs so the manifest path can be overridden:

// Option A: unique dir per test
const testDir = fs.mkdtempSync('/tmp/gh-aw-test-');
const manifestPath = path.join(testDir, 'safe-output-items.jsonl');

Or expose the manifest path as a parameter to parseSafeOutputsManifest(manifestPath?) so tests can inject a unique path.

@copilot please address this.

* @returns {{ total_items: number, items_by_type: Record<string, number> } | null}
*/
function parseSafeOutputsManifest() {
const manifestPath = "/tmp/gh-aw/safe-output-items.jsonl";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] parseSafeOutputsManifest() has a hardcoded path /tmp/gh-aw/safe-output-items.jsonl. Every other parser in this file (parseFirewallLogs, parseGatewayLogs, parseSessionLogs) derives its paths via globSync or a configurable variable. The hardcoded path makes the function untestable with unique paths per test (the tests must write to the exact global path) and makes the module harder to reuse in non-standard environments.

💡 Accept an optional path parameter
function parseSafeOutputsManifest(manifestPath = "/tmp/gh-aw/safe-output-items.jsonl") {
  if (!fs.existsSync(manifestPath)) {
    return null;
  }
  // ...
}

Tests can then pass a temp-unique path, and main() continues to use the default.

@copilot please address this.

usageArtifactName := prefix + "usage"
safeOutputsItemsArtifactName := prefix + constants.SafeOutputItemsArtifactName
return []string{
" - name: Download safe outputs items manifest\n",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] The download step has no id: field. When continue-on-error: true silently masks a failure (e.g., the artifact doesn't exist yet, or there's a naming mismatch), there is no way to detect this in subsequent steps via steps.<id>.outcome. Future debugging will require reading the full workflow log to find out whether the download actually succeeded.

💡 Add an id for observability
- name: Download safe outputs items manifest
  id: download-safe-outputs-manifest
  if: always()
  continue-on-error: true
  uses: actions/download-artifact@...

Then a later diagnostic step can check ${{ steps.download-safe-outputs-manifest.outcome }} without scanning raw logs.

@copilot please address this.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Generated by 👨‍🍳 PR Sous Chef · 6.92 AIC · ⌖ 15.9 AIC · ⊞ 6.4K ·

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Gemini

  • GitHub MCP Testing: ✅
  • Web Fetch Testing: ✅
  • File Writing Testing: ✅
  • Bash Tool Testing: ✅
  • Build gh-aw: ❌

Overall Status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

Smoke Gemini — Powered by Gemini ·
Comment /smoke-gemini to run again

@pelikhan

pelikhan commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

@copilot merge main and recompile

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Smoke test: FAIL

  • feat: migrate 10 copilot sub-agents from small to kiwi / mai-code: ✅
  • fix: prevent tool denial failures in daily-agent-of-the-day-blog-writer: ✅
  • Serena find_symbol: ❌
  • web-fetch: ❌
  • set_issue_field: ⏭️ skipped

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex · 13.6 AIC · ⊞ 10.9K ·
Comment /smoke-codex to run again

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Comment Memory

Quiet test runner
Git and browsers hum in sync
Smoke turns into proof

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex · 13.6 AIC · ⊞ 10.9K ·
Comment /smoke-codex to run again

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

💥 Smoke Test: Claude — Run 28636821292

Core #1-12: ✅ all passed (GitHub read, gh CLI, make build, Playwright, Tavily, file/bash, discussion, AW status, Slack, code-scan alert, check-run)
PR review #13-17: ✅ update PR, 2 review comments, submit review, resolve thread, add reviewer
#18 push: ⚠️ skipped — branch history has files outside allowed-files config
#19 close PR: ⚠️ skipped — no safe test PR

Overall: PARTIAL (all executable tests passed; 2 skipped by design)

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · 100.4 AIC · ⌖ 31.5 AIC · ⊞ 8.7K ·
Comment /smoke-claude to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💥 Automated smoke test review - all systems nominal!

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · 100.4 AIC · ⌖ 31.5 AIC · ⊞ 8.7K
Comment /smoke-claude to run again

@@ -77,9 +77,20 @@ func (c *Compiler) buildConclusionJob(data *WorkflowData, mainJobName string, sa

// buildUsageArtifactUploadSteps creates steps that collect and upload a compact usage artifact.
// The artifact includes aw_info.json, aw-info.jsonl, agent_usage.json, agent_usage.jsonl, detection_usage.jsonl, and agent/detection token usage JSONL files (when present).
// It also downloads the safe-outputs-items artifact so that generate_usage_activity_summary.cjs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice — the doc comment clearly explains the new manifest download behavior. 👍

func buildUsageArtifactUploadSteps(prefix string, pinAction func(string) string) []string {
usageArtifactName := prefix + "usage"
safeOutputsItemsArtifactName := prefix + constants.SafeOutputItemsArtifactName

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deriving the artifact name from the constant keeps this DRY — looks good.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Smoke tests: 15 total, 0 passed, 15 failed ❌\nOverall status: FAIL\nTested PRs: none\n@app/copilot-swe-agent @pelikhan

📰 BREAKING: Report filed by Smoke Copilot - AOAI (Entra) · 29.6 AIC · ⌖ 5.76 AIC · ⊞ 17.6K ·
Comment /smoke-copilot-aoai-entra to run again
Add label smoke to run again

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

PR: fix(safe-outputs): bundle safe-outputs-items manifest into usage artifact to restore SafeItemsCount
1✅ 2❌ 3✅ 4✅ 5❌ 6✅
7✅ 8✅ 9✅ 10✅ 11✅
12✅ 13✅ 14❌ 15✅ 16✅
Overall: FAIL | Author: @app/copilot-swe-agent | Assignees: pelikhan, Copilot

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · 630.6 AIC · ⌖ 9.82 AIC · ⊞ 19K ·
Comment /smoke-copilot to run again
Add label smoke to run again

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Comment Memory

Signal lights whisper
Bots rehearse their careful steps
Logs bloom into dawn

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · 630.6 AIC · ⌖ 9.82 AIC · ⊞ 19K ·
Comment /smoke-copilot to run again
Add label smoke to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smoke test review submitted successfully.

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · 630.6 AIC · ⌖ 9.82 AIC · ⊞ 19K
Comment /smoke-copilot to run again
Add label smoke to run again

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Smoke Test Results:\n- List merged PRs: ✅\n- mcpscripts-gh PR list: ❌\n- Serena find_symbol: ❌\n- Playwright open/screenshot: ✅\n- Fetch GitHub: ✅\n- Create/read file: ✅\n- Discussion query: ❌\n- Build project: ❌\nOverall: FAIL

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot - AOAI (apikey) · 141.1 AIC · ⌖ 15.9 AIC · ⊞ 17.9K ·
Comment /smoke-copilot-aoai-apikey to run again
Add label smoke to run again

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

@copilot merge main and recompile

Done — merged origin/main (all conflicts were in generated .lock.yml files, resolved by taking main's version) and recompiled all 258 workflows successfully.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

Generated by 👨‍🍳 PR Sous Chef · 6.07 AIC · ⌖ 7.49 AIC · ⊞ 6.4K ·

…rdering test

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, address unresolved review comments, and rerun checks once the branch is up to date.

...

Fixed the ordering bug in logs_run_processor.goextractCreatedItemsFromManifest now runs before applyUsageActivitySummaryToResult so the backfill only activates when the manifest returns 0. Added TestExtractThenApplyProcessorOrderingBackfillsSafeItemsCount to catch regressions (commit fix(safe-outputs): fix processor ordering bug and add extract+apply ordering test). All local checks pass.

@pelikhan pelikhan merged commit d87a2f2 into main Jul 3, 2026
28 checks passed
@pelikhan pelikhan deleted the copilot/deep-report-fix-safe-output-aggregator-undercount branch July 3, 2026 05:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[deep-report] Fix safe-output aggregator undercount (SafeItemsCount=0 / read_only since 05-31)

4 participants