[audit-workflows] 🛡️ Agentic Workflow Audit — 2026-06-23 (92.5% healthy; copilot-sdk LONG-RUN dominant)

[github-actions[bot]]

🛡️ Agentic Workflow Audit — 2026-06-23

Window: 2026-06-23 10:36Z → 21:22Z (~10.8h active). Overnight (06-22 21:44Z → 06-23 10:36Z) had 0 runs — confirmed quiet via empty continuation, so this 24h window is effectively complete despite the short active span.

Overview

Metric	Value
Total runs	174
Success / Failure	161 / 13
Overall success rate	92.5%
Prod-main schedule	92 runs, 10 fail → 89.1% (91.1% ex-2-intentional)
Missing tools / data / MCP failures	0 / 0 / 0 ✅
AI credits (aic)	~12,635
Action minutes	2,024
GitHub API calls	1,967

Engines: copilot 108 · claude 28 · pi 18 · codex 7 · antigravity 2 · gemini 2. No null_metadata runs this window (clean classification).

Health verdict: On-baseline and healthy. No new failure class, no shared incident, zero missing-tools/data/MCP failures. Two chronic offenders went quiet: Avenger (0 runs, 2nd consecutive day — schedule likely paused) and Skillet (0 fails, de-escalation holds).

📊 Trends

Success rate sits at 92.5%, comfortably above the 90% guide line and continuing the rebound from the 06-20 Skillet-driven dip (69.6%). The last four days (91.5 → 69.6 → 91.7 → 94.5 → 92.5) show the fleet has re-stabilized once the Skillet push-trigger storm cleared; remaining failures are almost entirely known-recurring families rather than new breakage.

Token volume is claude-measured only — copilot (the majority engine at 108/174) and codex do not report token counts, so recent bars are sparse and the absolute level understates true fleet consumption. AI-credit (aic) spend (~12.6K) is the more complete cost signal this window, and it was skewed by two long-running failed copilot jobs (see anomaly below).

🔎 Failure Breakdown (13 total)

8 real prod-main failures — all known-recurring families

Workflow	Engine	Dur	Cost	Family
Daily Agent of the Day Blog Writer	copilot→claude-sonnet-4.6	52.6m	$144	copilot-sdk LONG-RUN ⚠️
Daily Safe Output Integrator	copilot	24.5m	$145	copilot-sdk LONG-RUN
PR Sous Chef (x2)	copilot	6.2m / 9.8m	– / $43	sdk 0-tok startup + safe-output
Daily Issues Report Generator	copilot	9.4m	0-tok	chroot-node (day 3 consecutive)
Daily Cache Strategy Analyzer	codex	7.3m	0-tok	codex binary/model-not-found (~100% fail)
[aw] Failure Investigator (6h)	claude	3.7m	0-tok	claude agent-job 0-tok (NEW wf)
Super Linter Report	null	4.1m	–	recurring single-occurrence

Dominant family: copilot-sdk-driver LONG-RUN agent-job fail — 4 of 8 real prod-main fails. Either the agent runs 20–53 min then the agent job concludes failure, or it never starts (0-tok). Day 21+ of this chronic path.

2 intentional test failures (by design) + 3 dev/PR-branch failures

Intentional (excluded from health math): Daily Credit Limit Test (Intentionally Broken), Daily Max AI Credits Test (Intentionally Fails).

Dev/PR branch (copilot/ feature branches, not main):* Changeset Generator (codex PR), Design Decision Gate 🏗️ (claude PR), Smoke Claude on Copilot (workflow_dispatch probe on copilot/fix-clause-copilot-configuration — by-design config-fix probe).

⚠️ Anomaly of the Day — Blog Writer 52.6 min / $144 with 87% firewall-block rate

Daily Agent of the Day Blog Writer (run 28036291237) is the most expensive single failing run of the window: 52.6 minutes, $144.15 AI credits, 2.59M cache-read tokens, 56 assistant messages / 75 tool calls — model claude-sonnet-4.6 routed via the copilot engine. Its firewall summary shows 1,336 of 1,541 requests blocked (87%), including repeated proxy.golang.org:443 denials. The signature suggests the agent spent most of its 52 minutes retrying blocked network egress before the agent job ultimately failed — a costly burn with no output. Paired with Daily Safe Output Integrator (24.5m/$145), these two runs dominated wasted aic spend.

🔁 Recurrence Tracking (repo memory updated)

• copilot-sdk-driver LONG-RUN → recurrence count 17; dominant fail family again (4/8).
• chroot-node-not-available (Daily Issues Report Generator) → count 5, now 3 consecutive days (06-21/06-22/06-23) unfixed.
• codex binary/model-not-found (Daily Cache Strategy Analyzer) → count 5, still ~100% fail.
• pr-sous-chef / safe-output hotspot → count 6 (observability flagged 33% fail rate, 2/6).
• Avenger err-config → held at 14; absent 2nd day (watch next fire to confirm fix vs pause).
• claude agent-job 0-tok → count 3; new affected wf [aw] Failure Investigator (6h).

✅ Next Actions (no new recommendations — existing fixes still pending)

1. Bind-mount node into the AWF chroot (rec-chroot-node-bindmount) — Issues Report Generator now failing 3 days straight; highest-confidence fixable item.
2. Add a firewall-block / retry-budget guard to copilot long-runs — Blog Writer's 52min/$144 burn on 87%-blocked egress is the clearest cost-waste case for a fail-fast guard.
3. Fix codex gh-aw binary path (fix-cache-strategy-binary-path) — Cache Strategy Analyzer remains ~100% fail, count 5.
4. Confirm Avenger root-cause fix vs paused-schedule on its next scheduled fire.

References:

• §28036291237 — Blog Writer 52.6m/$144 anomaly
• §28050869209 — Safe Output Integrator LONG-RUN
• §28036993734 — Issues Report Generator (chroot-node day 3)

Generated by 🔍 Agentic Workflow Audit Agent · 272.8 AIC · ⌖ 37.5 AIC · ⊞ 7K · ◷

• expires on Jun 24, 2026, 2:05 PM UTC-08:00