You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: .github/workflows/agentic-token-audit.md
+29-24Lines changed: 29 additions & 24 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
description: Daily audit of token usage across all agentic workflows with historical trend tracking
2
+
description: Daily audit of AI credit spend across all agentic workflows with historical trend tracking
3
3
on:
4
4
schedule:
5
5
- cron: "daily around 12:00 on weekdays"
@@ -71,15 +71,15 @@ steps:
71
71
timeout-minutes: 25
72
72
---
73
73
74
-
# Daily Agentic Workflow Token Usage Audit
74
+
# Daily Agentic Workflow AI Credit Spend Audit
75
75
76
-
You are the Agentic Workflow Token Auditor — a workflow that tracks daily token consumption across all agentic workflows in this repository and maintains a historical record for trend analysis.
76
+
You are the Agentic Workflow Auditor — a workflow that tracks daily AI credit spend and token consumption across all agentic workflows in this repository and maintains a historical record for trend analysis.
77
77
78
78
## Mission
79
79
80
-
1. Parse the pre-downloaded agentic workflow logs and compute per-workflow token usage metrics.
80
+
1. Parse the pre-downloaded agentic workflow logs and compute per-workflow AI credit spend and token usage metrics.
81
81
2. Persist today's snapshot to repo-memory so the optimizer (and future runs of this audit) can read historical data.
82
-
3. Publish a concise audit issue summarizing today's usage and trend highlights.
82
+
3. Publish a concise audit issue summarizing today's AI credit spend and trend highlights.
83
83
84
84
## Data Sources
85
85
@@ -103,8 +103,9 @@ Each element of `.runs` is a `RunData` object with (among others):
4. Compute an overall summary: total runs, total AI credits, total tokens, total action minutes.
134
+
5. Sort workflows descending by `total_ai_credits`.
134
135
6. Save the result to `/tmp/gh-aw/token-audit/audit_snapshot.json` with this shape:
135
136
136
137
```json
@@ -139,13 +140,16 @@ Write a Python script to `/tmp/gh-aw/token-audit/process_audit.py` and run it. T
139
140
"period_days": 30,
140
141
"overall": {
141
142
"total_runs": N,
143
+
"total_ai_credits": F,
142
144
"total_tokens": N,
143
145
"total_action_minutes": F
144
146
},
145
147
"workflows": [
146
148
{
147
149
"workflow_name": "...",
148
150
"run_count": N,
151
+
"total_ai_credits": F,
152
+
"avg_ai_credits": F,
149
153
"total_tokens": N,
150
154
"avg_tokens": N,
151
155
"total_turns": N,
@@ -159,15 +163,15 @@ Write a Python script to `/tmp/gh-aw/token-audit/process_audit.py` and run it. T
159
163
}
160
164
```
161
165
162
-
Handle null/missing `token_usage` by treating them as 0.
166
+
Handle null/missing `ai_credits` and `token_usage` by treating them as 0.
163
167
164
168
## Phase 2 — Persist Snapshot to Repo-Memory
165
169
166
170
1. Read the snapshot from `/tmp/gh-aw/token-audit/audit_snapshot.json`.
167
171
2. Copy it to `/tmp/gh-aw/repo-memory/default/YYYY-MM-DD.json` (today's UTC date).
168
172
3. This file is what the optimizer workflow reads to identify high-usage workflows.
169
173
170
-
Also maintain a rolling summary file at `/tmp/gh-aw/repo-memory/default/rolling-summary.json` that contains an array of daily overall totals (date, total_tokens, total_runs, total_action_minutes) for the last 90 entries. Load the existing file, append today's entry, trim to 90, and save.
174
+
Also maintain a rolling summary file at `/tmp/gh-aw/repo-memory/default/rolling-summary.json` that contains an array of daily overall totals (date, total_ai_credits, total_tokens, total_runs, total_action_minutes) for the last 90 entries. Load the existing file, append today's entry, trim to 90, and save.
171
175
172
176
Do not append a synthetic zero-valued entry to `rolling-summary.json` when either of these conditions is true:
173
177
@@ -180,8 +184,8 @@ Report those two cases differently in the issue as described below so the empty-
180
184
181
185
Create up to two chart images in `/tmp/gh-aw/token-audit/charts/` using Python, `matplotlib`, and `seaborn` with `whitegrid` styling:
182
186
183
-
1.**Token usage by workflow** (`token_by_workflow.png`): a horizontal bar chart of the top 15 workflows by total tokens from `audit_snapshot.json`.
184
-
2.**Historical token trend** (`token_trend.png`): a line chart from `rolling-summary.json`.
187
+
1.**AI credit spend by workflow** (`ai_credits_by_workflow.png`): a horizontal bar chart of the top 15 workflows by total AI credits from `audit_snapshot.json`.
188
+
2.**Historical AI credit trend** (`ai_credits_trend.png`): a line chart from `rolling-summary.json`.
185
189
186
190
Chart requirements:
187
191
@@ -191,8 +195,8 @@ Chart requirements:
191
195
- Save only PNG files.
192
196
- If there are fewer than 2 rolling-summary points, skip the trend chart and explain why in the issue.
193
197
- After generating each chart, call `upload_asset` with its file path.
194
-
- In the issue template below, replace `UPLOAD_URL_WORKFLOW_PLACEHOLDER` with the URL returned for `token_by_workflow.png`.
195
-
- In the issue template below, replace `UPLOAD_URL_TREND_PLACEHOLDER` with the URL returned for `token_trend.png`.
198
+
- In the issue template below, replace `UPLOAD_URL_WORKFLOW_PLACEHOLDER` with the URL returned for `ai_credits_by_workflow.png`.
199
+
- In the issue template below, replace `UPLOAD_URL_TREND_PLACEHOLDER` with the URL returned for `ai_credits_trend.png`.
196
200
- If a chart is skipped, omit that image markdown line entirely instead of leaving a placeholder behind.
197
201
198
202
## Phase 4 — Publish Audit Issue
@@ -213,45 +217,46 @@ Create an issue with these sections:
213
217
214
218
- **Period**: last 24 hours (YYYY-MM-DD to YYYY-MM-DD)
215
219
- **Total runs**: N
220
+
- **Total AI credits**: N.NN AIC
216
221
- **Total tokens**: N (formatted with commas)
217
222
- **Total Actions minutes**: X.X min
218
223
- **Active workflows**: N
219
224
220
-
### 🏆 Top 5 Workflows by Token Usage
225
+
### 🏆 Top 5 Workflows by AI Credit Spend
221
226
222
-
| Workflow | Runs | Total Tokens | Avg Tokens |
227
+
| Workflow | Runs | Total AI Credits | Avg AI Credits |
223
228
|---|---|---|---|
224
229
| ... | ... | ... | ... |
225
230
226
231
### 📈 Trends
227
232
228
233
Embed chart images using uploaded asset URLs when available:
229
234
230
-

235
+

Copy file name to clipboardExpand all lines: .github/workflows/agentic-token-optimizer.md
+16-12Lines changed: 16 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
description: Daily optimizer that identifies a high-token-usage agentic workflow, audits its runs, and recommends efficiency improvements including inline sub-agent refactors when warranted
2
+
description: Daily optimizer that identifies high-AI-credit-spend agentic workflows, audits their runs, and recommends efficiency improvements including inline sub-agent refactors when warranted
You are the Agentic Workflow Token Optimizer. Pick one high-token workflow, audit recent runs, and create a conservative optimization issue with measurable improvements. Your recommendations may include prompt, tool, reliability, setup-prefix, and inline sub-agent improvements when the evidence supports them.
115
+
You are the Agentic Workflow Optimizer. Pick one high-AI-credit-spend workflow, audit recent runs, and create a conservative optimization issue with measurable improvements. Your recommendations may include prompt, tool, reliability, setup-prefix, and inline sub-agent improvements when the evidence supports them.
113
116
114
117
## Objectives
115
118
@@ -148,23 +151,24 @@ Prefer `--jq` on `gh api` calls over a separate `| jq` step when the filter is s
148
151
## Data Inputs
149
152
150
153
-`/tmp/gh-aw/token-audit/all-runs.json`: full 7-day run data (`gh aw logs --json`).
151
-
-`/tmp/gh-aw/token-audit/top-workflows.json`: pre-aggregated top 10 workflows by total tokens.
154
+
-`/tmp/gh-aw/token-audit/top-workflows.json`: pre-aggregated top 10 workflows by total AI credits.
Copy file name to clipboardExpand all lines: CHANGELOG.md
+6Lines changed: 6 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,6 +23,12 @@ Changelog entries should describe what changed for users of these workflows, not
23
23
- Omit vague filler like "various fixes" when the actual effect can be named.
24
24
- Do not include contributor handles, reviewer names, or other attribution in release entries.
25
25
26
+
## [0.3.0] - 2026-06-08
27
+
28
+
### Changed
29
+
30
+
- Updated both workflows to track and report AI credits (AIC) as the primary billing metric, aligned with the GitHub Copilot AI credits billing model introduced on June 1, 2026 (1 AIC = $0.01 USD). The daily audit snapshot and rolling summary now include `total_ai_credits`; audit and optimizer issues now lead with AI credit spend. Effective tokens remain available as a legacy compatibility field in the run data but are no longer the primary reporting metric.
Copy file name to clipboardExpand all lines: README.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,16 +1,16 @@
1
1
# `agentic-ops`
2
-
> "Audit token spend, surface waste, and optimize agentic workflows with confidence!"
2
+
> "Audit AI credit spend, surface waste, and optimize agentic workflows with confidence!"
3
3
4
-
`agentic-ops` is a focused bundle of GitHub Agentic Workflows for teams scaling agentic automation and wanting better visibility into usage, trends, and optimization opportunities. Instead of guessing which workflows are token-heavy or where token waste is hiding, this package gives you an audit trail, historical reporting, and conservative recommendations you can review before making changes.
4
+
`agentic-ops` is a focused bundle of GitHub Agentic Workflows for teams scaling agentic automation and wanting better visibility into usage, trends, and optimization opportunities. Instead of guessing which workflows are driving the highest AI credit spend or where waste is hiding, this package gives you an audit trail, historical reporting, and conservative recommendations you can review before making changes.
5
5
6
6
## Introduction
7
7
8
8
It is built for platform engineers, developer productivity teams, and repository maintainers who are scaling agentic workflows and need a practical way to keep them efficient. The bundle helps solve a common problem with AI automation: token usage grows quickly, but the signals for where to improve are scattered across workflow runs and logs. With `agentic-ops`, you get repeatable workflows that make usage measurable, optimization opportunities actionable, and efficiency work easier to operationalize.
9
9
10
10
## Key Features
11
11
12
-
-**Clear operational visibility** with a daily token audit that captures usage, trends, and workflow-level hotspots.
13
-
-**Actionable optimization guidance** that identifies high-token workflows and proposes safe, conservative improvements.
12
+
-**Clear operational visibility** with a daily audit that captures AI credit spend, usage trends, and workflow-level hotspots.
13
+
-**Actionable optimization guidance** that identifies high-AI-credit-spend workflows and proposes safe, conservative improvements.
14
14
-**Faster efficiency improvements** by helping teams find waste before it becomes recurring operational overhead.
15
15
-**Built for real GitHub workflows** using GitHub Agentic Workflows, so installation and adoption fit naturally into existing repositories.
16
16
-**Useful historical context** through shared snapshots that support trend analysis instead of one-off debugging.
@@ -41,17 +41,17 @@ Required configuration after installation:
41
41
42
42
After installation, you can use the included workflows to:
43
43
44
-
- run a daily audit of workflow token usage
45
-
- identify the workflows consuming the most tokens
44
+
- run a daily audit of AI credit spend and workflow token usage
45
+
- identify the workflows consuming the most AI credits
46
46
- generate optimization recommendations grounded in recent run data
47
47
- surface workflows that are good candidates for inline sub-agent refactors
48
48
49
49
Included workflows:
50
50
51
51
| Workflow | What it does |
52
52
| ----- | --- |
53
-
|[`Daily Agentic Workflow Token Usage Audit`](https://github.com/githubnext/agentic-ops/blob/main/workflows/agentic-token-audit.md?plain=1)| Collects recent agentic workflow usage and creates a daily audit snapshot. |
54
-
|[`Agentic Workflow Token Usage Optimizer`](https://github.com/githubnext/agentic-ops/blob/main/workflows/agentic-token-optimizer.md?plain=1)| Analyzes token-heavy workflows and proposes conservative token-reduction changes, including inline sub-agent opportunities when they are a strong fit. |
53
+
|[`Daily Agentic Workflow AI Credit Spend Audit`](https://github.com/githubnext/agentic-ops/blob/main/workflows/agentic-token-audit.md?plain=1)| Collects recent agentic workflow usage and creates a daily AI credit spend snapshot. |
54
+
|[`Agentic Workflow AI Credit Spend Optimizer`](https://github.com/githubnext/agentic-ops/blob/main/workflows/agentic-token-optimizer.md?plain=1)| Analyzes high-AI-credit-spend workflows and proposes conservative efficiency changes, including inline sub-agent opportunities when they are a strong fit. |
0 commit comments