Skip to content

Commit bddb860

Browse files
github-actions[bot]github-actions[bot]
authored
docs(site): add ado-aw-debug reference page (#749)
1 parent 57726e9 commit bddb860

2 files changed

Lines changed: 150 additions & 0 deletions

File tree

‎site/astro.config.mjs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ export default defineConfig({
5858
{ label: 'Tools', slug: 'reference/tools' },
5959
{ label: 'Runtimes', slug: 'reference/runtimes' },
6060
{ label: 'Safe Outputs', slug: 'reference/safe-outputs' },
61+
{ label: 'ado-aw-debug', slug: 'reference/ado-aw-debug' },
6162
{ label: 'Targets', slug: 'reference/targets' },
6263
{ label: 'Network', slug: 'reference/network' },
6364
{ label: 'MCP', slug: 'reference/mcp' },

‎site/src/content/docs/reference/ado-aw-debug.mdx‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
---
2+
title: "ado-aw-debug reference"
3+
description: "Debug-only front-matter section for dogfood pipelines and development workflows."
4+
---
5+
6+
:::danger[Dogfood pipelines only]
7+
Anything declared under `ado-aw-debug:` is **not** part of the regular agent surface and is not recommended for general use. These knobs exist so the team can validate `githubnext/ado-aw` changes against real Azure DevOps pipelines and file failures back to GitHub for triage. Each knob bypasses or weakens a normal safety control.
8+
:::
9+
10+
The compiler accepts a top-level `ado-aw-debug:` block in agent front matter. Currently exposed knobs:
11+
12+
| Knob | Purpose | Default |
13+
| --- | --- | --- |
14+
| `skip-integrity` | Omit the "Verify pipeline integrity" step from the generated YAML. OR-ed with the `--skip-integrity` CLI flag. | `false` |
15+
| `create-issue` | Enable the [debug-only `create-issue`](#create-issue) safe output. | absent (disabled) |
16+
17+
Unrecognised keys under `ado-aw-debug:` cause a compile-time error (`#[serde(deny_unknown_fields)]`).
18+
19+
## create-issue
20+
21+
Files a GitHub issue against an operator-configured target repository. Used to surface failures from ADO-hosted dogfood pipelines back to `githubnext/ado-aw` for triage.
22+
23+
### Why it's gated
24+
25+
`create-issue` is **default-deny** at three layers:
26+
27+
1. **MCP layer.** The SafeOutputs MCP server lists `create-issue` in `DEBUG_ONLY_TOOLS`, so the route is removed from the tool router unless the compiler explicitly opts in via `--enabled-tools`.
28+
2. **Compiler layer.** `--enabled-tools create-issue` is only emitted when `ado-aw-debug.create-issue:` is present in front matter. The compiler also rejects `safe-outputs.create-issue:` outright, so the tool can't be smuggled in via the regular safe-outputs surface.
29+
3. **Executor layer.** Stage 3 maintains a separate `ExecutionContext.debug_enabled_tools` set populated only from `ado-aw-debug:`. The executor refuses any NDJSON `create-issue` entry that isn't in that set, so a forged or smuggled NDJSON entry fails closed before any token is read.
30+
31+
### Front-matter schema
32+
33+
```yaml
34+
ado-aw-debug:
35+
create-issue:
36+
target-repo: githubnext/ado-aw # REQUIRED. Operator-only; agent has no override.
37+
title-prefix: "[pipeline-failure] " # Optional; prepended to every agent title.
38+
labels: # Optional; static labels always applied.
39+
- pipeline-failure
40+
- automated
41+
allowed-labels: # Optional; default-deny — see below.
42+
- "agent-*"
43+
- "pipeline-failure"
44+
assignees: # Optional; static assignees always applied.
45+
- "jamesdevine"
46+
max: 3 # Optional; per-run budget. Default 1.
47+
```
48+
49+
**Configuration fields:**
50+
51+
- **`target-repo`** (required) — Target GitHub repository in `owner/repo` format. The agent has no parameter to override it; you cannot redirect issues to a different repository at runtime.
52+
- **`title-prefix`** (optional) — Prepended to every agent-supplied title at execution time. The final title length (prefix + agent title) must be ≤ 256 characters; longer titles fail at Stage 3.
53+
- **`labels`** (optional) — Static labels applied unconditionally to every issue, on top of any agent-supplied labels that pass `allowed-labels`.
54+
- **`allowed-labels`** (optional, default-deny) — Allowlist for agent-supplied labels. An empty or absent list means **no agent-supplied labels are accepted**. To accept any agent label, set `allowed-labels: ["*"]` explicitly. Patterns may include `*` wildcards (e.g. `"agent-*"`).
55+
- **`assignees`** (optional) — Static assignees always added regardless of agent input. Merged with agent-supplied assignees.
56+
- **`max`** (optional, default: 1) — Per-run budget controlling the maximum number of issues the agent can file.
57+
58+
:::note[Case-insensitive label matching]
59+
Allowed-label matching is case-insensitive. It uses the same `tag_matches_pattern` helper as ADO tag allow-lists. GitHub labels are case-sensitive, so `allowed-labels: ["safe"]` will also admit `SAFE` and `Safe` — keep that in mind when modelling policy.
60+
:::
61+
62+
:::note[No assignee allowlist]
63+
There is intentionally no `allowed-assignees` allowlist in v1. If you need assignee restrictions, configure assignees only via the static `assignees:` list and skip the agent parameter.
64+
:::
65+
66+
### Agent-supplied parameters
67+
68+
The agent calls the `create-issue` MCP tool with:
69+
70+
```jsonc
71+
{
72+
"title": "Pipeline failure on main",
73+
"body": "<markdown body, ≥ 30 chars>",
74+
"labels": ["pipeline-failure"], // optional
75+
"assignees": ["copilot"] // optional
76+
}
77+
```
78+
79+
The MCP-side `Validate` impl rejects ADO pipeline-command sequences in labels and assignees. Stage 3 also neutralises `##vso[…]` in any error messages it produces, so agent-supplied content cannot escape the executor's stdout.
80+
81+
### Pipeline variable: ADO_AW_DEBUG_GITHUB_TOKEN
82+
83+
Stage 3 authenticates against GitHub using the **`ADO_AW_DEBUG_GITHUB_TOKEN`** ADO pipeline variable. The compiler emits
84+
85+
```yaml
86+
env:
87+
SYSTEM_ACCESSTOKEN: $(SC_WRITE_TOKEN) # if permissions: write is set
88+
ADO_AW_DEBUG_GITHUB_TOKEN: $(ADO_AW_DEBUG_GITHUB_TOKEN) # only when ado-aw-debug.create-issue is set
89+
```
90+
91+
into the executor step's `env:` block. The token is **not** exposed to the agent in Stage 1 — the read-only `GITHUB_TOKEN` the agent sees is a separate variable wired through `engine.env` and used only for GitHub MCP read access.
92+
93+
### Setting up the PAT
94+
95+
1. **Generate a fine-grained PAT** scoped to **only** `target-repo` (e.g. `githubnext/ado-aw`). Required permissions:
96+
- Repository access: only the target repo.
97+
- Permissions: **Issues** = Read and write. Nothing else.
98+
2. **Store as a secret pipeline variable** named exactly `ADO_AW_DEBUG_GITHUB_TOKEN`. Mark it secret. Do **not** copy it into `engine.env` or any non-secret variable.
99+
3. **Confirm the operator-configured target-repo matches the PAT scope.** The compiler validator only checks shape (`owner/repo`); it cannot verify the PAT has access. If the PAT lacks Issues:write, the Stage 3 call fails with the GitHub API error and Stage 3 reports `succeeded with issues`.
100+
4. `ado-aw secrets set` does **not** automate this variable today — set it manually in the ADO pipeline definition.
101+
102+
### Auto-footer
103+
104+
Every issue gets an auto-appended traceability footer that looks like:
105+
106+
```markdown
107+
<!-- ado-aw -->
108+
---
109+
Pipeline: `dogfood-failure-reporter`
110+
Run: <https://dev.azure.com/myorg/MyProject/_build/results?buildId=42>
111+
Trigger: `Manual`
112+
```
113+
114+
The `<!-- ado-aw -->` marker is stable so that future tooling can locate the generated content without parsing prose. The footer is built from `BUILD_BUILDID`, `BUILD_DEFINITIONNAME`, `BUILD_REASON`, `SYSTEM_TEAMFOUNDATIONCOLLECTIONURI` and `SYSTEM_TEAMPROJECT` — these are present whenever Stage 3 runs inside an ADO pipeline.
115+
116+
:::caution[Sensitive metadata]
117+
If your pipeline / org / project names are sensitive, do not enable `create-issue` against a public repo.
118+
:::
119+
120+
### Security checklist
121+
122+
Before enabling `create-issue`:
123+
124+
- [ ] Target repo's GitHub PAT is scoped to that repo only and only has Issues:write.
125+
- [ ] `ADO_AW_DEBUG_GITHUB_TOKEN` is stored as a secret pipeline variable, never hard-coded or printed.
126+
- [ ] `allowed-labels` is set explicitly. Empty means default-deny; `["*"]` accepts any agent label — pick deliberately.
127+
- [ ] `target-repo` is private if the agent's prompts or pipeline metadata are sensitive (the auto-footer publishes ADO run URLs and pipeline names).
128+
- [ ] `skip-integrity` is **not** enabled in pipelines triggered by untrusted PRs.
129+
130+
## skip-integrity
131+
132+
Equivalent to passing `--skip-integrity` on the `ado-aw compile` CLI. Setting either OR setting both omits the `Verify pipeline integrity` step from the generated YAML.
133+
134+
The integrity step downloads the same `ado-aw` binary the pipeline was compiled with and runs `ado-aw check` against the committed pipeline file. Without it, a tampered `*.yml` won't be caught at run time.
135+
136+
Use this only for short-lived dogfood pipelines where you're iterating on the compiler and re-compiling frequently.
137+
138+
**Example:**
139+
140+
```yaml
141+
ado-aw-debug:
142+
skip-integrity: true
143+
```
144+
145+
## See also
146+
147+
- [Safe Outputs](/ado-aw/reference/safe-outputs/) — regular safe-outputs surface (`create-issue` is **not** in it).
148+
- [CLI Commands](/ado-aw/setup/cli/) — `--skip-integrity` CLI flag.
149+
- [Template Markers](/ado-aw/reference/template-markers/) — `{{ executor_ado_env }}` and `{{ integrity_check }}` markers and their conditional behaviour.

0 commit comments

Comments
 (0)