Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,7 @@ fail-closed and only pauses when the agent actually proposed a reviewed output.
│ ├── exec-context-repo/ # Repository identity context source (bundled to exec-context-repo.js)
│ ├── conclusion/ # Conclusion-job reporter source (bundled to conclusion.js)
│ ├── approval-summary/ # Safe-outputs summary renderer (bundled to approval-summary.js; end-of-Agent-job summary tab)
│ ├── github-app-token/ # GitHub App token minter (bundled to github-app-token.js; mints installation token in Agent + Detection when engine.github-app-token is set)
│ └── shared/ # Shared modules across bundles (auth, ado-client, env-facts, types.gen.ts)
├── tests/ # Integration tests and fixtures
├── docs/ # Per-concept reference documentation (see index below)
Expand Down Expand Up @@ -380,7 +381,11 @@ index to jump to the right page.
- [`docs/ado-script.md`](docs/ado-script.md) — `ado-script` workspace
(`scripts/ado-script/`): the bundled TypeScript runtime helpers
(`gate.js`, `import.js`, the execution-context `exec-context-*.js`
bundles, `conclusion.js`, and `approval-summary.js`), schemars-driven
- [`docs/ado-script.md`](docs/ado-script.md) — `ado-script` workspace
(`scripts/ado-script/`): the bundled TypeScript runtime helpers
(`gate.js`, `import.js`, the execution-context `exec-context-*.js`
bundles, `conclusion.js`, `approval-summary.js`, and
`github-app-token.js`), schemars-driven
type codegen, the A2 design decision, and the bundle env contract
modelled in `src/compile/ado_bundle.rs`.
- [`docs/local-development.md`](docs/local-development.md) — local development
Expand Down
49 changes: 35 additions & 14 deletions docs/ado-script.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,18 @@ pipeline** as runtime helpers. Today it produces thirteen bundles:
review is configured), and attaches it to the build's
`ado-aw-safe-outputs` summary tab via `##vso[task.uploadsummary]`. See
[`safe-outputs.md`](safe-outputs.md).
- `github-app-token.js` — GitHub App token minter that runs immediately
before the Copilot invocation in the **Agent and Detection jobs** when
`engine.github-app-token` is configured. Builds an RS256 JWT from the App ID
(argv) + private key (masked env secret), resolves the installation for the
owner, exchanges it for an installation access token, and exposes it as a
masked same-job `GITHUB_APP_TOKEN`. Invoked again with a `revoke` argument
after the Copilot run (best-effort) to delete the token via
`DELETE /installation/token`. Compiler-owned, non-secret inputs (`--app-id`,
`--owner`, `--output-var`, `--repositories`, `--api-url`) are argv flags, not
env vars, so a pipeline variable can't shadow them (only the private key /
minted token ride in masked env). Runs outside AWF. See
[`engine.md`](engine.md#github-app-backed-copilot-engine-auth).

> **Internal-only.** `ado-script` is not a user-facing front-matter
> feature. Authors never write an `ado-script:` block in their agent
Expand Down Expand Up @@ -328,6 +340,16 @@ collection URI is likewise read from the auto-injected `SYSTEM_COLLECTIONURI`
(`scripts/ado-script/src/shared/auth.ts::getWebApi`, see #1307), so it
is never part of a step's env contract.

Not every bundle authenticates to Azure DevOps. The `github-app-token` bundle
(issue #1316) is `BundleAuth::None`: it authenticates to the **GitHub** API with
its own App JWT / minted installation token, so it carries no
`SYSTEM_ACCESSTOKEN` and no ADO predefined mirrors. Its only env var is the
masked private-key secret (`GH_APP_PRIVATE_KEY`, and `GH_APP_TOKEN` for revoke);
every other, non-secret input (App ID, owner, repositories, output-variable
name, api-url) is a single-quoted **argv flag** rather than an env var, so a
pipeline variable can never shadow it (ADO injects pipeline variables into a
step's env, but argv comes only from the compiler-authored script).


## End-to-end data flow

Expand Down Expand Up @@ -492,6 +514,9 @@ scripts/ado-script/
│ └── conclusion/ # conclusion.js entry point + Conclusion-job reporter
│ ├── index.ts # main(): inspect upstream results + safe-outputs manifest → file/append work items
│ └── __tests__/ # unit tests for signal detection and work-item filing behaviour
│ └── github-app-token/ # github-app-token.js entry point + GitHub App token minter
│ ├── index.ts # main(): RS256 JWT → resolve installation → mint installation token → masked GITHUB_APP_TOKEN
│ └── __tests__/ # unit tests for JWT signing / installation resolution / token minting
├── test/ # End-to-end smoke tests (gate, import, exec-context-pr)
├── gate.js # ncc bundle output (gitignored)
├── import.js # ncc bundle output (gitignored)
Expand All @@ -505,23 +530,19 @@ scripts/ado-script/
├── exec-context-pr-checks.js # ncc bundle output (gitignored)
├── exec-context-repo.js # ncc bundle output (gitignored)
├── conclusion.js # ncc bundle output (gitignored)
└── approval-summary.js # ncc bundle output (gitignored)
├── approval-summary.js # ncc bundle output (gitignored)
└── github-app-token.js # ncc bundle output (gitignored)
```

The release workflow (`.github/workflows/release.yml`) runs
`npm ci && npm run build`, then zips `scripts/ado-script/gate.js`,
`scripts/ado-script/import.js`,
`scripts/ado-script/exec-context-pr.js`,
`scripts/ado-script/exec-context-pr-synth.js`,
`scripts/ado-script/exec-context-manual.js`,
`scripts/ado-script/exec-context-pipeline.js`,
`scripts/ado-script/exec-context-ci-push.js`,
`scripts/ado-script/exec-context-workitem.js`,
`scripts/ado-script/exec-context-schedule.js`,
`scripts/ado-script/exec-context-pr-checks.js`,
`scripts/ado-script/exec-context-repo.js`,
`scripts/ado-script/conclusion.js`, and
`scripts/ado-script/approval-summary.js` into the
`npm ci && npm run build`, then globs `scripts/ado-script/*.js` — which
captures every bundle, including `gate.js`, `import.js`,
`exec-context-pr.js`, `exec-context-pr-synth.js`,
`exec-context-manual.js`, `exec-context-pipeline.js`,
`exec-context-ci-push.js`, `exec-context-workitem.js`,
`exec-context-schedule.js`, `exec-context-pr-checks.js`,
`exec-context-repo.js`, `conclusion.js`, `approval-summary.js`, and
`github-app-token.js` — into the
`ado-script.zip` release asset. Pipelines download that asset at
runtime by URL pinned to the compiler's `CARGO_PKG_VERSION`, verify
its SHA-256 against the `checksums.txt` asset, then extract.
Expand Down
113 changes: 113 additions & 0 deletions docs/engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ engine:
| `args` | list | `[]` | Custom CLI arguments appended after compiler-generated args. Subject to shell-safety validation and blocked from overriding compiler-controlled flags (`--prompt`, `--additional-mcp-config`, `--allow-tool`, `--allow-all-tools`, `--allow-all-paths`, `--disable-builtin-mcps`, `--no-ask-user`, `--ask-user`). |
| `env` | map | *(none)* | Engine-specific environment variables merged into the sandbox step's `env:` block. Keys must be valid env var names. Values are literal-only and must not contain ADO expressions (`$(`, `${{`, `$[`) or pipeline command injection (`##vso[`), **except** the Copilot BYOM provider keys (`COPILOT_PROVIDER_BASE_URL`, `COPILOT_PROVIDER_API_KEY`, `COPILOT_PROVIDER_BEARER_TOKEN`, `COPILOT_PROVIDER_WIRE_API`), which may carry an ADO macro (`$(...)`) expression — see [Copilot BYOM / BYOK provider configuration](#copilot-byom--byok-provider-configuration). Compiler-controlled keys (`GITHUB_TOKEN`, `PATH`, `BASH_ENV`, etc.) are blocked. |
| `command` | string | *(none)* | Custom engine executable path (skips the default engine binary installation — NuGet for `target: 1es`, GitHub Releases for all other targets). The path must be accessible inside the AWF container (e.g., `/tmp/...` or workspace-mounted paths). |
| `github-app-token` | map | *(none)* | GitHub App-backed Copilot engine authentication. When set, the compiler mints (and, by default, revokes) a GitHub App installation token in the Agent and Detection jobs and sources `GITHUB_TOKEN` from it (for Copilot only). See [GitHub App-backed Copilot engine auth](#github-app-backed-copilot-engine-auth). |


### `timeout-minutes`
Expand All @@ -42,6 +43,118 @@ The `timeout-minutes` field sets a wall-clock limit (in minutes) for the entire

When omitted, Azure DevOps uses its default job timeout (60 minutes). When set, the compiler emits `timeoutInMinutes: <value>` on the agentic job.

### GitHub App-backed Copilot engine auth

By default the Copilot engine authenticates with the `GITHUB_TOKEN` pipeline
variable you provide (`ado-aw secrets set GITHUB_TOKEN <pat>`). For
organization-backed Copilot authentication you can instead have the compiler
mint a **GitHub App installation access token** at runtime — mirroring
[gh-aw's](https://github.com/githubnext/gh-aw) `create-github-app-token` model,
adapted to Azure DevOps.

```yaml
engine:
id: copilot
github-app-token:
app-id: 1234567 # literal App ID or client ID (required)
owner: octo-org # installation owner (org or user login)
repositories: [octo-repo] # optional; scopes the token to these repos
# api-url: https://ghe.example.com/api/v3 # optional; GHES base URL
# skip-token-revocation: false # optional; revoke by default
# private-key: MY_SECRET_VAR # optional override; see below
```

Store the private key once and you're done:

```bash
ado-aw secrets set GITHUB_APP_PRIVATE_KEY "$(cat app-private-key.pem)"
```

| Field | Required | Description |
|-------|----------|-------------|
| `app-id` | yes | The GitHub App ID — a **literal** value: either a numeric App ID (e.g. `1234567`, quoted or unquoted) or an alphanumeric client ID (e.g. `Iv23liABC…`). The App ID is not secret (it is plain per-app config, like `owner`), so it is written verbatim, not indirected through a variable. |
| `owner` | yes | GitHub installation owner (organization or user login). |
| `repositories` | no | Repository names (owner-relative) to scope the installation token to. Omit to span every repository the installation grants. |
| `api-url` | no | GitHub API base URL. Defaults to `https://api.github.com` (GHEC). For GitHub Enterprise Server, set the `/api/v3` base URL (e.g. `https://ghe.example.com/api/v3`). Must be an `https://` URL. |
| `skip-token-revocation` | no | When `true`, do not revoke the minted token after the Copilot run. Defaults to `false` (the token is revoked — see below). |
| `private-key` | no | Name of the ADO **secret** pipeline variable holding the private key (PEM). **Defaults to `GITHUB_APP_PRIVATE_KEY`** — the compiler owns the name, exactly like `GITHUB_TOKEN`, so the common case sets no field and just runs `ado-aw secrets set GITHUB_APP_PRIVATE_KEY …`. Set this only to point at a differently-named secret (e.g. reusing an existing variable). |

#### Setup

1. Create the GitHub App and install it on the owning org/user, ensuring the
installation is suitable for Copilot organization-backed
authentication/billing in your tenant. Copilot capability is granted on the
**App/org side** — the installation token inherits it automatically; you do
not (and cannot) configure it here.
2. Store the private key as an ADO **secret** pipeline variable — the default
name `GITHUB_APP_PRIVATE_KEY` unless you set a `private-key` override:

```bash
ado-aw secrets set GITHUB_APP_PRIVATE_KEY "$(cat app-private-key.pem)"
```
3. Set `engine.github-app-token` (`app-id` + `owner` at minimum) and compile.
Each compile prints a non-blocking advisory reminding you to store the
private-key variable as a secret — this is expected (the compiler cannot
verify secrecy itself).

#### What the compiler generates

- A **token-mint step** (the `github-app-token` ado-script bundle) runs
immediately before the Copilot invocation in **both** the Agent and Detection
jobs. It builds a short-lived RS256 JWT from the App ID + private key,
resolves the installation for `owner`, exchanges it for an installation
access token (optionally scoped to `repositories`), and exposes it as a
**masked, same-job** `GITHUB_APP_TOKEN` variable.
- The Copilot engine's `GITHUB_TOKEN` is then sourced from `$(GITHUB_APP_TOKEN)`
instead of the operator-provided `$(GITHUB_TOKEN)` variable.
- A **token-revocation step** runs after the Copilot invocation in both jobs
(unless `skip-token-revocation: true`). It deletes the minted token
(`DELETE /installation/token`) so it does not remain valid for its full ~1h
lifetime — matching `actions/create-github-app-token`'s default. Revocation is
best-effort (`always()` + `continueOnError`) and never fails the build.
- The token is **never** provided to SafeOutputs, user-authored `steps:`,
ManualReview, Teardown, or Conclusion.

The mint/revoke steps run **outside** the AWF network sandbox (like the
ADO-token and execution-context steps), reaching the GitHub API over the build
agent pool's normal network — no AWF `network.allowed` entry is required.

#### Scope and boundaries — what this token is *not*

- **ADO API permissions** (`permissions.read` / `permissions.write`) are
entirely separate: they describe Azure DevOps OAuth/service-connection tokens
used by the pipeline and Stage 3 executor. The GitHub App token has no effect
on them.
- The GitHub App token is for **Copilot engine authentication only**. It does
**not** authenticate the `gh` CLI, grant GitHub MCP permissions, or give the
agent sandbox GitHub write access.
- There is **no `permissions:` sub-field** to scope the token. Copilot access
rides on the App's own granted capability (configured App/org-side); narrowing
the installation token's permissions at mint time cannot grant Copilot access
and risks stripping the capability it needs.

#### Notes and limitations

- **Copilot engine only.** `github-app-token` is rejected at compile time for
any other `engine.id` (the minted token is wired into `GITHUB_TOKEN` only on
the Copilot path), so a misconfiguration fails fast rather than silently
no-opping.
- **Secrecy is your responsibility.** The compiler validates the private-key
*variable name* but cannot verify the ADO variable is actually marked secret,
so every compile of a `github-app-token` workflow emits an advisory:
`Warning: engine.github-app-token uses pipeline variable '<name>' … Ensure
'<name>' is stored as a SECRET …`. It is non-blocking — heed it by setting the
value with `ado-aw secrets set` (which stores it as a secret).
- **GHEC by default; GHES via `api-url`.** The mint/revoke steps target
`https://api.github.com` unless you set `api-url` to your GitHub Enterprise
Server `/api/v3` base URL. This is independent of `engine.api-target`, which
configures the Copilot API host, not the GitHub App API host.
- **`openssl` is not required** — the token is minted with Node's built-in
crypto. The build agent needs Node (installed automatically) and network
access to the GitHub API host.
- You may still need to pin `engine.version` until the relevant Copilot CLI auth
behavior is broadly available.

### Copilot BYOM / BYOK provider configuration

The Copilot engine can route requests to an external LLM provider — for example a
Expand Down
19 changes: 19 additions & 0 deletions prompts/create-ado-agentic-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,25 @@ engine:
timeout-minutes: 30
```

**Org-backed Copilot auth (GitHub App).** By default Copilot authenticates with
the `GITHUB_TOKEN` pipeline variable. For organization-backed authentication,
add a `github-app-token` block so the compiler mints a GitHub App installation
token at runtime (Copilot engine only):
```yaml
engine:
id: copilot
github-app-token:
app-id: 1234567 # literal App ID or client ID
owner: octo-org # installation owner (org or user)
# repositories: [octo-repo] # optional; scopes the token
```
Store the private key once with `ado-aw secrets set GITHUB_APP_PRIVATE_KEY
"$(cat key.pem)"`. Only add this when the workflow explicitly needs org-backed
Copilot auth — omit it otherwise. See
[`docs/engine.md`](../docs/engine.md#github-app-backed-copilot-engine-auth) for
the full field reference (`private-key` override, `api-url` for GHES,
`skip-token-revocation`).

### Step 3 — Schedule

Use the **fuzzy schedule syntax** (deterministic time scattering based on agent name hash prevents load spikes). Omit `on.schedule` (or omit `on:` entirely) for manual/trigger-only pipelines.
Expand Down
1 change: 1 addition & 0 deletions prompts/debug-ado-agentic-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,7 @@ network:
timeout-minutes: 120
```
- **API rate limiting**: The model provider is rate-limiting requests. Check Copilot CLI logs for 429 responses.
- **GitHub App auth failures** (when `engine.github-app-token` is set): if Copilot fails to authenticate, check the `Mint GitHub App token` step logs in the Agent (and Detection) job. Common causes: the `GITHUB_APP_PRIVATE_KEY` secret (or your `private-key` override) is unset or not marked secret; a wrong `app-id`/`owner` (the App must be installed on that owner); or GHES without `api-url` set to the `/api/v3` base URL. `github-app-token` is Copilot-engine only — the compiler rejects it on other engines. See [`docs/engine.md`](../docs/engine.md#github-app-backed-copilot-engine-auth).

### Agent Tool Errors

Expand Down
28 changes: 28 additions & 0 deletions prompts/update-ado-agentic-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,34 @@ permissions:

`permissions.write` is **optional** — the Stage 3 executor defaults to `$(System.AccessToken)`. Set it only when you need cross-org writes or named-identity attribution instead of `Project Collection Build Service`.

### Enabling GitHub App-backed Copilot Auth

Add a `github-app-token` block under `engine:` so Copilot authenticates with an
org-backed GitHub App installation token (Copilot engine only):

```yaml
engine:
id: copilot
github-app-token:
app-id: 1234567 # literal App ID or client ID
owner: octo-org # installation owner (org or user)
# repositories: [octo-repo] # optional; scopes the token
# api-url: https://ghe.example.com/api/v3 # optional; GHES
# private-key: MY_SECRET_VAR # optional; defaults to GITHUB_APP_PRIVATE_KEY
# skip-token-revocation: false # optional; token is revoked by default
```

Then store the private key as a secret:
```bash
ado-aw secrets set GITHUB_APP_PRIVATE_KEY "$(cat app-private-key.pem)"
```

Notes: this is distinct from `permissions:` (ADO API tokens) — it only sources
the Copilot engine's `GITHUB_TOKEN`. Compile prints a non-blocking advisory
reminding you to mark the private-key variable secret. See
[`docs/engine.md`](../docs/engine.md#github-app-backed-copilot-engine-auth).
Requires recompilation.

### Adding Pre/Post Steps

**Inline steps** run inside the `Agent` job:
Expand Down
1 change: 1 addition & 0 deletions scripts/ado-script/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@ exec-context-pr-checks.js
exec-context-repo.js
approval-summary.js
conclusion.js
github-app-token.js
schema
*.tsbuildinfo
Loading
Loading