build(docker): add build-tools sysroot image and runner.topology config for arc-dind#5696
Conversation
Add support for ARC/DinD deployments where the runner and Docker daemon have separate filesystems and no root access is available in the workflow. Key changes: - New containers/build-tools/Dockerfile: Ubuntu 22.04 image with build-essential, gcc, g++, make, cmake, dev libraries, and system utilities. Provides the chroot base that the agent needs on ARC/DinD. - New runner.topology config field (standard | arc-dind): Single declarative knob that activates sysroot staging, network isolation defaults, and tool cache validation for ARC/DinD. - New runner.sysrootImage config field: Override the default build-tools image for custom sysroot setups. - Compose generator emits sysroot-stage init service when topology is arc-dind: copies build-tools image FS into a named volume via cp -a, agent mounts it read-only at /host. - Tool cache warning when RUNNER_TOOL_CACHE is under /opt on ARC/DinD (invisible to the DinD daemon filesystem). - Release workflow: new build-build-tools job builds and publishes the build-tools image to GHCR with cosign signing and SBOM. - Updated docs/arc-dind.md with runner.topology usage, build-tools walkthrough, and tool cache redirection guidance. - 32 new tests covering sysroot service, config mapping, schema validation, and compose generation. Closes #5693 Refs: gh-aw#42368, #5541, #5591 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 98.37% | 98.42% | 📈 +0.05% |
| Statements | 98.30% | 98.34% | 📈 +0.04% |
| Functions | 99.54% | 99.54% | ➡️ +0.00% |
| Branches | 94.43% | 94.38% | 📉 -0.05% |
📁 Per-file Coverage Changes (2 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/compose-generator.ts |
98.7% → 98.9% (+0.20%) | 98.7% → 98.9% (+0.20%) |
src/workdir-setup.ts |
92.7% → 94.5% (+1.82%) | 92.7% → 94.5% (+1.82%) |
✨ New Files (1 files)
src/services/sysroot-service.ts: 100.0% lines
Coverage comparison generated by scripts/ci/compare-coverage.ts
|
The documentation build workflow (run 28409144039) actually completed successfully -- the |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (2 files)
✨ New Files (1 files)
Coverage comparison generated by |
There was a problem hiding this comment.
Pull request overview
This PR introduces first-class ARC/DinD support by adding a “sysroot staging” mechanism (a build-tools image copied into a named volume mounted at /host) and a new runner.topology configuration surface intended to activate ARC/DinD-specific behavior with minimal caller configuration.
Changes:
- Add
runner.topology(standard|arc-dind) plusrunner.sysrootImageto the config schema and config-file mapping. - Generate a
sysroot-stageinit service andsysroot:/host:rovolume mount when ARC/DinD topology is enabled. - Add a new
containers/build-toolsimage and publish it from the release workflow; update ARC/DinD documentation accordingly.
Show a summary per file
| File | Description |
|---|---|
| src/types/platform-options.ts | Adds RunnerTopology type and new platform-level options (runnerTopology, sysrootImage). |
| src/types/docker.ts | Extends Docker service typing with entrypoint. |
| src/services/sysroot-service.ts | Introduces sysroot-stage init service builder + sysroot enable/image resolution helpers. |
| src/services/sysroot-service.test.ts | Adds unit tests for sysroot enablement/image resolution/service generation. |
| src/config-mapper.ts | Maps runner.topology and runner.sysrootImage from config file into CLI options. |
| src/config-file.ts | Extends AwfFileConfig with a new runner section. |
| src/config-file-validation.test.ts | Adds schema validation tests for the new runner section. |
| src/config-file-mapping.test.ts | Adds mapping tests for runner.topology and runner.sysrootImage. |
| src/compose-generator.ts | Adds sysroot-stage service generation, tool-cache warning, and sysroot named volume mounting. |
| src/compose-generator.test.ts | Adds compose output tests for sysroot-stage service and sysroot volume behavior. |
| src/awf-config-schema.json | Updates generated schema to include the new runner section. |
| schemas/token-usage.schema.json | JSON formatting-only changes (enum/property formatting). |
| docs/awf-config.schema.json | Updates documented schema to include the new runner section. |
| docs/arc-dind.md | Documents runner.topology usage and sysroot/tool-cache behavior for ARC/DinD. |
| containers/build-tools/Dockerfile | Adds the build-tools sysroot image (Ubuntu 22.04 + build essentials/libs/utilities). |
| .github/workflows/release.yml | Adds a job to build/push/sign/attest the build-tools image and gates release on it. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 16/16 changed files
- Comments generated: 5
- Review effort level: Low
| if (sysrootActive) { | ||
| // The sysroot named volume provides /host content (system binaries, libs, etc.) | ||
| // via the sysroot-stage init container instead of per-directory bind mounts. | ||
| agentVolumes.push('sysroot:/host:ro'); | ||
| } |
| platformType: config.platform?.type, | ||
|
|
||
| runnerTopology: config.runner?.topology, | ||
| sysrootImage: config.runner?.sysrootImage, |
| When `runner.topology` is set to `"arc-dind"`, AWF applies overridable defaults: | ||
|
|
||
| | Behavior | Default | Override | | ||
| |----------|---------|----------| | ||
| | Network isolation (no NET_ADMIN) | `true` | `network.isolation` | | ||
| | DinD pre-stage dirs | `true` | `dind.preStageDirs` | | ||
| | Sysroot image for `/host` base | `build-tools:<tag>` | `runner.sysrootImage` | | ||
| | Tool cache warning if under `/opt` | Emitted | Set `RUNNER_TOOL_CACHE` to shared path | |
|
@copilot address review feedback |
Addressed the review feedback in commit b9b2bb3. This wires |
|
✅ Copilot review passed with no inline comments. @lpcox Add the |
|
❌ Security Guard failed. Please review the logs for details. |
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded. |
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ |
|
✅ Smoke Gemini completed. All facets verified. 💎 |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
🔌 Smoke Services — All services reachable! ✅ |
Overall: PASS cc @lpcox
|
|
build(docker): add build-tools sysroot image and runner.topology config for arc-dind Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "registry.npmjs.org"See Network Configuration for more information.
|
🔍 Smoke Test: API Proxy OpenTelemetry Tracing
All scenarios pass. OTEL tracing integration is fully functional.
|
Chroot Version Comparison Results
Overall: ❌ Not all versions match — Python and Node.js differ between host and chroot environments.
|
🔥 Smoke Test: Copilot PAT — PASS
PR: build(docker): add build-tools sysroot image and runner.topology config for arc-dind Overall: PASS ✅
|
Smoke Test: Copilot BYOK (Direct Mode) ✅ PASS
Mode: Direct BYOK via All tests passed. See #5696.
|
Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra Overall: PASS
|
Gemini Engine Smoke Test Results
Overall Status: FAIL Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
Smoke Test Results — Services Connectivity
Overall: FAIL
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
Reconcile duplicate runner topology definitions: - Remove runnerTopology/sysrootImage from PlatformOptions (now in RunnerOptions) - Add "standard" to RunnerOptions.runnerTopology enum for consistency - Remove duplicate runnerTopology property in build-config.ts - Fix test assertion for updated topology enum values Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add missing `runner` section to the Section 4 Data Model table and add Section 5 CLI Mapping entries for `runner.topology` and `runner.sysrootImage` in docs/awf-config-spec.md. These fields were introduced in PRs #5696 and #5697 and correctly propagated to src/awf-config-schema.json, docs/awf-config.schema.json, src/types/runner-options.ts, src/types/wrapper-config.ts, src/config-file.ts, and src/config-mapper.ts — but the spec doc was not updated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: propagate runner config fields to spec Add missing `runner` section to the Section 4 Data Model table and add Section 5 CLI Mapping entries for `runner.topology` and `runner.sysrootImage` in docs/awf-config-spec.md. These fields were introduced in PRs #5696 and #5697 and correctly propagated to src/awf-config-schema.json, docs/awf-config.schema.json, src/types/runner-options.ts, src/types/wrapper-config.ts, src/config-file.ts, and src/config-mapper.ts — but the spec doc was not updated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * docs: fix sysrootImage default registry description --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Landon Cox <landon.cox@microsoft.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Summary
Adds support for ARC/DinD deployments where the runner and Docker daemon have separate filesystems and no root access is available in the workflow.
Closes #5693
Refs: github/gh-aw#42368, #5541, #5591
Key changes
1. Build-tools sysroot image (
containers/build-tools/Dockerfile)Ubuntu 22.04 image with system-level build infrastructure:
These packages require root to install and cannot be added at workflow runtime on ARC. The image is used as an init container that copies its filesystem into a named volume mounted at
/hostby the agent.2.
runner.topologyconfig fieldNew config section in the AWF JSON schema:
{ "runner": { "topology": "arc-dind", "sysrootImage": "ghcr.io/github/gh-aw-firewall/build-tools:latest" } }When
topologyis"arc-dind", AWF:sysroot-stageinit service in the compose filesysrootnamed volume and mounts it on the agent at/host:roRUNNER_TOOL_CACHEis under/opt(invisible to DinD daemon)This is the single stable contract between gh-aw and AWF for ARC deployments (see github/gh-aw#42368).
3. Release workflow
New
build-build-toolsjob in.github/workflows/release.yml:4. Documentation
Updated
docs/arc-dind.mdwith:Architecture
build-tools = static system base (same for all workflows, requires root → baked at image build time).
Tool cache = dynamic, per-workflow (setup-* actions choose versions, no root needed → installed at workflow runtime).
Privilege hierarchy
apt-get install gcc make libssl-dev ...setup-*actions, checkout, config generationTests
32 new tests covering:
sysroot-service.test.ts: Unit tests for sysroot service builder, image resolution, enable detectioncompose-generator.test.ts: Integration tests for sysroot-stage in compose outputconfig-file-mapping.test.ts: Config mapper tests for runner.topology and sysrootImageconfig-file-validation.test.ts: Schema validation for runner sectionAll 3353 tests pass.