fix: detect and expose system CA bundle for RHEL/Amazon Linux in chroot#5783
Conversation
The compose-generator test 'filters out workDir and home dot-directory bind mounts on split-fs' hardcoded '/host/home' as the prefix for filtering home-target volumes. This only works on Linux where home directories are under /home/. On macOS (home under /Users/), the filter matched nothing, causing the workspace mount assertion to fail. Use getRealUserHome() to derive the actual home prefix, making the test portable across platforms. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
On Amazon Linux / RHEL-family runners, the system CA bundle lives under
/etc/pki/ which is not mounted into the AWF chroot. This causes TLS
failures ('No CA certificates were loaded from the system') for tools
like Copilot CLI running inside the sandbox.
Add copy_system_ca_bundle() to the agent entrypoint that:
- Detects the host CA bundle from known candidate paths
- If already accessible via existing /etc/ssl mounts, sets TLS env vars
- If under /etc/pki (not mounted), copies to /tmp/awf-lib/ and sets
SSL_CERT_FILE, NODE_EXTRA_CA_CERTS, REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE,
GIT_SSL_CAINFO
- When SSL Bump is active, appends system bundle to AWF CA cert
Fixes #5733
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
There was a problem hiding this comment.
Pull request overview
This PR addresses TLS failures in AWF chroot mode on RHEL/Amazon Linux self-hosted runners by ensuring a host system CA bundle is discoverable inside the chroot (without mounting /etc/pki), aligning with the “copy a resolved bundle” mitigation described in #5733.
Changes:
- Add
copy_system_ca_bundle()to the agent entrypoint to detect common CA bundle locations and, when needed, copy the bundle into/tmp/awf-lib/and set common TLS env vars. - Update chroot-mode documentation to describe the new system CA detection/copy behavior.
- Update tests to include the new chroot helper phase and improve split-fs home mount filtering assertions.
Show a summary per file
| File | Description |
|---|---|
containers/agent/entrypoint.sh |
Adds system CA bundle detection/copy and integrates it into the chroot init sequence. |
docs/chroot-mode.md |
Documents system CA bundle detection and the env vars set for chrooted tools. |
tests/entrypoint-phase-functions.test.sh |
Ensures the new entrypoint helper is invoked in the expected chroot phase order. |
src/compose-generator.test.ts |
Adjusts split-fs home mount filtering expectations to use the effective real home path. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 2
- Review effort level: Low
| if [ -n "$AWF_CA_CHROOT" ]; then | ||
| # SSL Bump already configured TLS env vars; detect system bundle and append | ||
| # it to the AWF CA so tools trust both the AWF proxy CA AND the upstream CAs. | ||
| local SYSTEM_BUNDLE="" | ||
| for candidate in \ | ||
| /host/etc/ssl/certs/ca-certificates.crt \ | ||
| /host/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem \ | ||
| /host/etc/pki/tls/certs/ca-bundle.crt \ | ||
| /host/etc/pki/tls/cert.pem \ | ||
| /host/etc/ssl/cert.pem; do | ||
| if [ -s "$candidate" ]; then | ||
| SYSTEM_BUNDLE="$candidate" | ||
| break | ||
| fi | ||
| done | ||
| if [ -n "$SYSTEM_BUNDLE" ]; then | ||
| # Append system bundle to the AWF CA cert so both are trusted | ||
| if cat "$SYSTEM_BUNDLE" >> /host/tmp/awf-lib/awf-ca.crt 2>/dev/null; then | ||
| echo "[entrypoint] System CA bundle ($SYSTEM_BUNDLE) appended to AWF CA cert" | ||
| fi | ||
| fi | ||
| return | ||
| fi |
| if mkdir -p /host/tmp/awf-lib 2>/dev/null; then | ||
| if cp "$SYSTEM_BUNDLE" /host/tmp/awf-lib/system-ca-certificates.crt 2>/dev/null && \ | ||
| [ -s /host/tmp/awf-lib/system-ca-certificates.crt ]; then | ||
| local CA_PATH="/tmp/awf-lib/system-ca-certificates.crt" | ||
| export SSL_CERT_FILE="$CA_PATH" | ||
| export NODE_EXTRA_CA_CERTS="$CA_PATH" | ||
| export REQUESTS_CA_BUNDLE="$CA_PATH" | ||
| export CURL_CA_BUNDLE="$CA_PATH" | ||
| export GIT_SSL_CAINFO="$CA_PATH" | ||
| echo "[entrypoint] System CA bundle copied from $SYSTEM_BUNDLE to $CA_PATH" |
|
@copilot address review feedback |
|
✅ Copilot review passed with no inline comments. @lpcox Add the |
|
📡 Smoke OTel Tracing reports failed. OTel tracing regression detected. |
|
📰 DEVELOPING STORY: Smoke Copilot reports failed. Our correspondents are investigating the incident... |
|
🔌 Smoke Services — Service connectivity failed |
|
❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed... |
|
❌ Security Guard failed. Please review the logs for details. |
|
❌ Smoke Claude failed |
|
❌ Smoke Gemini reports failed. Facets need polishing... |
|
🌑 The shadows whisper... Smoke Codex failed. The oracle requires further meditation... |
|
❌ Smoke Copilot BYOK reports failed. BYOK mode investigation needed... |
|
Chroot tests failed Smoke Chroot failed - See logs for details. |
|
🔑 Smoke Copilot PAT reports failed. PAT auth path may have issues... |
|
❌ Contribution Check failed. Please review the logs for details. |
|
Build Test Failed Build Test Suite - See logs for details |
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed... |
|
✅ Build Test Suite completed successfully! |
|
✅ Security Guard completed successfully! PR #5783 security review: System CA bundle detection and copying. No security-weakening changes detected. File operations are safe (source validation with |
|
✅ Contribution Check completed successfully! PR follows the applicable CONTRIBUTING.md guidelines: clear description with related issue, tests included for the entrypoint change, documentation updated, and files are organized appropriately. |
|
🌑 The shadows whisper... Smoke Codex failed. The oracle requires further meditation... |
|
🔌 Smoke Services — All services reachable! ✅ |
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓 |
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ |
|
✅ Smoke Gemini completed. All facets verified. 💎 |
Smoke Test: Claude Engine Validation
Overall result: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Copilot BYOK (Direct) Mode✅ BYOK Inference Path Active (agent → api-proxy → api.githubcopilot.com) Status: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
|
Smoke test results:
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) Overall: PASS cc @lpcox Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
|
@lpcox Smoke Test ✅
Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw). Overall: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Copilot PAT Auth — FAIL
Overall: FAIL — workflow template variables were not expanded (SMOKE_HTTP_CODE, SMOKE_FILE_PATH, etc. passed as literals). Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
🔍 Smoke Test Results
PR: fix: detect and expose system CA bundle for RHEL/Amazon Linux in chroot Overall: FAIL — pre-step template variables were not substituted; file test unverifiable. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
📡 Smoke Test: API Proxy OpenTelemetry Tracing
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Services Connectivity
Result: FAIL Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test Results
Overall Status: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS Notes
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Summary
Fixes #5733
On Amazon Linux / RHEL-family self-hosted runners, the system CA bundle lives under
/etc/pki/which is not mounted into the AWF chroot. This causes TLS failures for tools like Copilot CLI running inside the sandbox:Changes
Adds a
copy_system_ca_bundle()function to the agent container entrypoint that:Detects the host system CA bundle from known candidate paths:
/etc/ssl/certs/ca-certificates.crt(Debian/Ubuntu)/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem(RHEL/Amazon Linux)/etc/pki/tls/certs/ca-bundle.crt(RHEL/CentOS)/etc/pki/tls/cert.pem(Fedora)/etc/ssl/cert.pem(macOS/Alpine)If already accessible in the chroot (under
/etc/ssl/or/etc/ca-certificates/which are already mounted), sets TLS environment variables pointing to it.If not accessible (e.g.,
/etc/pki/paths), copies the bundle to/tmp/awf-lib/system-ca-certificates.crtand sets:SSL_CERT_FILENODE_EXTRA_CA_CERTSREQUESTS_CA_BUNDLECURL_CA_BUNDLEGIT_SSL_CAINFOWhen SSL Bump is active, appends the system CA bundle to the AWF CA cert so tools trust both the proxy CA and upstream CAs.
Design Decision
The issue proposed two approaches. This implements the preferred "copy-a-single-resolved-bundle" approach rather than mounting
/etc/pkiinto the chroot, because it exposes less of the host filesystem to the agent.Testing
bash -nsyntax validation passes