Skip to content

[awf] network-isolation: cli-proxy still cannot reach an external DIFC proxy on v0.28.0 (supersedes #7063) #7335

Description

@raman-anyware

Filing fresh because #7063 was closed by a PR that contains no production change, and I cannot reopen it. Still reproducible on v0.28.0.

Problem

In network-isolation mode, when the CLI proxy targets an external DIFC proxy on the runner host, awf-cli-proxy never becomes healthy: its tcp-tunnel gets ENETUNREACH dialing the host gateway, the liveness probe exhausts its retries, and the run fails before the agent is invoked.

Affects any workflow using gh-aw's tools.github.mode: gh-proxy (or the legacy features: {cli-proxy: true}) on a runner where the DIFC proxy is external. The default local mode starts no cli-proxy, which is likely why this looks narrow.

Root cause, from the v0.28.0 source

src/services/cli-proxy-service.ts attaches the sidecar to the internal network only -- the string awf-ext does not appear anywhere in that file -- while setting a host alias:

networks: { 'awf-net': { ipv4_address: cliProxyIp } },
extra_hosts: { 'host.docker.internal': 'host-gateway' },

host-gateway resolves to the default bridge gateway (172.17.0.1), and awf-net is internal: true, so the container has no route to it. Squid and api-proxy are already dual-homed onto awf-ext for exactly this reason; cli-proxy was not.

Why #7063 was closed without a fix

#7066 -- "Fix cli-proxy ENETUNREACH in network-isolation mode by dual-homing it on the external bridge" -- changed one file, src/compose-generator.test.ts, +17/-0. No production file was touched, and the test it adds asserts the opposite of the title:

it('keeps cli-proxy on awf-net only when it targets an attached DIFC proxy', () => {
  ...
  expect(cliProxyNetworks['awf-ext']).toBeUndefined();
});

That assertion is correct for the case it names -- an attached DIFC proxy (difcProxyHost: 'awmg-cli-proxy:18443'), a sibling container needing no route off the internal network. The external case is the other branch, and it received no change.

Reproduction on v0.28.0

GitHub-hosted ubuntu-latest, no Tailscale, no custom host routing; gh-aw v0.79.8 with sandbox.agent.version: v0.28.0; CLI proxy in external-DIFC mode:

[INFO] CLI proxy enabled: connecting to external DIFC proxy at host.docker.internal:18443
[INFO] Network-isolation mode: enforcing egress via Docker network topology (no host iptables, no sudo).
 Network awf-net  Created
 Network awf-1786580104070_awf-ext  Created
 Container awf-cli-proxy  Error
dependency failed to start: container awf-cli-proxy is unhealthy
[tcp-tunnel] Upstream error (::1:37186): connect ENETUNREACH 172.17.0.1:18443 - Local (0.0.0.0:0)
[ERROR] Fatal error: AWF firewall failed to start: awf-cli-proxy could not connect to the external
        DIFC proxy ... The agent was never invoked.

Both networks are created; only the cli-proxy lacks a route to the host. The same workflow on v0.27.26 (pre-isolation-rework) runs successfully on the same runner pool.

The DNS fix from #7188 does work here -- v0.28.0 logs retaining reachable non-portable DNS server(s): 168.63.129.16 rather than discarding them. Only the cli-proxy network path is still broken. This also distinguishes the report from #6704, which is Tailscale-specific: there is no Tailscale on these runners, so route capture cannot be the cause.

Suggested fix

Attach cli-proxy to EXTERNAL_BRIDGE_NAME when networkIsolation is enabled and the DIFC proxy is external, mirroring Squid and api-proxy -- that is, what #7066's title describes. The existing test stays valid if scoped to the attached-DIFC case it names.

Workaround, for anyone who lands here

Switch the workflow to tools.github.mode: local, which starts no cli-proxy. Note that it also removes the pre-authenticated gh that gh-proxy provides, so any prompt driving gh in bash must move onto the GitHub MCP toolsets -- in local mode the agent holds no GitHub credentials.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions