Skip to content

Rootless cleanup chmod warning obscures the primary AWF startup failure #6934

Description

@lpcox

Summary

When AWF exits after a primary startup failure, a best-effort host-side chmod in cleanup can emit a large warning and error object after the real failure. Consumers that summarize the last agent output then report the cleanup warning as the engine failure, obscuring the actionable root cause.

Observed with AWF 0.27.43 in:

What happened

The actual failure occurred while attaching configured topology peers:

[INFO] Attaching 2 trusted container(s) to the internal network...
[INFO] Network-isolation: connecting container "awmg-mcpg" to "awf-net"...
[ERROR] Fatal error: Error: Failed to connect container "awmg-mcpg" to network "awf-net": Error response from daemon: No such container: awmg-mcpg

The run configured:

"topologyAttach": ["awmg-mcpg", "awmg-cli-proxy"]

The workflow exported a Docker command that could launch awmg-mcpg, but only the CLI proxy was started. No MCP gateway container was launched before AWF attempted docker network connect.

During cleanup, AWF then attempted:

chmod -R a+rX /tmp/gh-aw/sandbox/firewall/logs

The runner did not own the Squid-created directory/files, so the command failed with Operation not permitted. preserveDirectory() logged the complete execa error at warning level after the primary failure:

[WARN] Could not fix squid log permissions: Error: Command failed with exit code 1: chmod -R a+rX ...
...
stderr: "chmod: changing permissions ... Operation not permitted"

As a result, github/gh-aw#50384 shows only the chmod error under Last agent output, making a secondary, expected rootless cleanup failure look like the cause of the engine failure.

The later Docker-based rootless permission repair already treats Operation not permitted/EACCES as benign debug output, but the preceding direct chmod in src/artifact-preservation.ts does not use the same classification.

Expected behavior

  • Preserve the primary AWF failure as the final actionable diagnostic.
  • Treat an expected rootless chmod failure during best-effort artifact preservation as concise debug output, consistent with fixArtifactPermissionsForRootless().
  • Do not dump the full execa object for a known-benign cleanup failure.
  • Continue surfacing unexpected cleanup failures as warnings.

Suggested fix

Apply the benign permission-error classification used by fixArtifactPermissionsForRootless() to the direct chmod path in preserveDirectory(), or avoid the direct host chmod when running rootless and rely on the existing repair helper.

Regression coverage should simulate:

  1. A primary container/topology startup failure.
  2. chmod -R a+rX failing with EPERM.
  3. Cleanup retaining the primary error as the last prominent diagnostic without a full secondary stack/object dump.

Audit note

This issue tracks AWF's misleading cleanup diagnostics. The missing awmg-mcpg topology peer that caused this specific run is a separate caller/configuration lifecycle problem.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions