Share bounded ingress conformance test harness

[Copilot]

Bounded-agent and bounded-query ingress conformance suites duplicated socket setup, broker stubs, request helpers, and transport-parity assertions. This refactor centralizes that security-relevant test harness while preserving broker-specific protocol expectations.

• Shared harness
  • Adds ingress-conformance-test-harness.ts for Unix/TCP server lifecycle, broker recording, canonical responses, request helpers, and parity assertions.
• Suite-specific coverage
  • Keeps agent/query version headers, request bodies, capability behavior, and domain-specific assertions in their respective suites.

const harness = createIngressConformanceHarness({
  brokerDir,
  requestHeaders: { 'x-awf-query-version': '2' },
  defaultBody: 'print(True)',
});

await harness.start();
await harness.expectCanonicalTransportParity();

• Fixes [Duplicate Code] Bounded agent and query ingress conformance tests share the same harness #6979

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Note

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.

[Copilot]

Review details

Suppressed comments (1)

src/test-helpers/ingress-conformance-test-harness.ts:43

• This test-only module does not match the build exclusion **/*.test-utils.ts in tsconfig.json:20, so it will be compiled and shipped with the production dist, including its Jest-dependent harness. Please rename it to ingress-conformance-test-harness.test-utils.ts and update both suite imports, consistent with the other helpers in src/test-helpers/.

export function createIngressConformanceHarness(options: IngressConformanceHarnessOptions) {

• Files reviewed: 3/3 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[lpcox]

@copilot resolve merge conflicts

[Copilot]

@copilot resolve merge conflicts

Resolved in 9faf606. The merge accepts main’s removal of the legacy bounded-agent/query tests and removes the now-unused harness.

[github-actions]

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

[github-actions]

📰 DEVELOPING STORY: Smoke Copilot reports failed to deliver outputs. Our correspondents are investigating the incident...

[github-actions]

❌ Smoke Claude failed to deliver outputs

[github-actions]

🛡️ Smoke Copilot Network Isolation reports failed to deliver outputs while checking network isolation. Investigate the egress model.

[github-actions]

✅ Build Test Suite completed successfully!

[github-actions]

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

[github-actions]

✅ Contribution Check completed successfully!

Contribution check complete: the provided review context did not include any changed-file patches, so I could not assess code style, tests, documentation updates, PR description quality beyond the metadata, or file organization. No review comment was added.

[github-actions]

❌ Smoke Copilot BYOK AOAI (Entra) reports failed to deliver outputs. AOAI BYOK (Entra) mode investigation needed...

[github-actions]

🔌 Smoke Services — All services reachable! ✅

[github-actions]

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

[github-actions]

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

[github-actions]

✅ Smoke Gemini completed. All facets verified. 💎

[github-actions]

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

[github-actions]

❌ Smoke Copilot BYOK AOAI (api-key) reports failed to deliver outputs. AOAI BYOK (api-key) mode investigation needed...

Added smoke test PASS comment and label

[github-actions]

📰 DEVELOPING STORY: Smoke Docker Sbx reports failed to deliver outputs. Our correspondents are investigating the incident...

[github-actions]

🚀 Security Guard has started processing this pull request

[github-actions]

Smoke Test: Copilot Engine — @lpcox

• ✅ MCP: list_pull_requests OK (PR chore: upgrade gh-aw to v0.86.0 pre-release and recompile workflows #7027, fix(api-proxy): stop alias fallback picking arbitrary models #6996)
• ✅ GitHub.com connectivity (HTTP 200)
• ✅ File write/read test

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot
_{Add label ready-for-aw to run again}

[github-actions]

Smoke Test: Copilot BYOK (Direct) Mode — PASS ✅

Running in direct BYOK mode via COPILOT_PROVIDER_API_KEY (agent → api-proxy sidecar → api.githubcopilot.com)

Results:

• ✅ MCP Connectivity (retrieved merged PRs)
• ✅ GitHub.com HTTP 200
• ✅ File Write/Read Test
• ✅ BYOK Inference Path (agent responding)

All 4 tests passed. @lpcox

🔑 BYOK report filed by Smoke Copilot BYOK
_{Add label ready-for-aw to run again}

[github-actions]

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com) reachable: allowed=200
✅ Non-allowed domain (example.com) blocked: 403 proxy denial

Overall status: PASS

@lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
_{Add label ready-for-aw to run again}

[github-actions]

Smoke Test: Claude Engine Validation

Check	Status
API	✅ PASS
gh CLI	✅ PASS
File	✅ PASS

Overall result: PASS

Generated by Smoke Claude for #6987 · haiku45 · 55 AIC · ⊞ 3.6K · ◷
_{Add label ready-for-aw to run again}

[github-actions]

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.13	✅ YES
Node.js	v24.18.0	v22.23.2	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Overall: FAILED — Node.js version differs between host and chroot environment. smoke-chroot label not applied.

Tested by Smoke Chroot
_{Add label ready-for-aw to run again}

[github-actions]

Services connectivity smoke test:

• Redis PING: ❌ (Temporary failure in name resolution)
• pg_isready: ❌ (no response / DNS failure)
• Postgres SELECT 1: ❌ (could not translate host name)

Overall: FAIL — host.docker.internal did not resolve from within the AWF sandbox.

🔌 Service connectivity validated by Smoke Services
_{Add label ready-for-aw to run again}

[github-actions]

Smoke Test: API Proxy OpenTelemetry Tracing — Results

• ✅ Scenario 1 (Module Loading): otel.js loaded successfully, exporting startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled, plus exporter/parsing internals.
• ✅ Scenario 2 (Test Suite): 3 suites / 68 tests passed (otel.test.js, otel-fanout.test.js, otel-workload-identity.test.js).
• ✅ Scenario 3 (Env Var Forwarding): env-passthrough.ts forwards GITHUB_AW_OTEL_TRACE_ID/GITHUB_AW_OTEL_PARENT_SPAN_ID into the agent container; api-proxy-env-config.ts builds buildOtelEnv() forwarding OTLP endpoint/headers/service name and trace context into the api-proxy container.
• ✅ Scenario 4 (Token Tracker Integration): onUsage callback present in token-tracker-http.js (invoked after normalized usage extraction) as the OTEL hook point.
• ✅ Scenario 5 (OTEL Diagnostics): Workflow-level trace spans found at /tmp/gh-aw/otel.jsonl (gh-aw job/setup spans exported with proper resource attributes); no api-proxy request spans present this run (expected — no LLM API calls triggered).

Overall: All scenarios pass or are expected-pending. No issues found.

📡 OTel tracing validated by Smoke OTel Tracing
_{Add label ready-for-aw to run again}

[github-actions]

@lpcox

• GitHub MCP Testing: ✅
• GitHub.com Connectivity: ✅
• File Write/Read Test: ✅
• BYOK Inference Test: ✅
  Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
  Overall status: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
_{Add label ready-for-aw to run again}

[github-actions]

Recent PRs: chore: upgrade gh-aw to v0.86.0 pre-release and recompile workflows; fix(api-proxy): stop alias fallback picking arbitrary models

MCP connectivity: ✅
GitHub.com connectivity: ✅
File I/O: ✅
BYOK inference: ✅

Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra

Overall: PASS

cc @lpcox

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
_{Add label ready-for-aw to run again}

[github-actions]

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A (ran)	✅ PASS
.NET	json-parse	✅	N/A (ran)	✅ PASS
Go	color	✅	pass	✅ PASS
Go	env	✅	pass	✅ PASS
Go	uuid	✅	pass	✅ PASS
Java	gson	❌	0/0	❌ FAIL
Java	caffeine	❌	0/0	❌ FAIL
Node.js	clsx	✅	pass	✅ PASS
Node.js	execa	✅	pass	✅ PASS
Node.js	p-limit	✅	pass	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 7/8 ecosystems passed — FAIL

❌ Java failure details

Both gson and caffeine failed at mvn compile:

[ERROR] Could not create local repository at /home/runner/.m2/repository -> [Help 1]

Root cause: /home/runner/.m2 is owned by root:root (created earlier in the workflow, likely by a privileged step), so the runner user cannot create /home/runner/.m2/repository. This is an environment/permission issue unrelated to firewall domain allowlisting — Squid proxy config (squid-proxy:3128) was applied correctly via ~/.m2/settings.xml and JAVA_TOOL_OPTIONS, but Maven never got far enough to make network calls.

Generated by Build Test Suite for #6987 · auto · 33.8 AIC · ⊞ 11.5K · ◷
_{Add label ready-for-aw to run again}

[github-actions]

Smoke Test: Gemini Engine Validation

• GitHub MCP Testing: ❌ (Access Denied/Redacted)
• GitHub.com Connectivity: ❌ (Status 000)
• File Writing Testing: ✅
• Bash Tool Testing: ✅

Overall Status: FAIL

Note: GitHub MCP returned redacted results (#7064, #7036) due to secrecy policy. Network connectivity to github.com was blocked.

💎 Faceted by Smoke Gemini
_{Add label ready-for-aw to run again}

[github-actions]

Smoke Test

• Overall: FAIL
• Merged PRs: chore: upgrade gh-aw to v0.86.0 pre-release and recompile workflows; fix(api-proxy): stop alias fallback picking arbitrary models ✅
• PR list via safeinputs-gh: ❌ tool unavailable; fallback gh returned Add A19/B19 runner doctor entries: ARC/DinD safeoutputs mount + rootless cleanup chmod noise; [WIP] Fix cli-proxy connectivity issue in network-isolation mode
• Playwright title check: ✅
• File write/read: ✅
• Discussion query/comment: ❌ query succeeded, comment not permitted for discussions
• npm ci && npm run build: ✅

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
_{Add label ready-for-aw to run again}

[github-actions]

Docker Sbx Smoke Test

• ✅ GitHub MCP connectivity (list_pull_requests responded)
• ✅ github.com connectivity (HTTP 200)
• ✅ File write/read test

Pre-fetched PRs: #7027 "chore: upgrade gh-aw to v0.86.0 pre-release..." (merged), #6996 "fix(api-proxy): stop alias fallback..." (merged)

Overall: PASS

cc @lpcox

📰 BREAKING: Report filed by Smoke Docker Sbx
_{Add label ready-for-aw to run again}