Skip to content

refactor: pre-validate host port specs once in TypeScript, eliminate duplicate shell parser#6067

Merged
lpcox merged 3 commits into
mainfrom
copilot/duplicate-code-host-access-port-parsing
Jul 10, 2026
Merged

refactor: pre-validate host port specs once in TypeScript, eliminate duplicate shell parser#6067
lpcox merged 3 commits into
mainfrom
copilot/duplicate-code-host-access-port-parsing

Conversation

Copilot AI commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Host-access port-spec parsing was duplicated between parseValidPortSpecs() in TypeScript and parse_port_specs() in the agent init-container shell script, with three separate shell call sites each re-validating the same raw user input. This created drift risk in a security-critical firewall path.

Changes

  • src/services/agent-environment/api-proxy-environment.ts — calls parseValidPortSpecs() at environment-build time and stores the normalized results in two new env vars: AWF_VALID_ALLOW_HOST_PORTS and AWF_VALID_HOST_SERVICE_PORTS

  • src/services/agent-service.ts — forwards the new AWF_VALID_* vars into the iptables init container's environment block

  • containers/agent/setup-iptables.sh — replaces parse_port_specs() (full duplicate validator with whitespace trimming) with split_valid_port_specs(), a thin helper that splits the pre-validated comma-separated string and applies is_valid_port_spec() only as a fail-closed assertion (defence-in-depth). All three former parse_port_specs call sites now consume the AWF_VALID_* vars.

  • tests/setup-iptables-port-spec.test.sh — updated to test split_valid_port_specs instead of parse_port_specs; fixture-driven is_valid_port_spec coverage retained unchanged

  • TypeScript tests — added assertions that AWF_VALID_ALLOW_HOST_PORTS / AWF_VALID_HOST_SERVICE_PORTS are set on both the agent and iptables-init container environments

Before / After

// Before: raw string passed to container; shell re-parsed it three times
environment.AWF_ALLOW_HOST_PORTS = config.allowHostPorts;

// After: validated once, normalized string passed alongside the raw value
const validSpecs = parseValidPortSpecs(config.allowHostPorts, 'port spec');
if (validSpecs.length > 0) {
  environment.AWF_VALID_ALLOW_HOST_PORTS = validSpecs.join(',');
}
# Before: full duplicate parser at each call site
parse_port_specs gw_ports "$AWF_ALLOW_HOST_PORTS" "port spec"

# After: fail-closed split of the already-validated list
split_valid_port_specs gw_ports "$AWF_VALID_ALLOW_HOST_PORTS" "port spec"

Parse and validate --allow-host-ports / --allow-host-service-ports once
in TypeScript (parseValidPortSpecs in api-proxy-environment.ts), store
the result in new env vars AWF_VALID_ALLOW_HOST_PORTS and
AWF_VALID_HOST_SERVICE_PORTS, and pass them through to the iptables init
container.

Replace the three parse_port_specs() call sites in setup-iptables.sh
with a new split_valid_port_specs() helper that consumes the pre-validated
specs and applies is_valid_port_spec() only as a fail-closed assertion.
Remove the former parse_port_specs() function entirely.

Update setup-iptables-port-spec.test.sh to test split_valid_port_specs()
and add TypeScript tests verifying AWF_VALID_* vars are set on both the
agent and iptables-init container environments.

Closes #6057
Copilot AI changed the title [WIP] Refactor host-access port parsing to eliminate duplication refactor: pre-validate host port specs once in TypeScript, eliminate duplicate shell parser Jul 9, 2026
Copilot AI requested a review from lpcox July 9, 2026 19:05
Copilot finished work on behalf of lpcox July 9, 2026 19:05
@lpcox lpcox marked this pull request as ready for review July 9, 2026 19:05
Copilot AI review requested due to automatic review settings July 9, 2026 19:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors host-access port specification handling to reduce validator drift in a security-sensitive path by validating/normalizing port specs once in TypeScript and passing the results to the init-container via new AWF_VALID_* environment variables. The init-container script is updated to consume the pre-validated list via a thin splitter + defense-in-depth assertion, and tests are adjusted accordingly.

Changes:

  • Pre-validate and normalize allowHostPorts / allowHostServicePorts in TypeScript and export as AWF_VALID_ALLOW_HOST_PORTS / AWF_VALID_HOST_SERVICE_PORTS.
  • Update setup-iptables.sh to replace the full shell parser with split_valid_port_specs() and consume AWF_VALID_* at call sites.
  • Update/add tests to reflect the new env vars and split behavior.
Show a summary per file
File Description
tests/setup-iptables-port-spec.test.sh Updates shell unit tests to exercise split_valid_port_specs() instead of the removed full parser.
src/services/agent-service.ts Forwards the new AWF_VALID_* vars into the iptables-init container environment.
src/services/agent-service-build.test.ts Adds assertions for AWF_VALID_ALLOW_HOST_PORTS on the agent + forwarding into iptables-init.
src/services/agent-environment/api-proxy-environment.ts Performs TypeScript pre-validation and sets AWF_VALID_* env vars.
src/services/agent-environment-options.test.ts Adds assertion for AWF_VALID_HOST_SERVICE_PORTS on the agent environment.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 6/6 changed files
  • Comments generated: 5
  • Review effort level: Low

Comment thread containers/agent/setup-iptables.sh Outdated
Comment on lines +64 to +68
local -a _svps_entries=()
IFS=',' read -ra _svps_entries <<< "$_svps_input"
local _svps_spec=""
for _svps_spec in "${_svps_entries[@]}"; do
if [ -z "$_svps_spec" ]; then
Comment thread containers/agent/setup-iptables.sh Outdated
Comment on lines 296 to 300
# FILTER: also allow user-specified ports from --allow-host-ports (consume pre-validated list)
if [ -n "$AWF_VALID_ALLOW_HOST_PORTS" ]; then
local -a gw_ports=()
parse_port_specs gw_ports "$AWF_ALLOW_HOST_PORTS" "port spec"
split_valid_port_specs gw_ports "$AWF_VALID_ALLOW_HOST_PORTS" "port spec"
local port_spec=""
Comment thread containers/agent/setup-iptables.sh Outdated
Comment on lines 348 to 352
if [ -n "$AWF_VALID_HOST_SERVICE_PORTS" ] && [ -n "$AWF_ENABLE_HOST_ACCESS" ]; then
# Consume pre-validated port list (validated once by TypeScript parseValidPortSpecs()).
split_valid_port_specs HSP_PORTS "$AWF_VALID_HOST_SERVICE_PORTS" "host service port"

# Resolve host gateway IP (with AWF_HOST_GATEWAY_IP fallback, same as host access block)
Comment thread containers/agent/setup-iptables.sh Outdated
Comment on lines 398 to 402
if [ -n "$AWF_VALID_ALLOW_HOST_PORTS" ]; then
echo "[iptables] Redirect user-specified ports to Squid..."
local -a dnat_ports=()
parse_port_specs dnat_ports "$AWF_ALLOW_HOST_PORTS" "port spec"
split_valid_port_specs dnat_ports "$AWF_VALID_ALLOW_HOST_PORTS" "port spec"
local port_spec=""
Comment on lines 330 to 332
});

describe('toolchain var fallback to GITHUB_ENV', () => {
@lpcox

lpcox commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

@copilot address review feedback

Copilot AI commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address review feedback

Addressed the review feedback in 4274ecd.

Included:

  • whitespace trimming in split_valid_port_specs()
  • backward-compatible fallbacks to AWF_ALLOW_HOST_PORTS / AWF_HOST_SERVICE_PORTS when AWF_VALID_* is empty
  • updated shell test for whitespace-trim behavior
  • added test asserting AWF_VALID_HOST_SERVICE_PORTS is forwarded to iptables-init

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 99.15% 99.19% 📈 +0.04%
Statements 99.11% 99.15% 📈 +0.04%
Functions 99.45% 99.45% ➡️ +0.00%
Branches 95.77% 95.72% 📉 -0.05%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Gemini completed. All facets verified. 💎

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Security Guard failed. Please review the logs for details.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Contribution Check completed successfully!

PR #6067 follows the applicable CONTRIBUTING.md guidelines: it includes focused tests for the refactor, changes are organized in the appropriate source/container/test locations, documentation is not needed for this internal refactor, and the description clearly explains the change. No related issue is identified, so no issue reference is required.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Claude passed

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Build Test Suite completed successfully!

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Result
API status ✅ PASS
gh check ✅ PASS
File status ✅ PASS

Overall result: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Smoke Claude for #6067 · 55.1 AIC · ⊞ 3.3K ·
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: BYOK Direct Mode

  • ✅ Direct BYOK mode active (COPILOT_PROVIDER_API_KEY)
  • ✅ API proxy sidecar → api.githubcopilot.com routing
  • ✅ Placeholder credential isolation
  • ✅ MCP connectivity

Status: PASS — Running in direct BYOK mode via api-proxy sidecar

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot PAT Auth Validation ✅ PASS

Test Result
GitHub.com connectivity (HTTP 200)
File write/read
MCP connectivity

Overall: PASS | Auth mode: PAT (COPILOT_GITHUB_TOKEN) | CC: @lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🔑 PAT report filed by Smoke Copilot PAT
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

🔍 OTEL Tracing Smoke Test Results

Scenario Status Notes
1. Module Loading otel.js loads successfully; exports: startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled + internals
2. Test Suite 39/39 passed (otel.test.js) + 20/20 passed (otel-fanout.test.js) — 59 total, 0 failures
3. Env Var Forwarding src/services/api-proxy-env-config.ts forwards all 6 OTEL vars: GH_AW_OTLP_ENDPOINTS, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, GITHUB_AW_OTEL_TRACE_ID, GITHUB_AW_OTEL_PARENT_SPAN_ID, OTEL_SERVICE_NAME
4. Token Tracker Integration onUsage callback exists in token-tracker-http.js (lines 285, 343) as OTEL hook point
5. OTEL Diagnostics ✅ (expected) No spans exported — no OTEL endpoint configured in this run; graceful degradation confirmed (no errors)

All scenarios pass. 🎉

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

@lpcox

GitHub MCP Testing: ✅
GitHub.com Connectivity: ❌
File Write/Read Test: ❌
BYOK Inference Test: ✅

Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra

Overall: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Gemini Engine Validation

Test Results:

Overall Status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke test results:

  • ✅ chore: upgrade gh-aw to v0.82.7 pre-release and recompile workflows
  • ✅ [Test Coverage] Add branch coverage for diagnostic-collector, docker-socket, hosts-file
  • ✅ GitHub title check
  • ✅ File write/read
  • ✅ npm ci && npm run build
    Overall: PASS

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • awmgmcpg
  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

🤖 Copilot Smoke Test Results

Test Result
GitHub MCP connectivity
GitHub.com HTTP ✅ 200
File write/read

Overall: PASS

@lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Test Results

  • Redis PING: ❌ Network is unreachable
  • PostgreSQL pg_isready: ❌ No response
  • PostgreSQL SELECT 1: ❌ Network is unreachable

Overall: FAILhost.docker.internal (172.17.0.1) is unreachable from this environment. Services are not accessible.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Chroot Version Comparison Results

Runtime Host Version Chroot Version Match?
Python Python 3.12.13 Python 3.12.3 ❌ NO
Node.js v24.18.0 v22.23.1 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Result: Not all tests passed — Python and Node.js versions differ between host and chroot environments.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Smoke Test Summary

  • GitHub MCP Testing: ✅
  • GitHub.com Connectivity: ✅
  • File Write/Read Test: ✅
  • BYOK Inference Test: ✅

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)

Overall Status: PASS

cc @lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx all passed ✅ PASS
Node.js execa all passed ✅ PASS
Node.js p-limit all passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Build Test Suite for #6067 · 98 AIC · ⊞ 6.9K ·
Add label ready-for-aw to run again

@lpcox lpcox merged commit a89e47d into main Jul 10, 2026
86 of 88 checks passed
@lpcox lpcox deleted the copilot/duplicate-code-host-access-port-parsing branch July 10, 2026 02:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Duplicate Code] Host-access port parsing still duplicated in TypeScript and shell firewall rules

3 participants