From 30d3866df908609d7b3043e0ed0f14553334fe50 Mon Sep 17 00:00:00 2001 From: "Voight-Kampff (bot)" <151556158+sunholo-voight-kampff@users.noreply.github.com> Date: Wed, 5 Aug 2026 20:00:13 +0200 Subject: [PATCH 1/2] =?UTF-8?q?test(ci-flake):=20M3=20=E2=80=94=20gatelint?= =?UTF-8?q?=20legibility=20lint=20+=20AC10=20egress=20posture=20probe=20(i?= =?UTF-8?q?ncl.=20D5=3DA=20residual)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes AC8 and AC10 of m-ci-flake-systemic-fix. Pure Go, zero workflow edits. internal/testutil/gatelint — a legibility lint for the three known test-gating offenders, running as a plain Go test on every CI leg and every `make test`: R1 `testing.Short(` R2 `Getenv("CI")`/`Getenv("GITHUB_ACTIONS")` R3 `httpbin.org`/`ailang-packages` outside a live-network gate or the allowlist The walker matches *_test.go only, skips dot-dirs and testdata/, and excludes its own package — all three are measured-necessary (6 production files carry R3's tokens; an unscoped R3 would flag gatelint's own source and red CI on the very commit that introduces it, which was a blocking quorum objection). internal/testutil/egress_posture_test.go — AC10 (a) poison sentinel denies HTTP egress, (b) loopback bypasses the lane poison, (c) raw TCP stays open, and (d) the D5=Option-A residual asserted as OPEN: a nil-Proxy transport built the way internal/effects builds them bypasses the poison, with a ProxyFromEnvironment transport failing in the same run as the control that makes (d) non-vacuous. (d) is the tripwire that reds when Option B lands. Allowlist seed re-measured at 78f30e053 AFTER M2 (the plan's seed predated it): R1 empty (M1/M2 removed all 7), R2 one entry using the reason M2 authored in provider_script_test.go, R3 five entries. net_test.go and gitcache_test.go are gated by M2 and deliberately NOT allowlisted. Verified outside the sandbox: gatelint 941 files/0 violations; AC10 (a)-(d) pass in the default, poisoned, live and poisoned+live lanes; full suite 108 ok/0 FAIL; gofmt/vet/golangci-lint clean. Six mutations each proven landed and reverted byte-identical: R1 branch, walker suffix, dot-dir skip (executor); R3 gating check -> flags the 2 gated files, allowlist emptied -> flags exactly the 5 seeds, AC10(d) poison env stripped -> control catches it (controller). AC8 falsification drill fired: 942 files scanned, the scratch file named, removed, back to green. Co-Authored-By: codex gpt-5.6-sol Co-Authored-By: Claude Opus 4.8 --- internal/testutil/egress_posture_test.go | 111 ++++++++++++++ internal/testutil/gatelint/allowlist.go | 30 ++++ internal/testutil/gatelint/gatelint_test.go | 55 +++++++ internal/testutil/gatelint/scan.go | 137 ++++++++++++++++++ .../testdata/fixtures/cmd/r2_test.go.fixture | 3 + .../internal/.hidden/hidden_test.go.fixture | 3 + .../fixtures/internal/r1_test.go.fixture | 5 + .../fixtures/runtime/production.go.fixture | 3 + .../fixtures/std/clean_test.go.fixture | 3 + .../fixtures/tests/r3_test.go.fixture | 3 + 10 files changed, 353 insertions(+) create mode 100644 internal/testutil/egress_posture_test.go create mode 100644 internal/testutil/gatelint/allowlist.go create mode 100644 internal/testutil/gatelint/gatelint_test.go create mode 100644 internal/testutil/gatelint/scan.go create mode 100644 internal/testutil/gatelint/testdata/fixtures/cmd/r2_test.go.fixture create mode 100644 internal/testutil/gatelint/testdata/fixtures/internal/.hidden/hidden_test.go.fixture create mode 100644 internal/testutil/gatelint/testdata/fixtures/internal/r1_test.go.fixture create mode 100644 internal/testutil/gatelint/testdata/fixtures/runtime/production.go.fixture create mode 100644 internal/testutil/gatelint/testdata/fixtures/std/clean_test.go.fixture create mode 100644 internal/testutil/gatelint/testdata/fixtures/tests/r3_test.go.fixture diff --git a/internal/testutil/egress_posture_test.go b/internal/testutil/egress_posture_test.go new file mode 100644 index 000000000..2bba7b2cd --- /dev/null +++ b/internal/testutil/egress_posture_test.go @@ -0,0 +1,111 @@ +package testutil + +import ( + "net" + "net/http" + "net/http/httptest" + "net/url" + "os" + "strings" + "testing" + "time" +) + +const egressPoisonProxy = "http://127.0.0.1:9" + +func TestEgressPosture(t *testing.T) { + t.Run("poison_sentinel_denies_HTTP_egress", testPoisonSentinel) + t.Run("loopback_bypasses_lane_poison", testLoopbackBypass) + t.Run("raw_TCP_remains_open", testRawTCPResidual) + t.Run("effects_nil_proxy_remains_open", testEffectsProxyResidual) +} + +func testPoisonSentinel(t *testing.T) { + proxyURL, err := url.Parse(egressPoisonProxy) + if err != nil { + t.Fatalf("parse poison proxy: %v", err) + } + transport := &http.Transport{Proxy: http.ProxyURL(proxyURL)} + t.Cleanup(transport.CloseIdleConnections) + client := &http.Client{Transport: transport, Timeout: 5 * time.Second} + + _, err = client.Get("https://example.com") + assertPoisonProxyError(t, err) +} + +func testLoopbackBypass(t *testing.T) { + if !laneIsPoisoned() { + t.Skip("TestEgressPosture/loopback_bypasses_lane_poison requires HTTP_PROXY or HTTPS_PROXY=http://127.0.0.1:9") + } + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusNoContent) + })) + defer server.Close() + + transport := &http.Transport{Proxy: http.ProxyFromEnvironment} + t.Cleanup(transport.CloseIdleConnections) + response, err := (&http.Client{Transport: transport, Timeout: 5 * time.Second}).Get(server.URL) + if err != nil { + t.Fatalf("loopback GET under poisoned proxy: %v", err) + } + defer response.Body.Close() + if response.StatusCode != http.StatusNoContent { + t.Fatalf("loopback status = %d, want %d", response.StatusCode, http.StatusNoContent) + } +} + +func testRawTCPResidual(t *testing.T) { + requireLiveEgressPosture(t, "raw_TCP_remains_open") + connection, err := net.DialTimeout("tcp", "example.com:443", 10*time.Second) + if err != nil { + t.Fatalf("raw TCP residual unexpectedly closed: %v", err) + } + connection.Close() +} + +func testEffectsProxyResidual(t *testing.T) { + requireLiveEgressPosture(t, "effects_nil_proxy_remains_open") + t.Setenv("HTTP_PROXY", egressPoisonProxy) + t.Setenv("HTTPS_PROXY", egressPoisonProxy) + + // This first request intentionally trips red when Option B adds + // ProxyFromEnvironment to internal/effects. That red means the residual has + // closed; retire this tripwire and its Non-Goals text instead of "fixing" it. + effectsTransport := &http.Transport{} + t.Cleanup(effectsTransport.CloseIdleConnections) + response, err := (&http.Client{Transport: effectsTransport, Timeout: 10 * time.Second}).Get("https://example.com") + if err != nil { + t.Fatalf("nil-Proxy transport should bypass poison while D5 Option A remains: %v", err) + } + response.Body.Close() + + controlTransport := &http.Transport{Proxy: http.ProxyFromEnvironment} + t.Cleanup(controlTransport.CloseIdleConnections) + _, err = (&http.Client{Transport: controlTransport, Timeout: 5 * time.Second}).Get("https://example.com") + assertPoisonProxyError(t, err) +} + +func requireLiveEgressPosture(t *testing.T, leg string) { + t.Helper() + if os.Getenv("AILANG_LIVE_NET") != "1" { + t.Skipf("TestEgressPosture/%s requires AILANG_LIVE_NET=1", leg) + } +} + +func laneIsPoisoned() bool { + return os.Getenv("HTTP_PROXY") == egressPoisonProxy || os.Getenv("HTTPS_PROXY") == egressPoisonProxy +} + +func assertPoisonProxyError(t *testing.T, err error) { + t.Helper() + if err == nil { + t.Fatal("request through poison proxy unexpectedly succeeded") + } + message := err.Error() + for _, required := range []string{"proxyconnect", "127.0.0.1:9", "connection refused"} { + if !strings.Contains(strings.ToLower(message), required) { + t.Fatalf("poison error %q does not contain %q", message, required) + } + } + t.Logf("observed poison sentinel error: %v", err) +} diff --git a/internal/testutil/gatelint/allowlist.go b/internal/testutil/gatelint/allowlist.go new file mode 100644 index 000000000..d969538e1 --- /dev/null +++ b/internal/testutil/gatelint/allowlist.go @@ -0,0 +1,30 @@ +package gatelint + +type allowReason struct { + text string +} + +func mustReason(text string) allowReason { + if text == "" { + panic("gatelint allowlist reasons must not be empty") + } + return allowReason{text: text} +} + +var ruleAllowlist = map[Rule]map[string]allowReason{ + RuleR2: { + "internal/coordinator/provider_script_test.go": mustReason("this is a Unix shell/grandchild signal semantics test, not a live-network test, so the network opt-in helper would misstate its requirement. CI runners skip the known flaky shell behavior while local Unix runs retain coverage."), + }, + RuleR3: { + "internal/coordinator/agent_registry_test.go": mustReason("ailang-packages appears only in workspace-path fixtures; this test performs no live network call"), + "internal/parser/cli_integration_test.go": mustReason("httpbin.org appears only in expected CLI diagnostic fixture text; this test performs no live network call"), + "internal/parser/suggestion_errors_test.go": mustReason("httpbin.org appears only in parser suggestion/error fixtures; this test performs no live network call"), + "internal/messaging/config_test.go": mustReason("ailang-packages appears only in registry-mapping configuration fixtures; this test performs no live network call"), + "internal/pkg/manifest_test.go": mustReason("ailang-packages appears only in manifest parsing fixture URLs; this test performs no live network call"), + }, +} + +func isAllowlisted(rule Rule, path string) bool { + _, ok := ruleAllowlist[rule][path] + return ok +} diff --git a/internal/testutil/gatelint/gatelint_test.go b/internal/testutil/gatelint/gatelint_test.go new file mode 100644 index 000000000..08584b991 --- /dev/null +++ b/internal/testutil/gatelint/gatelint_test.go @@ -0,0 +1,55 @@ +package gatelint + +import ( + "fmt" + "path/filepath" + "sort" + "strings" + "testing" +) + +func TestGateLint_SelfTest(t *testing.T) { + root := filepath.Join("testdata", "fixtures") + violations, scanned := scan(root) + if scanned != 4 { + t.Fatalf("scanned %d candidate test files, want 4", scanned) + } + + got := violationSet(violations) + want := []string{ + "R1:internal/r1_test.go.fixture", + "R2:cmd/r2_test.go.fixture", + "R3:tests/r3_test.go.fixture", + } + if strings.Join(got, "\n") != strings.Join(want, "\n") { + t.Fatalf("violations mismatch\ngot:\n%s\nwant:\n%s", strings.Join(got, "\n"), strings.Join(want, "\n")) + } +} + +func TestGateLint_Repo(t *testing.T) { + root, err := filepath.Abs(filepath.Join("..", "..", "..")) + if err != nil { + t.Fatalf("resolve repository root: %v", err) + } + violations, scanned := scan(root) + if scanned == 0 { + t.Fatal("gatelint scanned zero *_test.go files; repository-root or walker scope is broken") + } + if len(violations) != 0 { + lines := make([]string, len(violations)) + for i, violation := range violations { + lines[i] = fmt.Sprintf("%s:%d: %s: %s", violation.Path, violation.Line, violation.Rule, violation.Message) + } + t.Fatalf("gatelint found %d violation(s) after scanning %d files:\n%s", len(violations), scanned, strings.Join(lines, "\n")) + } + t.Logf("scanned %d first-party test files", scanned) +} + +func violationSet(violations []Violation) []string { + set := make([]string, len(violations)) + for i, violation := range violations { + set[i] = string(violation.Rule) + ":" + violation.Path + } + sort.Strings(set) + return set +} diff --git a/internal/testutil/gatelint/scan.go b/internal/testutil/gatelint/scan.go new file mode 100644 index 000000000..e7628c862 --- /dev/null +++ b/internal/testutil/gatelint/scan.go @@ -0,0 +1,137 @@ +// Package gatelint enforces the repository's explicit test-gating conventions. +package gatelint + +import ( + "bufio" + "errors" + "io/fs" + "os" + "path/filepath" + "sort" + "strings" +) + +// Rule identifies a gatelint rule. +type Rule string + +const ( + RuleR1 Rule = "R1" + RuleR2 Rule = "R2" + RuleR3 Rule = "R3" +) + +// Violation describes one rule violation in a first-party test file. +type Violation struct { + Rule Rule + Path string + Line int + Message string +} + +var scanRoots = []string{"internal", "cmd", "runtime", "std", "tests"} + +// Scan checks first-party test files below root and returns violations in stable order. +func Scan(root string) []Violation { + violations, _ := scan(root) + return violations +} + +func scan(root string) ([]Violation, int) { + var violations []Violation + scanned := 0 + for _, scanRoot := range scanRoots { + base := filepath.Join(root, scanRoot) + err := filepath.WalkDir(base, func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + if entry.IsDir() { + if path != base && (strings.HasPrefix(entry.Name(), ".") || entry.Name() == "testdata") { + return filepath.SkipDir + } + return nil + } + + logicalName, fixture := logicalFileName(entry.Name()) + if !strings.HasSuffix(logicalName, "_test.go") { + return nil + } + rel, err := filepath.Rel(root, path) + if err != nil { + return err + } + rel = filepath.ToSlash(rel) + if strings.HasPrefix(rel, "internal/testutil/gatelint/") && !fixture { + return nil + } + + contents, err := os.ReadFile(path) + if err != nil { + return err + } + scanned++ + violations = append(violations, inspect(rel, logicalName, string(contents))...) + return nil + }) + if err != nil && !errors.Is(err, fs.ErrNotExist) { + violations = append(violations, Violation{Rule: "WALK", Path: filepath.ToSlash(scanRoot), Message: err.Error()}) + } + } + sort.Slice(violations, func(i, j int) bool { + if violations[i].Path != violations[j].Path { + return violations[i].Path < violations[j].Path + } + if violations[i].Line != violations[j].Line { + return violations[i].Line < violations[j].Line + } + return violations[i].Rule < violations[j].Rule + }) + return violations, scanned +} + +func logicalFileName(name string) (string, bool) { + const fixtureSuffix = ".fixture" + if strings.HasSuffix(name, fixtureSuffix) { + return strings.TrimSuffix(name, fixtureSuffix), true + } + return name, false +} + +func inspect(path, name, contents string) []Violation { + var violations []Violation + if line := tokenLine(contents, "testing.Short("); line != 0 && !isAllowlisted(RuleR1, path) { + violations = append(violations, Violation{Rule: RuleR1, Path: path, Line: line, Message: "testing.Short is inert in CI; delete the gate or use an explicit testutil opt-in helper"}) + } + if line := firstTokenLine(contents, `Getenv("CI")`, `Getenv("GITHUB_ACTIONS")`); line != 0 && !isAllowlisted(RuleR2, path) { + violations = append(violations, Violation{Rule: RuleR2, Path: path, Line: line, Message: "CI environment opt-out gate; use the explicit testutil gating convention"}) + } + if !strings.HasSuffix(name, "_live_test.go") && + !strings.Contains(contents, "testutil.RequiresLiveNetwork(") && + !isAllowlisted(RuleR3, path) { + if line := firstTokenLine(contents, "httpbin.org", "ailang-packages"); line != 0 { + violations = append(violations, Violation{Rule: RuleR3, Path: path, Line: line, Message: "known third-party token outside a live-network gate or documented allowlist"}) + } + } + return violations +} + +func firstTokenLine(contents string, tokens ...string) int { + first := 0 + for _, token := range tokens { + line := tokenLine(contents, token) + if line != 0 && (first == 0 || line < first) { + first = line + } + } + return first +} + +func tokenLine(contents, token string) int { + scanner := bufio.NewScanner(strings.NewReader(contents)) + for line := 1; scanner.Scan(); line++ { + if strings.Contains(scanner.Text(), token) { + return line + } + } + return 0 +} diff --git a/internal/testutil/gatelint/testdata/fixtures/cmd/r2_test.go.fixture b/internal/testutil/gatelint/testdata/fixtures/cmd/r2_test.go.fixture new file mode 100644 index 000000000..061528361 --- /dev/null +++ b/internal/testutil/gatelint/testdata/fixtures/cmd/r2_test.go.fixture @@ -0,0 +1,3 @@ +package fixture + +func testGate() string { return os.Getenv("CI") } diff --git a/internal/testutil/gatelint/testdata/fixtures/internal/.hidden/hidden_test.go.fixture b/internal/testutil/gatelint/testdata/fixtures/internal/.hidden/hidden_test.go.fixture new file mode 100644 index 000000000..431ccae64 --- /dev/null +++ b/internal/testutil/gatelint/testdata/fixtures/internal/.hidden/hidden_test.go.fixture @@ -0,0 +1,3 @@ +package fixture + +func hiddenGate() { if testing.Short() {} } diff --git a/internal/testutil/gatelint/testdata/fixtures/internal/r1_test.go.fixture b/internal/testutil/gatelint/testdata/fixtures/internal/r1_test.go.fixture new file mode 100644 index 000000000..45fb3ad1b --- /dev/null +++ b/internal/testutil/gatelint/testdata/fixtures/internal/r1_test.go.fixture @@ -0,0 +1,5 @@ +package fixture + +func testGate() { + if testing.Short() {} +} diff --git a/internal/testutil/gatelint/testdata/fixtures/runtime/production.go.fixture b/internal/testutil/gatelint/testdata/fixtures/runtime/production.go.fixture new file mode 100644 index 000000000..02d39540a --- /dev/null +++ b/internal/testutil/gatelint/testdata/fixtures/runtime/production.go.fixture @@ -0,0 +1,3 @@ +package fixture + +const inertRegistryFixture = "ailang-packages" diff --git a/internal/testutil/gatelint/testdata/fixtures/std/clean_test.go.fixture b/internal/testutil/gatelint/testdata/fixtures/std/clean_test.go.fixture new file mode 100644 index 000000000..4aeb30a79 --- /dev/null +++ b/internal/testutil/gatelint/testdata/fixtures/std/clean_test.go.fixture @@ -0,0 +1,3 @@ +package fixture + +func TestClean(t *testing.T) {} diff --git a/internal/testutil/gatelint/testdata/fixtures/tests/r3_test.go.fixture b/internal/testutil/gatelint/testdata/fixtures/tests/r3_test.go.fixture new file mode 100644 index 000000000..335100fc4 --- /dev/null +++ b/internal/testutil/gatelint/testdata/fixtures/tests/r3_test.go.fixture @@ -0,0 +1,3 @@ +package fixture + +const endpoint = "https://httpbin.org/post" From 4c5906c5c1cee7549f0148ae299d85406de1eab4 Mon Sep 17 00:00:00 2001 From: "Voight-Kampff (bot)" <151556158+sunholo-voight-kampff@users.noreply.github.com> Date: Wed, 5 Aug 2026 20:17:18 +0200 Subject: [PATCH 2/2] =?UTF-8?q?fix(ci-flake):=20AC10(a)=20poison=20asserti?= =?UTF-8?q?on=20is=20platform-portable=20=E2=80=94=20Windows=20worded=20it?= =?UTF-8?q?=20differently?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI caught what no local gate could: `test-windows` and `Build windows-latest` both failed on the M3 commit because Go reports a refused TCP connect as `connectex: No connection could be made because the target machine actively refused it.` on Windows, versus `connect: connection refused` on Unix. The assertion required the literal `connection refused`, so it could never pass there — the production behavior was correct; only the assertion was parochial. Requires `refused` instead. All three tokens must still match, so the check stays discriminating; verified against four verbatim error strings — Unix refused and Windows refused (the latter copied from the failing CI log) pass, a DNS failure and a proxy hang both fail. Mutation-proved by removing the sentinel proxy wiring (with `go vet` confirming it still compiled, so the run was informative): the assertion redded with "request through poison proxy unexpectedly succeeded". This is the second Windows-only defect in this package that local gates cannot see; M1's was env-var case-insensitivity. Found by the sprint evaluator (sonnet) and reproduced first-party against the job log before applying. Co-Authored-By: Claude Opus 4.8 --- internal/testutil/egress_posture_test.go | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/internal/testutil/egress_posture_test.go b/internal/testutil/egress_posture_test.go index 2bba7b2cd..0372f13ea 100644 --- a/internal/testutil/egress_posture_test.go +++ b/internal/testutil/egress_posture_test.go @@ -102,7 +102,13 @@ func assertPoisonProxyError(t *testing.T, err error) { t.Fatal("request through poison proxy unexpectedly succeeded") } message := err.Error() - for _, required := range []string{"proxyconnect", "127.0.0.1:9", "connection refused"} { + // "refused", not "connection refused": the refusal wording is platform-specific. + // Unix reports `connect: connection refused`; Windows reports `connectex: No + // connection could be made because the target machine actively refused it.` + // All three tokens must match, so this stays discriminating — a direct hit or a + // DNS failure carries no `proxyconnect`/`127.0.0.1:9`, and a proxy that hangs + // rather than refusing carries no `refused`. + for _, required := range []string{"proxyconnect", "127.0.0.1:9", "refused"} { if !strings.Contains(strings.ToLower(message), required) { t.Fatalf("poison error %q does not contain %q", message, required) }