test(ci-flake): M1 — internal/testutil live-network gate + bounded subprocess helpers - #591
Merged
Merged
Conversation
…bprocess helpers Milestone M1 of M-CI-FLAKE-SYSTEMIC-FIX. New package files only; nothing imports these symbols yet, so blast radius is nil. internal/testutil stays stdlib-only (go list -deps | grep sunholo == 1, itself). Adds: - LiveNetworkStatus() -> (decision, reason): a THREE-state predicate (skip / fatal / run) extracted so all three branches are directly testable. You cannot assert "t.Skip was not called" from inside a test that would be skipped — the runtime unwinds — so every in-process test of a skip helper's no-skip direction is vacuous by construction. Testing the predicate makes both directions real. - RequiresLiveNetwork(t): thin wrapper. Skips unless AILANG_LIVE_NET=1; FAILS LOUDLY (never skips, never unsets) when the live lane also carries the poisoned proxy. It must not unset at runtime: Go caches proxy config process-wide on first use, so a later unset silently does nothing (design doc V29). - HangGuard / HangGuardContext: min(cap, until(deadline)-20s), floored at 1s; returns cap unchanged when there is no deadline. Lets a hung operation fail its own subtest cleanly before Go's 300s per-binary panic reds the whole package. - RunBounded: exec.CommandContext + WaitDelay=5s, separate stdout/stderr capture, child exit code (-1 when signal-killed). Anti-vacuity: all 6 new assertions were mutation-tested by the controller outside the sandbox. Negative control (unmutated) green first; each mutation proven landed by byte-diff before its run; all 6 produced a genuine `--- FAIL: <TestName>`, and the exec.Command mutation produced the predicted `panic: test timed out after 30s`. Both source files reverted byte-identical (sha256). Controller correction to the executor's output: the delivered TestRunBounded_KillsHungChild carried a testing.Short() skip. -short is passed NOWHERE in this repo, so that gate is inert — the exact defect this sprint exists to remove, and M3's gatelint rule R1 admits zero exceptions. Measured 7 -> 8 first -party files with the doc's own V2 command; removed, back to 7 (delta empty, control confirms the 7 baseline files still detected). Closes AC5. Refs #583, #494, #509, #587, #561. Co-Authored-By: codex <gpt-5.6-sol> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e case-insensitive
CI caught this on `test-windows` and `Build windows-latest`; both jobs failed for
this single cause, and only the two lower-cased subtests
(TestLiveNetworkStatus_PoisonedProxyFatal/{http_proxy,https_proxy}).
Windows environment variables are case-INSENSITIVE, so HTTP_PROXY and http_proxy
are ONE variable there. The subtest sets the lower-cased name; LiveNetworkStatus
then walks proxyEnvironmentVariables in order, matches on the upper-cased entry
first, and reports that name. The reported name is CORRECT — it is the same
variable — but it is not byte-equal to the subtest's `poisoned` string, so a
case-sensitive strings.Contains could never match on Windows.
Production code is unchanged: it detects the poison correctly on every platform.
Only the test's assertion was over-specific about which casing gets reported.
Still non-vacuous after the relaxation: mutating the reason to drop the variable
name entirely ("a proxy variable points at the poison proxy") turns the test RED
on HTTP_PROXY and HTTPS_PROXY; negative control green; gate.go reverted
sha256-identical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
sunholo-voight-kampff
added a commit
that referenced
this pull request
Aug 5, 2026
…sting.Short() gate caught before it red-lighted its own sprint Gate 4 record for iteration 143. M1 of m-ci-flake-systemic-fix LANDED (PR #591 -> squash c440a16), Gate 3b GREEN SHA-addressed (19 check-runs, all 4 required contexts success), evaluator sonnet PASS 92/100 r1, zero blocking. Two defects caught, one by me before commit and one by CI: - The executor shipped an inert `testing.Short()` gate INTO the package built to replace them. -short is passed nowhere in this repo, so it is defect class C4, and M3's gatelint rule R1 (zero exceptions) would have red-lighted this sprint two milestones later. Measured 7->8 with the doc's own V2 command; removed. - Windows env vars are case-INSENSITIVE, so http_proxy and HTTP_PROXY are one variable; the predicate reports the upper-cased name and a case-sensitive assertion could never match there. Production code was correct. No skill edit. The strongest candidate was my own mutation harness reproducing rule 3a(i-c)'s zsh word-splitting trap (`go test $extra` -> one argv entry -> six false `[setup failed]` reds). But that rule already exists in the form that would have prevented this. A second instance of an existing rule is evidence I failed to apply it, not that the skill needs changing — recorded as a watch-item with the sharper framing instead: the rule reads as being about sed/file-lists, but it governs ANY unquoted variable holding multiple words. STATUS rotation scripted with the line-count assertion; net charter delta 0, 3 stamps remain, queue canaries verified present after the edit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Milestone M1 of
M-CI-FLAKE-SYSTEMIC-FIX(design docdesign_docs/planned/v0_33_1/m-ci-flake-systemic-fix.md, sprint plan7cb798d98).Blast radius: none. New package files only — nothing imports these symbols yet.
internal/testutilstays stdlib-only (go list -deps ./internal/testutil | grep sunholo | wc -l→ 1, itself).What lands
LiveNetworkStatus()— a three-state predicate (skip / fatal / run), deliberately extracted from the*testing.Twrapper. You cannot assert "t.Skipwas not called" from inside a test that would be skipped — the runtime unwinds — so every in-process test of a skip helper's no-skip direction is vacuous by construction. Testing the predicate makes both directions real.RequiresLiveNetwork(t)— skips unlessAILANG_LIVE_NET=1; fails loudly (never skips, never unsets) when the live lane also carries the poisoned proxy. It deliberately does not unset at runtime: Go caches proxy config process-wide on first use, so a later unset silently does nothing (design doc V29).HangGuard/HangGuardContext—min(cap, until(deadline)−20s), floored at 1s; returnscapunchanged when there is no deadline. Lets a hung operation fail its own subtest cleanly before Go's 300s per-binary panic reds the entire package.RunBounded—exec.CommandContext+WaitDelay=5s, separate stdout/stderr capture, child exit code (-1when signal-killed).Anti-vacuity evidence
All 6 new assertions were mutation-tested by the controller outside the sandbox:
AILANG_LIVE_NET != "1"→== "1"--- FAIL: TestLiveNetworkStatus_OptInSetRunsLiveNetworkRun--- FAIL: TestLiveNetworkStatus_UnsetSkipst.Fatalf→t.Skipfin FATAL branch--- FAIL: TestRequiresLiveNetwork_PoisonedLiveLaneFatalexec.CommandContext→exec.Commandpanic: test timed out after 30s(the predicted signal)HangGuardfloortime.Second→0--- FAIL: TestHangGuard_FloorsAtOneSecond0--- FAIL: TestHangGuard_NoDeadlineReturnsCap(-timeout 0)Negative control (unmutated tree) ran green first; each mutation was proven landed by byte-diff before its run; both source files reverted sha256-identical afterwards.
Controller correction to the executor's output
The delivered
TestRunBounded_KillsHungChildcarried atesting.Short()skip.-shortis passed nowhere in this repo, so that gate is inert — the exact defect this sprint exists to remove, and M3's gatelint rule R1 admits zero exceptions. Measured 7 → 8 first-party files with the doc's own V2 command; removed, back to 7 (delta empty; control confirms the 7 baseline files are still detected).Closes AC5. Refs #583, #494, #509, #587, #561.
🤖 Generated with Claude Code