diff --git a/evidence/511-named-gate-diagnostics/README.md b/evidence/511-named-gate-diagnostics/README.md new file mode 100644 index 00000000..fbe21a91 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/README.md @@ -0,0 +1,60 @@ +# Issue #511: a named gate that fails before its command now says why + +Every transcript here is captured output, redirected to the file by the command +it records. Each begins with the literal invocation and ends with its exit +status. The checkout is Linux x64; the reported incident was macOS arm64 and is +not reproduced by any of these runs. + +Common preamble for every run: `RELAYFLOWD_BIN` is passed explicitly because +`npm run test:prep` exports it inside a subshell that never reaches vitest. The +literal value is +`/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd` — +the kernel target lives outside the tree because `ops/cargo.sh` redirects +`CARGO_TARGET_DIR`. Revisions: head `f308f95`, parent `e21caad`. + +- `head-focused.txt` — the two new files at head: 22 passed, exit 0. +- `mutation-reverted.txt` — mutation verification, red half. + `git checkout e21caad -- src/named-gate-lowering.ts` reverts the only + production file; blob hashes before and after the revert are recorded against + `git rev-parse` of both revisions. Rebuilt, re-run: 13 failed, 9 passed. + The 9 are stream-capture cases that already passed before the change. +- `mutation-restored.txt` — green half. `git checkout HEAD --` restores; + `git hash-object` equals `git rev-parse HEAD:…` + (`8a4ac50391d95c95693eb20cdbc69869a5d02060`) and + `git status --porcelain` is empty, so the restore is byte-for-byte. Rebuilt, + re-run: 22 passed, exit 0. +- `head-package-test.txt` — the required package command, `npm test` from + `packages/sdk`: kernel build, typecheck, build, test typecheck, then vitest. + Complete output including every failure. `Test Files 3 failed | 155 passed | + 1 skipped (159)`, `Tests 30 failed | 2393 passed | 17 skipped (2440)`, + exit 1. The package is not green in this sandbox. +- `head-isolated-three-files.txt` — the three failing files alone at head: + 30 failed, 23 passed, 14 skipped. +- `baseline-in-place-e21caad.txt` — the same three files in the same working + directory with `packages/sdk` reverted to `e21caad` + (`git diff --stat e21caad -- packages/sdk` empty, the two added test files + removed). Identical result: 30 failed, 23 passed, 14 skipped, same cases, + same line numbers. This is the baseline behind the claim that those failures + are pre-existing; the revision is the only variable between it and + `head-isolated-three-files.txt`. +- `restore-identity.txt` — restoring the tree after that baseline. + `git status --porcelain -- packages/sdk` empty and all four changed blobs + hash-identical to `HEAD`. +- `unrelated-failure-cause.txt` — why 22 of those failures happen here: node + resolves `@relayflows/surface` to two different installs, one inside the + checkout and one in an ancestor directory of it + (`/home/daytona/.relayflow-v2-supervisor/durable/node_modules`), so the + handle the workflow creates is absent from the `WeakMap` the test's copy of + `getFlowDefinition` consults (`packages/surface/src/flow.ts:137-143`). A + sandbox layout defect, not a code one. +- `baseline-e21caad.txt` — the same three files at `e21caad` in a separate + `git worktree` under `/tmp` with `node_modules` symlinked. Reported for + completeness and **not** used as the baseline: only 1 failed there, because + `/tmp` has no ancestor `@relayflows` install, which changes the resolution + above. The worktree has since been removed. `baseline-in-place-e21caad.txt` + is the comparison that holds the environment fixed. + +Not run: the Rust suite (no kernel code changed) and any Cloud runner-log +publication, whose producer does not exist in this repository. Nothing here +demonstrates the `flows logs ` acceptance item; `summary.md` records it +as unmet. diff --git a/evidence/511-named-gate-diagnostics/baseline-e21caad.txt b/evidence/511-named-gate-diagnostics/baseline-e21caad.txt new file mode 100644 index 00000000..6e1574a2 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/baseline-e21caad.txt @@ -0,0 +1,107 @@ +# Baseline run at the parent revision e21caad, in a detached git worktree. +$ git -C /tmp/flows-baseline-e21caad rev-parse HEAD +e21caad1e4020d9e2675d2840301cb7eeb6f1aac +$ git -C /tmp/flows-baseline-e21caad status --porcelain +?? packages/sdk/node_modules +# node_modules is a symlink to the main checkout (npm install is not rerun): +$ ls -l /tmp/flows-baseline-e21caad/packages/sdk/node_modules +lrwxrwxrwx 1 daytona daytona 83 Sep 20 17:41 /tmp/flows-baseline-e21caad/packages/sdk/node_modules -> /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules + +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/live-kernel.test.ts tests/stuck-run-triage.test.ts tests/authored-node-runtime.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /tmp/flows-baseline-e21caad/packages/sdk + + ❯ tests/authored-node-runtime.test.ts (14 tests | 14 skipped) 19ms +stdout | tests/live-kernel.test.ts +LIVE_KERNEL relayflowd=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +LIVE_KERNEL flows=/tmp/flows-baseline-e21caad/packages/sdk/dist/cli.js + + ✓ tests/stuck-run-triage.test.ts (22 tests) 3143ms + ✓ stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS 3084ms +stdout | tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +LIVE_ANALYZER ready: claude -p --model claude-haiku-4-5-20251001 round-trip OK + +stdout | tests/live-kernel.test.ts > surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once +LIVE_KERNEL kill -9 pid=80459 run=01M2ZYKSAN5HW6C58HHTTVPSA8 while step=two state=Running + + ❯ tests/live-kernel.test.ts (31 tests | 1 failed) 56356ms + ✓ built flows CLI against live relayflowd > twenty-six-step reuses 25 durable completions after editing the failed final step 1953ms + ✓ built flows CLI against live relayflowd > runs rung (a), parks rung (b), and keeps JSON report-shaped 2298ms + ✓ built flows CLI against live relayflowd > allows a deterministic run to exceed the bounded request timeout 32442ms + ✓ built flows CLI against live relayflowd > follows a live worker dispatch through flows run 567ms + ✓ built flows CLI against live relayflowd > runs an agent CLI end to end through the SDK worker 472ms + ✓ built flows CLI against live relayflowd > f.agent lowers to a real agent step and dispatches through a live worker 576ms + ✓ built flows CLI against live relayflowd > can always get a parked run to a late-attaching worker 5577ms + ✓ built flows CLI against live relayflowd > reports a real manual-recovery NeedsHuman state as parked 413ms + ✓ built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) 418ms + ✓ built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields 440ms + ✓ built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text 460ms + ✓ built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) 527ms + ✓ built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) 534ms + ✓ built flows CLI against live relayflowd > AgentWorker passes a declared model to an identified wrapper as RELAYFLOW_MODEL 634ms + ✓ built flows CLI against live relayflowd > AgentWorker refuses a nonconforming journal-submitted wrapper before exposing RELAYFLOW_MODEL 470ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw claude adapter with its real model flag 432ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw codex adapter with its real model flag 479ms + ✓ built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model 541ms + × built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI 3805ms + → expected { …(3) } to match object { Object (gate, verdict) } +(1 matching property omitted from actual) + ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 720ms + ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 431ms + ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 807ms + ✓ a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant 627ms + +⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/authored-node-runtime.test.ts [ tests/authored-node-runtime.test.ts ] +AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality + +Expected: "1.4.0" +Received: "1.3.6" + + ❯ tests/authored-node-runtime.test.ts:18:77 + 16| + 17| beforeAll(() => { + 18| expect(spawnSync(bun, ['--version'], { encoding: 'utf8' }).stdout.tr… + | ^ + 19| expect(existsSync(daemon), 'build the current kernel or set RELAYFLO… + 20| stage = mkdtempSync(join(tmpdir(), 'authored-standalone-build-')); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/2]⎯ + +⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +AssertionError: expected { …(3) } to match object { Object (gate, verdict) } +(1 matching property omitted from actual) + +- Expected ++ Received + + Object { +- "gate": "json_schema", +- "verdict": "pass", ++ "gate": "execution", ++ "verdict": "fail", + } + + ❯ tests/live-kernel.test.ts:1317:49 + 1315| // the promoted output, not the test re-deriving it: the spec is + 1316| // the unmodified canonical one, so this record is the gate. + 1317| expect(stepCompleted!.payload.verification).toMatchObject({ + | ^ + 1318| gate: 'json_schema', + 1319| verdict: 'pass', + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/2]⎯ + + Test Files 2 failed | 1 passed (3) + Tests 1 failed | 52 passed | 14 skipped (67) + Start at 17:41:55 + Duration 57.63s (transform 1.01s, setup 0ms, collect 2.40s, tests 59.52s, environment 0ms, prepare 114ms) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/baseline-in-place-e21caad.txt b/evidence/511-named-gate-diagnostics/baseline-in-place-e21caad.txt new file mode 100644 index 00000000..69d2c02e --- /dev/null +++ b/evidence/511-named-gate-diagnostics/baseline-in-place-e21caad.txt @@ -0,0 +1,560 @@ +# In-place baseline: same working directory and environment as the HEAD run, +# with packages/sdk reverted to the parent revision e21caad. This isolates the +# revision as the only variable. +$ git checkout e21caad -- packages/sdk +$ rm packages/sdk/tests/named-gate-diagnostics.test.ts packages/sdk/tests/named-gate-journal.test.ts # files this PR adds +$ git diff --stat e21caad -- packages/sdk # empty = package content equals the parent revision +$ git hash-object packages/sdk/src/named-gate-lowering.ts; git rev-parse e21caad:packages/sdk/src/named-gate-lowering.ts +479136c28de4f97c5434800577999bbbb1eca59c +479136c28de4f97c5434800577999bbbb1eca59c + +$ cd packages/sdk && RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/live-kernel.test.ts tests/stuck-run-triage.test.ts tests/authored-node-runtime.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ❯ tests/stuck-run-triage.test.ts (22 tests | 22 failed) 77ms + × stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup 4ms + → expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it 1ms + → expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses an empty batch 0ms + → expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses a batch too large for the edge step lease 0ms + → expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound 3ms + → promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + × stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin 0ms + → expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > refuses a non-URL apiUrl 0ms + → expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > allows an approved origin and uses it in the curl 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > defaults to production Cloud 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > never publishes a run record the fetch did not produce 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > names the Worker on every wrangler invocation 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > records wrangler's own exit status rather than head's 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > parses under both sh and bash 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS 61ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > declares read-only permissions on every agent 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > tells the forensics agents their evidence is untrusted 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file 1ms + → expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason 0ms + → expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > bounds ids x workers, not just ids 0ms + → expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + ❯ tests/authored-node-runtime.test.ts (14 tests | 14 skipped) 12ms +stdout | tests/live-kernel.test.ts +LIVE_KERNEL relayflowd=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +LIVE_KERNEL flows=/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/dist/cli.js + +stdout | tests/live-kernel.test.ts > surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once +LIVE_KERNEL kill -9 pid=83147 run=01M2ZYXA39PCN57YRZ2FNSDGJ8 while step=two state=Running + + ❯ tests/live-kernel.test.ts (31 tests | 8 failed) 52176ms + ✓ built flows CLI against live relayflowd > twenty-six-step reuses 25 durable completions after editing the failed final step 1987ms + ✓ built flows CLI against live relayflowd > runs rung (a), parks rung (b), and keeps JSON report-shaped 2302ms + ✓ built flows CLI against live relayflowd > allows a deterministic run to exceed the bounded request timeout 32424ms + ✓ built flows CLI against live relayflowd > follows a live worker dispatch through flows run 547ms + ✓ built flows CLI against live relayflowd > runs an agent CLI end to end through the SDK worker 367ms + ✓ built flows CLI against live relayflowd > f.agent lowers to a real agent step and dispatches through a live worker 545ms + ✓ built flows CLI against live relayflowd > can always get a parked run to a late-attaching worker 5577ms + ✓ built flows CLI against live relayflowd > reports a real manual-recovery NeedsHuman state as parked 418ms + × built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) 568ms + → expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + × built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields 548ms + → expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + × built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text 519ms + → expected null not to be null + × built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) 493ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) 503ms + → Cannot read properties of null (reading 'env_present') + ✓ built flows CLI against live relayflowd > AgentWorker passes a declared model to an identified wrapper as RELAYFLOW_MODEL 565ms + ✓ built flows CLI against live relayflowd > AgentWorker refuses a nonconforming journal-submitted wrapper before exposing RELAYFLOW_MODEL 415ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw claude adapter with its real model flag 371ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw codex adapter with its real model flag 407ms + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model 439ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI 37ms + → LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 725ms + ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 449ms + ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 813ms + × a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant 436ms + → expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/authored-node-runtime.test.ts [ tests/authored-node-runtime.test.ts ] +AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality + +Expected: "1.4.0" +Received: "1.3.6" + + ❯ tests/authored-node-runtime.test.ts:18:77 + 16| + 17| beforeAll(() => { + 18| expect(spawnSync(bun, ['--version'], { encoding: 'utf8' }).stdout.tr… + | ^ + 19| expect(existsSync(daemon), 'build the current kernel or set RELAYFLO… + 20| stage = mkdtempSync(join(tmpdir(), 'authored-standalone-build-')); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/31]⎯ + +⎯⎯⎯⎯⎯⎯ Failed Tests 30 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) +AssertionError: expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "output": Object { +- "reasoning": "stub agent runtime — deterministic output for gate-2 clause-2 demo", +- "relevance_score": 5, +- "story_title": "stub", +- }, ++ "output": null, + "verification": Object { +- "gate": "json_schema", +- "verdict": "pass", ++ "gate": "execution", ++ "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:657:36 + 655| && (entry as { step_id?: string }).step_id === 'analyze-story', + 656| ) as { payload: { output: unknown; verification: unknown } } | und… + 657| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 658| output: { + 659| story_title: 'stub', + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields +AssertionError: expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "completionReason": "retries_exhausted", ++ "completionReason": "worker_error", + "output": null, + "verification": Object { +- "gate": "json_schema", ++ "gate": "execution", + "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:752:36 + 750| // its verification record names the json_schema rejection. The re… + 751| // parsed value is nulled before the completion is persisted. + 752| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 753| completionReason: 'retries_exhausted', + 754| output: null, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text +AssertionError: expected null not to be null + ❯ tests/live-kernel.test.ts:823:24 + 821| // here (parseJsonOutput returned null on non-JSON stdout) and + 822| // these assertions would all fail. + 823| expect(output).not.toBeNull(); + | ^ + 824| expect(output.exit_code).toBe(0); + 825| expect(output.stdout_tail).toContain('looked at the story'); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:891:42 + 889| ) as { payload: { output: { story_title: string; reasoning: string… + 890| expect(stepCompleted).toBeDefined(); + 891| expect(stepCompleted!.payload.output.story_title).toBe(`echoed:${s… + | ^ + 892| expect(stepCompleted!.payload.output.reasoning).toContain(String(s… + 893| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) +TypeError: Cannot read properties of null (reading 'env_present') + ❯ tests/live-kernel.test.ts:958:38 + 956| ) as { payload: { output: { env_present: boolean } } } | undefined; + 957| expect(completed).toBeDefined(); + 958| expect(completed!.payload.output.env_present).toBe(false); + | ^ + 959| + 960| delete process.env.RELAYFLOW_WAKE_CONTEXT; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:1194:38 + 1192| expect(completed).toBeDefined(); + 1193| // UNSET, not EMPTY and not the leaked parent value. + 1194| expect(completed!.payload.output.story_title).toBe('model:UNSET'); + | ^ + 1195| + 1196| delete process.env.RELAYFLOW_MODEL; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[7/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +Error: LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ❯ tests/live-kernel.test.ts:1223:15 + 1221| const notice = `LIVE_ANALYZER_UNAVAILABLE: ${readiness.detail}`; + 1222| if (process.env['RELAYFLOWS_ALLOW_ANALYZER_SKIP'] !== '1') { + 1223| throw new Error( + | ^ + 1224| `${notice} — failing because gate-2 acceptance requires the … + 1225| + 'Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is … + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[8/31]⎯ + + FAIL tests/live-kernel.test.ts > a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant +AssertionError: expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +- Expected: +Object { + "lag_ms": 43000, + "schedule_id": "heartbeat-1m", + "scheduled_for_ms": 1764000000000, + "slot": 29400000, +} + ++ Received: +null + + ❯ tests/live-kernel.test.ts:1665:39 + 1663| // The bound: the run reports the grid instant and its own lag, so… + 1664| // backfilled run can tell it is running for a slot from the past. + 1665| expect(completed!.payload.output).toEqual({ + | ^ + 1666| schedule_id: 'heartbeat-1m', + 1667| slot: 29_400_000, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[9/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: c649fe14/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[10/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: nope!/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[11/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an empty batch +AssertionError: expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/needs runIds/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses a batch too large for the edge step lease +AssertionError: expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/exceeds the 8 that fit/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound +AssertionError: promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + ❯ tests/stuck-run-triage.test.ts:62:40 + 60| it('accepts eight ids — the incident batch is inside the bound', asy… + 61| const ids = Array.from({ length: 8 }, (_, i) => `${ID_A.slice(0, -… + 62| await expect(drive({ runIds: ids })).resolves.toBeDefined(); + | ^ + 63| }); + 64| }); + +Caused by: TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + ❯ tests/stuck-run-triage.test.ts:62:18 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[14/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin +AssertionError: expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + +- Expected: +/refusing to send the Cloud bearer token to https:\/\/evil\.example/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[15/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses a non-URL apiUrl +AssertionError: expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/is not a URL/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[16/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > allows an approved origin and uses it in the curl +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:77:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[17/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > defaults to production Cloud +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:82:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[18/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > never publishes a run record the fetch did not produce +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:89:34 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[19/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > names the Worker on every wrangler invocation +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:98:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[20/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:107:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[21/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:115:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[22/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:123:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[23/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > records wrangler's own exit status rather than head's +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:129:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[24/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > parses under both sh and bash +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:137:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[25/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:157:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[26/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > declares read-only permissions on every agent +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:176:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[27/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > tells the forensics agents their evidence is untrusted +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:182:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[28/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file +AssertionError: expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate runIds: c649fe14-0c2e-4e51-9a6a-4f0d1b0f77aa/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[29/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason +AssertionError: expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate workers: w-one/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[30/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > bounds ids x workers, not just ids +AssertionError: expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/24 concurrent tails, over the 16/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[31/31]⎯ + + Test Files 3 failed (3) + Tests 30 failed | 23 passed | 14 skipped (67) + Start at 17:47:11 + Duration 53.45s (transform 943ms, setup 0ms, collect 2.43s, tests 52.26s, environment 0ms, prepare 149ms) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/head-focused.txt b/evidence/511-named-gate-diagnostics/head-focused.txt new file mode 100644 index 00000000..84111477 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/head-focused.txt @@ -0,0 +1,25 @@ +$ git rev-parse HEAD +f308f95683c46eaddcf5c22eefd35aa04d2d1500 +$ git status --porcelain -- src tests tsconfig.tests.json +$ git hash-object src/named-gate-lowering.ts +8a4ac50391d95c95693eb20cdbc69869a5d02060 +$ export RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +$ npm run build && npx vitest run tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ✓ tests/named-gate-diagnostics.test.ts (17 tests) 535ms + ✓ tests/named-gate-journal.test.ts (5 tests) 1664ms + ✓ a lowered subprocess_gate journals the gate command's streams > persists what the gate printed before a timeout killed it 816ms + ✓ a gate on an agent step > journals the command's streams and surfaces them in the authored failure 582ms + + Test Files 2 passed (2) + Tests 22 passed (22) + Start at 17:40:34 + Duration 2.88s (transform 692ms, setup 0ms, collect 1.46s, tests 2.20s, environment 0ms, prepare 75ms) + +EXIT=0 diff --git a/evidence/511-named-gate-diagnostics/head-isolated-three-files.txt b/evidence/511-named-gate-diagnostics/head-isolated-three-files.txt new file mode 100644 index 00000000..57a10f8f --- /dev/null +++ b/evidence/511-named-gate-diagnostics/head-isolated-three-files.txt @@ -0,0 +1,555 @@ +# Same three files at HEAD (f308f95), run in isolation, for a like-for-like comparison with baseline-e21caad.txt. +$ git rev-parse HEAD +f308f95683c46eaddcf5c22eefd35aa04d2d1500 +$ git status --porcelain -- src tests tsconfig.tests.json + +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/live-kernel.test.ts tests/stuck-run-triage.test.ts tests/authored-node-runtime.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ❯ tests/stuck-run-triage.test.ts (22 tests | 22 failed) 99ms + × stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup 5ms + → expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it 1ms + → expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses an empty batch 0ms + → expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses a batch too large for the edge step lease 0ms + → expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound 20ms + → promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + × stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin 1ms + → expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > refuses a non-URL apiUrl 1ms + → expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > allows an approved origin and uses it in the curl 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > defaults to production Cloud 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > never publishes a run record the fetch did not produce 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > names the Worker on every wrangler invocation 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > records wrangler's own exit status rather than head's 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > parses under both sh and bash 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS 65ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > declares read-only permissions on every agent 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > tells the forensics agents their evidence is untrusted 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file 1ms + → expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason 0ms + → expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > bounds ids x workers, not just ids 1ms + → expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + ❯ tests/authored-node-runtime.test.ts (14 tests | 14 skipped) 21ms +stdout | tests/live-kernel.test.ts +LIVE_KERNEL relayflowd=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +LIVE_KERNEL flows=/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/dist/cli.js + +stdout | tests/live-kernel.test.ts > surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once +LIVE_KERNEL kill -9 pid=81494 run=01M2ZYPBSJ865AJXCAN8WSB46S while step=two state=Running + + ❯ tests/live-kernel.test.ts (31 tests | 8 failed) 53093ms + ✓ built flows CLI against live relayflowd > twenty-six-step reuses 25 durable completions after editing the failed final step 2014ms + ✓ built flows CLI against live relayflowd > runs rung (a), parks rung (b), and keeps JSON report-shaped 2275ms + ✓ built flows CLI against live relayflowd > allows a deterministic run to exceed the bounded request timeout 32429ms + ✓ built flows CLI against live relayflowd > follows a live worker dispatch through flows run 533ms + ✓ built flows CLI against live relayflowd > runs an agent CLI end to end through the SDK worker 469ms + ✓ built flows CLI against live relayflowd > f.agent lowers to a real agent step and dispatches through a live worker 749ms + ✓ built flows CLI against live relayflowd > can always get a parked run to a late-attaching worker 5577ms + ✓ built flows CLI against live relayflowd > reports a real manual-recovery NeedsHuman state as parked 464ms + × built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) 513ms + → expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + × built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields 614ms + → expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + × built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text 573ms + → expected null not to be null + × built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) 607ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) 499ms + → Cannot read properties of null (reading 'env_present') + ✓ built flows CLI against live relayflowd > AgentWorker passes a declared model to an identified wrapper as RELAYFLOW_MODEL 568ms + ✓ built flows CLI against live relayflowd > AgentWorker refuses a nonconforming journal-submitted wrapper before exposing RELAYFLOW_MODEL 557ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw claude adapter with its real model flag 400ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw codex adapter with its real model flag 488ms + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model 446ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI 33ms + → LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 731ms + ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 439ms + ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 817ms + × a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant 514ms + → expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/authored-node-runtime.test.ts [ tests/authored-node-runtime.test.ts ] +AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality + +Expected: "1.4.0" +Received: "1.3.6" + + ❯ tests/authored-node-runtime.test.ts:18:77 + 16| + 17| beforeAll(() => { + 18| expect(spawnSync(bun, ['--version'], { encoding: 'utf8' }).stdout.tr… + | ^ + 19| expect(existsSync(daemon), 'build the current kernel or set RELAYFLO… + 20| stage = mkdtempSync(join(tmpdir(), 'authored-standalone-build-')); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/31]⎯ + +⎯⎯⎯⎯⎯⎯ Failed Tests 30 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) +AssertionError: expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "output": Object { +- "reasoning": "stub agent runtime — deterministic output for gate-2 clause-2 demo", +- "relevance_score": 5, +- "story_title": "stub", +- }, ++ "output": null, + "verification": Object { +- "gate": "json_schema", +- "verdict": "pass", ++ "gate": "execution", ++ "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:657:36 + 655| && (entry as { step_id?: string }).step_id === 'analyze-story', + 656| ) as { payload: { output: unknown; verification: unknown } } | und… + 657| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 658| output: { + 659| story_title: 'stub', + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields +AssertionError: expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "completionReason": "retries_exhausted", ++ "completionReason": "worker_error", + "output": null, + "verification": Object { +- "gate": "json_schema", ++ "gate": "execution", + "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:752:36 + 750| // its verification record names the json_schema rejection. The re… + 751| // parsed value is nulled before the completion is persisted. + 752| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 753| completionReason: 'retries_exhausted', + 754| output: null, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text +AssertionError: expected null not to be null + ❯ tests/live-kernel.test.ts:823:24 + 821| // here (parseJsonOutput returned null on non-JSON stdout) and + 822| // these assertions would all fail. + 823| expect(output).not.toBeNull(); + | ^ + 824| expect(output.exit_code).toBe(0); + 825| expect(output.stdout_tail).toContain('looked at the story'); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:891:42 + 889| ) as { payload: { output: { story_title: string; reasoning: string… + 890| expect(stepCompleted).toBeDefined(); + 891| expect(stepCompleted!.payload.output.story_title).toBe(`echoed:${s… + | ^ + 892| expect(stepCompleted!.payload.output.reasoning).toContain(String(s… + 893| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) +TypeError: Cannot read properties of null (reading 'env_present') + ❯ tests/live-kernel.test.ts:958:38 + 956| ) as { payload: { output: { env_present: boolean } } } | undefined; + 957| expect(completed).toBeDefined(); + 958| expect(completed!.payload.output.env_present).toBe(false); + | ^ + 959| + 960| delete process.env.RELAYFLOW_WAKE_CONTEXT; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:1194:38 + 1192| expect(completed).toBeDefined(); + 1193| // UNSET, not EMPTY and not the leaked parent value. + 1194| expect(completed!.payload.output.story_title).toBe('model:UNSET'); + | ^ + 1195| + 1196| delete process.env.RELAYFLOW_MODEL; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[7/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +Error: LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ❯ tests/live-kernel.test.ts:1223:15 + 1221| const notice = `LIVE_ANALYZER_UNAVAILABLE: ${readiness.detail}`; + 1222| if (process.env['RELAYFLOWS_ALLOW_ANALYZER_SKIP'] !== '1') { + 1223| throw new Error( + | ^ + 1224| `${notice} — failing because gate-2 acceptance requires the … + 1225| + 'Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is … + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[8/31]⎯ + + FAIL tests/live-kernel.test.ts > a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant +AssertionError: expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +- Expected: +Object { + "lag_ms": 43000, + "schedule_id": "heartbeat-1m", + "scheduled_for_ms": 1764000000000, + "slot": 29400000, +} + ++ Received: +null + + ❯ tests/live-kernel.test.ts:1665:39 + 1663| // The bound: the run reports the grid instant and its own lag, so… + 1664| // backfilled run can tell it is running for a slot from the past. + 1665| expect(completed!.payload.output).toEqual({ + | ^ + 1666| schedule_id: 'heartbeat-1m', + 1667| slot: 29_400_000, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[9/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: c649fe14/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[10/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: nope!/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[11/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an empty batch +AssertionError: expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/needs runIds/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses a batch too large for the edge step lease +AssertionError: expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/exceeds the 8 that fit/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound +AssertionError: promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + ❯ tests/stuck-run-triage.test.ts:62:40 + 60| it('accepts eight ids — the incident batch is inside the bound', asy… + 61| const ids = Array.from({ length: 8 }, (_, i) => `${ID_A.slice(0, -… + 62| await expect(drive({ runIds: ids })).resolves.toBeDefined(); + | ^ + 63| }); + 64| }); + +Caused by: TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + ❯ tests/stuck-run-triage.test.ts:62:18 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[14/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin +AssertionError: expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + +- Expected: +/refusing to send the Cloud bearer token to https:\/\/evil\.example/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[15/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses a non-URL apiUrl +AssertionError: expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/is not a URL/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[16/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > allows an approved origin and uses it in the curl +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:77:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[17/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > defaults to production Cloud +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:82:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[18/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > never publishes a run record the fetch did not produce +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:89:34 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[19/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > names the Worker on every wrangler invocation +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:98:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[20/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:107:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[21/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:115:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[22/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:123:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[23/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > records wrangler's own exit status rather than head's +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:129:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[24/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > parses under both sh and bash +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:137:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[25/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:157:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[26/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > declares read-only permissions on every agent +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:176:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[27/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > tells the forensics agents their evidence is untrusted +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:182:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[28/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file +AssertionError: expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate runIds: c649fe14-0c2e-4e51-9a6a-4f0d1b0f77aa/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[29/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason +AssertionError: expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate workers: w-one/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[30/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > bounds ids x workers, not just ids +AssertionError: expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/24 concurrent tails, over the 16/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[31/31]⎯ + + Test Files 3 failed (3) + Tests 30 failed | 23 passed | 14 skipped (67) + Start at 17:43:23 + Duration 54.32s (transform 839ms, setup 0ms, collect 2.22s, tests 53.21s, environment 0ms, prepare 120ms) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/head-package-test.txt b/evidence/511-named-gate-diagnostics/head-package-test.txt new file mode 100644 index 00000000..22ac5b54 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/head-package-test.txt @@ -0,0 +1,935 @@ +# Required package command at HEAD (f308f95), run from packages/sdk. +# RELAYFLOWD_BIN is passed explicitly because test:prep exports it in a subshell. +$ git rev-parse HEAD +f308f95683c46eaddcf5c22eefd35aa04d2d1500 +$ git status --porcelain -- packages/sdk +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm test + +> @relayflows/sdk@2.0.22 test +> sh scripts/test.sh + + +> @relayflows/sdk@2.0.22 test:prep +> ( cd ../../kernel && sh ../ops/cargo.sh build ) && ( [ ! -d ../../testdata/preflight ] || find ../../testdata/preflight -name '*-cli' -type f -exec chmod +x {} + ) + + Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.07s + +> @relayflows/sdk@2.0.22 typecheck +> tsc --noEmit && tsc -p tsconfig.type-tests.json + + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + +> @relayflows/sdk@2.0.22 typecheck:tests +> tsc -p tsconfig.tests.json + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + +stdout | tests/live-kernel.test.ts +LIVE_KERNEL relayflowd=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +LIVE_KERNEL flows=/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/dist/cli.js + + ✓ tests/cloud-read.test.ts (39 tests) 73ms + ✓ tests/preflight.test.ts (57 tests) 92ms + ✓ tests/cli.test.ts (65 tests) 1415ms + ✓ flows check CLI > binds a checked relative wrapper to the flow directory for worker execution 375ms + ✓ tests/cloud-sync.test.ts (40 tests) 706ms + ✓ tests/observer-link.test.ts (39 tests) 146ms + ✓ tests/agent-transcript.test.ts (29 tests) 269ms + ✓ tests/cloud-run.test.ts (58 tests) 682ms +(node:84245) ExperimentalWarning: SQLite is an experimental feature and might change at any time +(Use `node --trace-warnings ...` to show where the warning was created) + ✓ tests/cli-status.test.ts (26 tests) 901ms + ✓ flows status > resolves the run with no arguments from inside a worker-spawned agent 711ms + ✓ tests/relay-cli-surface.test.ts (66 tests) 24ms + ✓ tests/authored-flow.test.ts (25 tests) 726ms + ✓ tests/daemon-lifecycle.test.ts (42 tests) 35ms + ✓ tests/run-state.test.ts (21 tests) 11ms + ✓ tests/cloud-deploy.test.ts (40 tests) 1072ms + ✓ tests/cloud-connect.test.ts (24 tests) 3290ms + ✓ hosted verbs connect before they submit > flows run --cloud submits once the prompt connected the integration 2152ms + ✓ tests/step-failure-diagnostic.test.ts (21 tests) 36ms + ✓ tests/worker-cli.test.ts (18 tests) 25255ms + ✓ registered CLI model defaults > passes the same priced Claude default to the real provider invocation 397ms + ✓ step discovery environment > names the run, step, attempt and an absolute data dir for a direct agent spawn 404ms + ✓ step discovery environment > exports none of the four without a data dir, even when the worker inherited them 334ms + ✓ wrapper discovery environment > exports none of the four to a wrapper without a data dir, even when the worker inherited them 365ms + ✓ custom wrapper execution identity > passes an explicit safe environment at identification and execution 346ms + ✓ custom wrapper execution identity > refuses a wrapper symlink retarget before delivering private values 408ms + ✓ custom wrapper execution identity > bounds wrapper execution after acknowledgement 426ms + ✓ custom wrapper execution identity > bounds captured wrapper output 342ms + ✓ custom wrapper execution identity > refuses a duplicate execute protocol frame 353ms + ✓ custom wrapper execution bounds are reader-owned > resolves when a conforming wrapper leaks a stdio pipe to a background helper 1938ms + ✓ custom wrapper execution bounds are reader-owned > resolves when the leaked helper inherits stderr only 1911ms + ✓ custom wrapper execution bounds are reader-owned > resolves when a wrapper leaks a stdio pipe and exits before identifying 3689ms + ✓ custom wrapper execution bounds are reader-owned > journals a completionReason at the default bound when a wrapper leaks a stdio pipe 11627ms + ✓ custom wrapper execution bounds are reader-owned > accepts an execute token and an over-8KiB payload flushed in one write 410ms + ✓ custom wrapper execution bounds are reader-owned > accepts the same over-8KiB payload whether or not it coalesces with the execute token 1185ms + ✓ custom wrapper execution bounds are reader-owned > still bounds an un-terminated handshake buffer and names the bound 377ms + ✓ delivers the journaled memory pack to the real wrapper and excludes its charge from completion usage 458ms + ❯ tests/authored-node-runtime.test.ts (14 tests | 14 skipped) 13ms + ✓ tests/close-pr-flow.test.ts (28 tests) 332ms + ✓ tests/journal-client.test.ts (15 tests) 77ms + ✓ tests/validate.test.ts (68 tests) 20ms + ✓ tests/verb-field-lint.test.ts (96 tests) 271ms + ✓ tests/authored-root.test.ts (12 tests) 175ms + ✓ tests/mcp.test.ts (30 tests) 20250ms + ✓ MCP preflight and transports > flows check refuses an undeclared server with exit 2 and no daemon 545ms + ✓ MCP preflight and transports > flows check reports a refusing server and leaves no PID 543ms + ✓ MCP preflight and transports > kills a SIGTERM-resistant silent child after a parent-owned handshake deadline 1319ms + ✓ MCP preflight and transports > reaps a SIGTERM-resistant descendant with inherit stdio before cleanup finishes 1121ms + ✓ MCP preflight and transports > reaps a SIGTERM-resistant descendant with ignore stdio before cleanup finishes 2065ms + ✓ MCP preflight and transports > reports malformed connection configuration as config_invalid 542ms + ✓ authored MCP effects against the real kernel > reports a dropped tool connection as a failed CLI run 13173ms + ✓ tests/tick-source.test.ts (33 tests) 23ms + ✓ tests/agent-relay-transport.test.ts (16 tests) 2220ms + ✓ Relay completion at the journal boundary > does not complete at readiness and journals exact output, receipt, and priced accounting 1005ms + ✓ Relay completion at the journal boundary > aborts polling on rejected renewal and never writes a stale completion 1003ms + ✓ tests/authored-flow-lifecycle-executor.test.ts (27 tests) 605ms + ✓ tests/pr-review-post.test.ts (21 tests) 2462ms + ✓ tests/flow-executor-chain.test.ts (14 tests) 10570ms + ✓ flow executor LLM and output-binding chain > runs f.llm -> f.agent -> f.run with schema-verified journal output and the exact allowed model 1018ms + ✓ flow executor LLM and output-binding chain > runs a dollar-budgeted authored Claude agent with the same default used by preflight 614ms + ✓ flow executor LLM and output-binding chain > preserves JSON values without promoting them to process wrappers: [1,2] 332ms + ✓ flow executor LLM and output-binding chain > preserves JSON values without promoting them to process wrappers: "hello" 365ms + ✓ flow executor LLM and output-binding chain > runs the exact authored flagship f.llm -> f.agent -> f.run path through the durable CLI root 1512ms + ✓ flow executor LLM and output-binding chain > resumes an interrupted durable authored root without replaying completed flagship effects 3531ms + ✓ flow executor LLM and output-binding chain > passes a declarative verified value through an agent into a deterministic artifact 886ms + ✓ flow executor LLM and output-binding chain > journals a missing optional field as a failure before the consuming command executes 378ms + ✓ flow executor LLM and output-binding chain > flows run consumes YAML bindings and resume reuses the original journal output 914ms + ✓ tests/authored-flow-slack.test.ts (7 tests) 1633ms + ✓ authored Slack helper effects > replays after SIGKILL before confirm with the same token and one successful completion 493ms + ✓ authored Slack helper effects > replays after SIGKILL before complete with the same token and one successful completion 474ms +(node:86295) ExperimentalWarning: SQLite is an experimental feature and might change at any time +(Use `node --trace-warnings ...` to show where the warning was created) + ✓ tests/cli-replay.test.ts (37 tests) 1011ms + ✓ flows replay > --json is byte-identical across two CLI invocations (diff) 685ms + ✓ tests/tick-runner.test.ts (22 tests) 2093ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms fractional as an invocation error 344ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms exponent notation as an invocation error 339ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms hex as an invocation error 338ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms trailing text as an invocation error 331ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms empty as an invocation error 363ms + ✓ CLI argument parsing refuses coercion rather than accepting it > accepts an exact integer and proceeds past parsing 350ms +stdout | tests/live-kernel.test.ts > surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once +LIVE_KERNEL kill -9 pid=87028 run=01M2ZZ0GYBQ2EHWGJW8AXSB9E2 while step=two state=Running + + ✓ tests/stop-process-group.test.ts (6 tests) 6406ms + ✓ every stop reaches the process group, not just the direct child > exits the run after an execution-timeout stop 852ms + ✓ every stop reaches the process group, not just the direct child > exits the run after a protocol terminate stop 463ms + ✓ every stop reaches the process group, not just the direct child > kills a SIGTERM-deaf grandchild after a protocol terminate stop 1630ms + ✓ every stop reaches the process group, not just the direct child > kills a SIGTERM-deaf grandchild after an execution-timeout stop 2118ms + ✓ every stop reaches the process group, not just the direct child > holds the loop open long enough for the escalation to run 1082ms + ✓ tests/gate-contract.test.ts (20 tests) 118ms + ❯ tests/live-kernel.test.ts (31 tests | 8 failed) 53047ms + ✓ built flows CLI against live relayflowd > twenty-six-step reuses 25 durable completions after editing the failed final step 2008ms + ✓ built flows CLI against live relayflowd > runs rung (a), parks rung (b), and keeps JSON report-shaped 2400ms + ✓ built flows CLI against live relayflowd > allows a deterministic run to exceed the bounded request timeout 32417ms + ✓ built flows CLI against live relayflowd > follows a live worker dispatch through flows run 539ms + ✓ built flows CLI against live relayflowd > runs an agent CLI end to end through the SDK worker 563ms + ✓ built flows CLI against live relayflowd > f.agent lowers to a real agent step and dispatches through a live worker 584ms + ✓ built flows CLI against live relayflowd > can always get a parked run to a late-attaching worker 5578ms + ✓ built flows CLI against live relayflowd > reports a real manual-recovery NeedsHuman state as parked 521ms + × built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) 558ms + → expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + × built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields 501ms + → expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + × built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text 525ms + → expected null not to be null + × built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) 525ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) 501ms + → Cannot read properties of null (reading 'env_present') + ✓ built flows CLI against live relayflowd > AgentWorker passes a declared model to an identified wrapper as RELAYFLOW_MODEL 426ms + ✓ built flows CLI against live relayflowd > AgentWorker refuses a nonconforming journal-submitted wrapper before exposing RELAYFLOW_MODEL 512ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw claude adapter with its real model flag 458ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw codex adapter with its real model flag 487ms + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model 529ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI 37ms + → LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 746ms + ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 481ms + ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 805ms + × a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant 609ms + → expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + ✓ tests/authored-human.test.ts (13 tests) 90ms + ✓ tests/bundle.test.ts (23 tests) 8576ms + ✓ immutable bundles > returns exit 2 naming a byte-flipped payload and refuses to reuse corruption 352ms + ✓ immutable bundles > verifies with --verify in any position and answers --json with one object 697ms + ✓ immutable bundles > refuses --out with --verify rather than ignoring the destination 353ms + ✓ immutable bundles > builds and verifies the canonical YAML fixture through the compiled CLI 1037ms + ✓ immutable bundles > emits the ephemeral warning on CLI stderr and uses the default output directory 713ms + ✓ immutable bundles > refuses build-provable CLI resolution errors without environment probes 351ms + ✓ immutable bundles > builds a standalone TS fixture twice with identical executable hashes 2708ms + ✓ immutable bundles > refuses to label installed dependency drift with lockfile pins 346ms + ✓ immutable bundles > refuses invalid CLI arguments %j 347ms + ✓ immutable bundles > refuses invalid CLI arguments "--out" 370ms + ✓ immutable bundles > refuses invalid CLI arguments "--verify" 359ms + ✓ immutable bundles > refuses invalid CLI arguments "--verify" 373ms + ✓ immutable bundles > refuses invalid CLI arguments "--out" 459ms + ✓ tests/cli-hn-monitor.test.ts (16 tests) 93ms + ✓ tests/named-gate-diagnostics.test.ts (17 tests) 532ms + ✓ tests/cloud-schedule.test.ts (17 tests) 4003ms + ✓ schedule lowering > marks a non-grid cron as Cloud-only rather than approximating it, with a silence budget from its own cadence 1705ms + ✓ flows check prints declared schedules > shows the lowering for a fixed interval and the Cloud-only note for a real cron 1616ms + ✓ tests/authored-node-result.test.ts (38 tests) 39ms + ✓ tests/direct-input.test.ts (6 tests) 5342ms + ✓ direct .flow.ts input through the built CLI and live runtime > returns exit 3 for an authored human handoff and persists its outcome 634ms + ✓ direct .flow.ts input through the built CLI and live runtime > returns exit 1 for an authored step_failed verdict and persists its outcome 583ms + ✓ direct .flow.ts input through the built CLI and live runtime > executes inline and file JSON input through relayflowd 1770ms + ✓ direct .flow.ts input through the built CLI and live runtime > refuses missing and malformed input before contacting relayflowd 1430ms + ✓ direct .flow.ts input through the built CLI and live runtime > does not run the authored body before daemon availability 509ms + ✓ direct .flow.ts input through the built CLI and live runtime > refuses oversized file input before contacting relayflowd 415ms + ✓ tests/daemon-lifecycle-live.test.ts (9 tests) 6385ms + ✓ flows run against a data dir with no daemon (§6 test 7) > cold start spawns exactly one daemon, the run succeeds, and the daemon outlives the CLI 409ms + ✓ flows run against a data dir with no daemon (§6 test 7) > polls, bounded, for a daemon that holds the lock before it binds 1334ms + ✓ flows run against a data dir with no daemon (§6 test 7) > attaches to a serving daemon that has not published a connection file 524ms + ✓ flows run against a data dir with no daemon (§6 test 7) > a second run attaches to the daemon the first one started, spawning nothing 974ms + ✓ flows run against a data dir with no daemon (§6 test 7) > detects a stale connection file left by a hard kill and starts a fresh daemon 873ms + ✓ concurrent invocations against one empty data dir (§6 test 15) > ends with exactly one daemon owning the socket, and both runs succeed 1101ms + ✓ refusals from a spawn that cannot produce a daemon > names relayflowd_not_found rather than falling through to PATH 359ms + ✓ refusals from a spawn that cannot produce a daemon > names daemon_start_failed and quotes the daemon log when startup dies 413ms + ✓ refusals from a spawn that cannot produce a daemon > refuses a daemon speaking another protocol version instead of binding over it 397ms +(node:88133) Warning: Transcript tail for run-9/analyze attempt 1 (stdout) could not be written; the step continues without it: EACCES: permission denied, mkdir '/tmp/transcript-tail-c9FNaN/runs/run-9/steps' +(Use `node --trace-warnings ...` to show where the warning was created) + ✓ tests/transcript-tail.test.ts (11 tests) 831ms + ✓ direct agent spawn > completes the step when the tail directory cannot be created 442ms + ✓ tests/authored-agent-artifacts.test.ts (4 tests) 469ms + ✓ tests/authored-helpers.test.ts (6 tests) 3034ms + ✓ runs every available provider through the real kernel and resumes completed effects without a second write 1653ms + ✓ replays after SIGKILL before confirm with the same token and one successful completion 493ms + ✓ replays after SIGKILL before complete with the same token and one successful completion 471ms + ✓ tests/backlog-picker.test.ts (14 tests) 40ms + ✓ tests/backlog-picker-flow.test.ts (6 tests) 316ms + ✓ tests/preflight-permissions-unenforced.test.ts (17 tests) 261ms + ❯ tests/stuck-run-triage.test.ts (22 tests | 22 failed) 63ms + × stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup 4ms + → expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it 0ms + → expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses an empty batch 0ms + → expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses a batch too large for the edge step lease 0ms + → expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound 3ms + → promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + × stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin 0ms + → expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > refuses a non-URL apiUrl 0ms + → expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > allows an approved origin and uses it in the curl 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > defaults to production Cloud 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > never publishes a run record the fetch did not produce 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > names the Worker on every wrangler invocation 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > records wrangler's own exit status rather than head's 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > parses under both sh and bash 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS 48ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > declares read-only permissions on every agent 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > tells the forensics agents their evidence is untrusted 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file 0ms + → expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason 1ms + → expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > bounds ids x workers, not just ids 0ms + → expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + ✓ tests/worker-transcript.test.ts (5 tests) 187ms + ✓ tests/flow-requirements.test.ts (13 tests) 691ms + ✓ flows check prints REQUIRES > names the helper, the harness and the mcp server of an authored flow 446ms + ✓ tests/webhook.test.ts (9 tests) 500ms + ✓ webhook ingress > checks TS declarations against flows.json without invoking handlers 431ms + ✓ tests/authored-run-failure-evidence.test.ts (8 tests) 747ms + ✓ the child index after the process that wrote it is gone > still names every child, with its own run id, after a daemon restart 441ms + ✓ tests/webhook-live.test.ts (6 tests) 9547ms + ✓ executes and deduplicates 'app_mention' only for its provider and matching payload 1453ms + ✓ executes and deduplicates 'reaction_added' only for its provider and matching payload 1398ms + ✓ executes and deduplicates 'pull_request' only for its provider and matching payload 1415ms + ✓ flows serve-webhook writes JSON before the daemon starts, then journals and archives exactly once 1407ms + ✓ replays a dropped file after SIGKILL before spawn 435ms + ✓ resumes the same journal after SIGKILL after spawn and before acknowledgement 3438ms + ✓ tests/cli-watch.test.ts (10 tests) 14714ms + ✓ flows check --watch > rechecks syntax errors, clears once, and returns the last refusal on Ctrl-C 1209ms + ✓ flows check --watch > streams JSON lines without ANSI, recovers after atomic saves, and exits zero after repair 1712ms + ✓ flows check --watch > coalesces 20 concurrent saves into at most two rechecks 1721ms + ✓ flows check --watch > watches transitive relative use imports, cycles, and nearest config changes 2213ms + ✓ flows check --watch > refreshes the import graph and notices missing imports being created 2215ms + ✓ flows check --watch > reloads authored TypeScript instead of reusing the first imported definition 1454ms + ✓ flows check --watch > detects a nearer config appearing and falls back after it is deleted 1700ms + ✓ flows check --watch > keeps watching after the target is deleted and recreated 1704ms + ✓ flows check --watch > queues changes during a slow check without overlapping checks 781ms + ✓ tests/agent-transcript-live.test.ts (4 tests) 41163ms + ✓ the transcript digest through the built CLI, a real daemon and the local agent > preserves structured agent failure details and its completed root index 12627ms + ✓ the transcript digest through the built CLI, a real daemon and the local agent > preserves structured llm failure details and its completed root index 14532ms + ✓ the transcript digest through the built CLI, a real daemon and the local agent > journals the digest in trajectory_tail on a successful agent step and writes the file it points at 805ms + ✓ the transcript digest through the built CLI, a real daemon and the local agent > on a failed agent step, names the failure and the transcript in the terminal diagnostic, redacted 13198ms + ✓ tests/authored-step-failed.test.ts (10 tests) 37ms + ✓ tests/authored-flow-operation.test.ts (23 tests) 382ms + ✓ tests/agent-artifacts-live.test.ts (5 tests) 44324ms + ✓ agent artifacts and gates through the built CLI, a real daemon and the local agent > journals the files the agent wrote, and both artifact gates pass on that journal 1031ms + ✓ agent artifacts and gates through the built CLI, a real daemon and the local agent > fails the run when the artifact_exists gate names a file the agent did not write 14623ms + ✓ agent artifacts and gates through the built CLI, a real daemon and the local agent > fails the run with the author reason when a predicate gate returns false, journaling the verdict 13955ms + ✓ review follow-ups > applies a predicate gate on a helper step too, and journals its verdict 14047ms + ✓ review follow-ups > records predicate verdicts on the root run so a resume reuses them instead of re-running the closure 667ms + ✓ tests/budget-preflight.test.ts (25 tests) 13ms + ✓ tests/authored-step-index.test.ts (12 tests) 11ms + ✓ tests/artifact-gates.test.ts (6 tests) 119ms + ✓ tests/human-live.test.ts (3 tests) 5799ms + ✓ f.human against a real daemon > parks with the question, refuses wrong answers, records one, and resumes to success 3362ms + ✓ f.human against a real daemon > a "no" is a value the body branches on: declined, exit 0, no effect 1547ms + ✓ f.human against a real daemon > refuses to answer a run the daemon does not know 889ms + ✓ tests/provider-trigger-contract.test.ts (7 tests) 586ms + ✓ provider trigger contract > fails `flows check` before deployment and passes once the event is real 378ms + ✓ tests/work-package-consumer.test.ts (13 tests) 132ms + ✓ tests/budget-unmetered-live.test.ts (3 tests) 1273ms + ✓ unmetered budget spend through the live kernel > runs an unpriced step under a dollar budget without tripping it, journaling unknown dollars 506ms + ✓ unmetered budget spend through the live kernel > still counts an unpriced step toward a token budget 403ms + ✓ unmetered budget spend through the live kernel > accrues a priced step and stops the run when it crosses the dollar budget 363ms + ✓ tests/helpers-fanout.test.ts (96 tests) 145ms + ✓ tests/spec-parity.test.ts (31 tests) 307ms + ✓ tests/generate-triggers.test.ts (7 tests) 1030ms + ✓ discovers new adapters, preserves exact event names, and prefers adapter-local mappings 327ms + ✓ tests/named-gate-journal.test.ts (5 tests) 1959ms + ✓ a lowered subprocess_gate journals the gate command's streams > persists what the gate printed before a timeout killed it 822ms + ✓ a gate on an agent step > journals the command's streams and surfaces them in the authored failure 834ms + ✓ tests/webhook-hardening.test.ts (11 tests) 54ms + ✓ tests/human-to.test.ts (8 tests) 9ms + ✓ tests/plugin-loader.test.ts (9 tests) 165ms + ✓ tests/pty-sidechannel.test.ts (11 tests) 6628ms + ✓ view attach preserves worker completion and marks only drive 906ms + ✓ drive attach preserves worker completion and marks only drive 451ms + ✓ passthrough attach preserves worker completion and marks only drive 899ms + ✓ none attach preserves worker completion and marks only drive 885ms + ✓ none subscriber lets an unattended CLI read EOF 531ms + ✓ view subscriber lets an unattended CLI read EOF 473ms + ✓ passthrough subscriber lets an unattended CLI read EOF 464ms + ✓ incomplete subscriber lets an unattended CLI read EOF 515ms + ✓ rejects drive after EOF without marking human intervention 819ms + ✓ delivers all drive bytes in order across child stdin backpressure 679ms + ✓ tests/worker-lease.test.ts (7 tests) 10ms + ✓ tests/yaml-helpers.test.ts (33 tests) 63ms + ✓ tests/authored-agent-permissions.test.ts (26 tests) 782ms + ✓ tests/deploy.test.ts (11 tests) 4627ms + ✓ flows deploy file buckets > publishes the full signed layout byte-for-byte and redeploys as a noop 735ms + ✓ flows deploy file buckets > answers --json with one object per outcome 716ms + ✓ flows deploy file buckets > reports a refusal as JSON under --json 343ms + ✓ flows deploy file buckets > refuses a missing local bundle before creating the bucket 349ms + ✓ flows deploy file buckets > refuses an unreachable bucket before copying 352ms + ✓ flows deploy file buckets > refuses an unwritable bucket 342ms + ✓ flows deploy file buckets > refuses local tampering of spec.canonical.json 336ms + ✓ flows deploy file buckets > refuses local tampering of identity.json 342ms + ✓ flows deploy file buckets > refuses asset bundles instead of using daemon-relative files 371ms + ✓ flows deploy file buckets > never labels a corrupt existing deployment as a noop 720ms + ✓ tests/redact.test.ts (35 tests) 8ms + ✓ tests/communication.test.ts (10 tests) 13ms + ✓ tests/typed-output.test.ts (14 tests) 180ms + ✓ tests/budget-attribution.test.ts (5 tests) 6ms + ✓ tests/json-schema-bound.test.ts (71 tests) 2113ms + ✓ JSON Schema termination bound > walks a deep schema with an explicit stack rather than recursion 1724ms + ✓ tests/effect-channel.test.ts (5 tests) 378ms + ✓ tests/mcp-lifecycle.test.ts (4 tests) 13ms + ✓ tests/model-selection.test.ts (10 tests) 15ms + ✓ tests/f-memory.test.ts (7 tests) 955ms + ✓ tests/relayflowd-path.test.ts (10 tests) 6ms + ✓ tests/worker-cli-result-exit.test.ts (5 tests) 32907ms + ✓ a Claude agent step completes on its result, not only on process exit > settles a hung, successful run within the grace and stops its whole tree 31611ms + ✓ a Claude agent step completes on its result, not only on process exit > maps an error result on a hung run to a failed exit 31612ms + ✓ a Claude agent step completes on its result, not only on process exit > leaves a hang before any result to the existing stops 32013ms + ✓ an agent tree does not outlive the process that spawned it > kills the agent group when the run process is terminated by SIGTERM 800ms + ✓ tests/authored-plugin-effect.test.ts (6 tests) 49ms + ✓ tests/yaml-local-agent-live.test.ts (7 tests) 3806ms + ✓ YAML --local-agent through the built CLI and real daemon > runs with the checked step CLI and model and journals done 541ms + ✓ YAML --local-agent through the built CLI and real daemon > runs with the checked named CLI and model and journals done 564ms + ✓ YAML --local-agent through the built CLI and real daemon > runs with the checked flow CLI and model and journals done 557ms + ✓ YAML --local-agent through the built CLI and real daemon > runs with the checked project CLI and model and journals done 576ms + ✓ YAML --local-agent through the built CLI and real daemon > still parks without --local-agent 494ms + ✓ YAML --local-agent through the built CLI and real daemon > reports the agent process failure 575ms + ✓ YAML --local-agent through the built CLI and real daemon > preserves declared workspace surfaces that the local worker cannot pin 498ms + ✓ tests/local-dev-ux.test.ts (8 tests) 18ms + ✓ tests/relay-cli-surface-live.test.ts (3 tests) 467ms + ✓ tests/authored-declined.test.ts (13 tests) 56ms + ✓ tests/resume-failure.test.ts (2 tests) 5ms + ✓ tests/dependency-validation.test.ts (6 tests) 577ms + ✓ dependency validation > accepts a valid 10,000-step reverse chain through every direct public boundary 322ms + ✓ tests/input-binding.test.ts (12 tests) 171ms + ✓ tests/communication-review.test.ts (5 tests) 332ms + ✓ tests/yaml-helper-effect.test.ts (4 tests) 75ms + ✓ tests/deterministic-llm.test.ts (5 tests) 47ms + ✓ tests/scope-preflight.test.ts (6 tests) 7ms + ✓ tests/bin.test.ts (7 tests) 2178ms + ✓ built flows binary > refuses through a symlink to the built artifact 344ms + ✓ built flows binary > refuses through a symlinked directory component 333ms + ✓ built flows binary > classifies a signal-terminated auth probe as probe_failed 449ms + ✓ built flows binary > classifies an unavailable PATH resolver as probe_failed 358ms + ✓ built flows binary > does not describe a present non-executable CLI as missing 344ms + ✓ built flows binary > runs one auth probe for three steps sharing a flow CLI 349ms + ✓ tests/build-gate.test.ts (3 tests) 1046ms + ✓ flows build gates on flows check green (#318) > refuses a flow with an unresolvable named-agent CLI and leaves no artifacts 342ms + ✓ flows build gates on flows check green (#318) > --json emits one CheckReport object on stdout on refusal, exits 2, no artifacts 347ms + ✓ flows build gates on flows check green (#318) > builds the bundle on success (regression: gate must not block valid flows) 356ms + ✓ tests/scope-compiler.test.ts (25 tests) 11ms + ✓ tests/run-from-digest.test.ts (6 tests) 3922ms + ✓ flows run digest input > submits the sealed canonical spec through the normal journal path without checkout 398ms + ✓ flows run digest input > uses a verified cache hit even after the bucket is removed 369ms + ✓ flows run digest input > resolves deploy.bucket from flows.json and honors explicit override 1069ms + ✓ flows run digest input > refuses an unconfigured bucket 705ms + ✓ flows run digest input > refuses tampered spec.canonical.json before creating run data 704ms + ✓ flows run digest input > refuses tampered identity.json before creating run data 676ms + ✓ tests/communication-worker.test.ts (15 tests) 1497ms + ✓ tests/hn-poller.test.ts (6 tests) 7ms + ✓ tests/plugin-add.test.ts (7 tests) 1113ms + ✓ typechecks the augmented verb and rejects unknown namespaces 824ms + ✓ tests/authored-step-failed-exit.test.ts (3 tests) 7ms + ✓ tests/direct-run-failure.test.ts (8 tests) 11ms + ✓ tests/dir-watcher-poller.test.ts (6 tests) 5ms + ✓ tests/model-pricing.test.ts (10 tests) 5ms + ✓ tests/yaml-helper-live.test.ts (1 test) 893ms + ✓ runs compiled YAML helpers through the built CLI and kernel effect journal 892ms + ✓ tests/provider-trigger-executor.test.ts (4 tests) 287ms + ✓ tests/transcript-tail-close.test.ts (2 tests) 858ms + ✓ a stalled transcript-tail close > does not hold the spawn open past its bounded window 498ms + ✓ a stalled tail close beside a transcript that finished > still journals the transcript pointer 360ms + ✓ tests/wrapper-artifacts-cwd.test.ts (2 tests) 65ms + ✓ tests/hello-deterministic.test.ts (5 tests) 15ms + ✓ tests/transcript-exclusion-timeout.test.ts (1 test) 183ms + ✓ tests/cli-adapter.test.ts (4 tests) 5ms + ✓ tests/communication-mixed-resume.test.ts (1 test) 167ms + ✓ tests/work-package-validator.test.ts (7 tests) 5ms + ✓ tests/authored-use-loader.test.ts (5 tests) 819ms + ✓ tests/authored-declined-live.test.ts (1 test) 1508ms + ✓ runs an input guard and resumes its completed declined root without repeated effects 1507ms + ✓ tests/cli-answer.test.ts (15 tests) 8ms + ✓ tests/bundle-preflight.test.ts (4 tests) 791ms + ✓ bundle execution preflight > ignores surrounding cache configuration on a verified cache hit 370ms + ✓ bundle execution preflight > uses the built alias for a nameless flow even in a digest-only cache directory 391ms + ✓ tests/agent-relay-hardening.test.ts (12 tests) 12ms + ✓ tests/classify-outcome.test.ts (2 tests) 2160ms + ✓ classifyOutcome > gives up and reports when a running run never becomes classifiable 2007ms + ✓ tests/communication-preflight.test.ts (13 tests) 33ms + ✓ tests/agent-artifacts.test.ts (6 tests) 13ms + ↓ tests/real-cli-adapters.test.ts (3 tests | 3 skipped) + ✓ tests/memoization.test.ts (57 tests) 51ms + ✓ tests/parse-json-output.test.ts (7 tests) 3ms + ✓ tests/journal-client-completion.test.ts (4 tests) 99ms + ✓ tests/worker-cli-abort.test.ts (2 tests) 2542ms + ✓ stops claude and its process group when lease ownership is lost 1252ms + ✓ stops wrapper.mjs and its process group when lease ownership is lost 1289ms + ✓ tests/communication-environment-preflight.test.ts (6 tests) 3ms + ✓ tests/budget-authored-live.test.ts (2 tests) 158ms + ✓ tests/slack-writeback.test.ts (1 test) 257ms + ✓ tests/authored-surface-authority.test.ts (2 tests) 15ms + ✓ tests/adapters/claude.test.ts (7 tests) 4ms + ✓ tests/worker-cli-cwd.test.ts (2 tests) 351ms + ✓ runAgentCli — cwd propagation (flows#357) > omits cwd when not provided (inherits parent cwd) 346ms + ✓ tests/adapters/codex.test.ts (7 tests) 4ms + ✓ tests/slack-block-kit.test.ts (5 tests) 14ms + ✓ tests/communication-history.test.ts (1 test) 3ms + ✓ tests/adapters/registry.test.ts (4 tests) 4ms + ✓ tests/step-lease.test.ts (36 tests) 66424ms + ✓ f.run leases against the live kernel > enforces 10000 ms for 'sleep 5; printf ok' 5071ms + ✓ f.run leases against the live kernel > enforces 40000 ms for 'sleep 31; printf ok' 31064ms + ✓ f.run leases against the live kernel > enforces 30000 ms for 'sleep 31; printf ok' 30090ms + ✓ tests/authored-declined-report.test.ts (6 tests) 7ms + ✓ tests/communication-refusal.test.ts (1 test) 12ms + ✓ tests/check-command-cwd.test.ts (1 test) 12ms + ✓ tests/communication-lazy.test.ts (1 test) 3ms + ✓ tests/cli-progress-wait.test.ts (2 tests) 3ms + ✓ tests/bundle-transport.test.ts (20 tests) 2338ms + ✓ digest references > accepts and deploys the build output for hello 372ms + ✓ digest references > accepts and deploys the build output for Hello 360ms + ✓ digest references > accepts and deploys the build output for hello.world 378ms + ✓ digest references > accepts and deploys the build output for hello_world 434ms + ✓ digest references > accepts and deploys the build output for 123 431ms + ✓ digest references > accepts and deploys the build output for A_b.c-1 358ms + ✓ tests/run-digest-live.test.ts (1 test) 858ms + ✓ executes a deployed digest on the real kernel after deleting the authoring tree 857ms + ✓ tests/placement.test.ts (54 tests) 16ms + ✓ tests/communication-tools.test.ts (1 test) 80ms + ✓ tests/authored-admission.test.ts (2 tests) 2ms + ✓ tests/memory.test.ts (18 tests) 8ms + ✓ tests/worker-platform.test.ts (1 test) 3ms + ✓ tests/run-digest.test.ts (4 tests) 1391ms + ✓ digest run configuration refusals > reports config_invalid before fetching or starting a run for {invalid json 354ms + ✓ digest run configuration refusals > reports config_invalid before fetching or starting a run for {"deploy":{}} 330ms + ✓ digest run configuration refusals > reports config_invalid before fetching or starting a run for {"deploy":{"bucket":123}} 354ms + ✓ digest run configuration refusals > reports config_invalid before fetching or starting a run for {"deploy":{"bucket":""}} 353ms + ✓ tests/local-agent-live.test.ts (5 tests) 64632ms + ✓ built CLI local agent against a real daemon > dispatches through the wrapper and keeps --json stdout report-shaped 705ms + ✓ built CLI local agent against a real daemon > runs beyond the initial 30-second lease without a second invocation 35717ms + ✓ built CLI local agent against a real daemon > renders actual agent completion in text output 764ms + ✓ built CLI local agent against a real daemon > returns a failed run when the agent process fails 13068ms + ✓ built CLI local agent against a real daemon > refuses a workspace it cannot pin before invoking the agent 14377ms + +⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/authored-node-runtime.test.ts [ tests/authored-node-runtime.test.ts ] +AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality + +Expected: "1.4.0" +Received: "1.3.6" + + ❯ tests/authored-node-runtime.test.ts:18:77 + 16| + 17| beforeAll(() => { + 18| expect(spawnSync(bun, ['--version'], { encoding: 'utf8' }).stdout.tr… + | ^ + 19| expect(existsSync(daemon), 'build the current kernel or set RELAYFLO… + 20| stage = mkdtempSync(join(tmpdir(), 'authored-standalone-build-')); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/31]⎯ + +⎯⎯⎯⎯⎯⎯ Failed Tests 30 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) +AssertionError: expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "output": Object { +- "reasoning": "stub agent runtime — deterministic output for gate-2 clause-2 demo", +- "relevance_score": 5, +- "story_title": "stub", +- }, ++ "output": null, + "verification": Object { +- "gate": "json_schema", +- "verdict": "pass", ++ "gate": "execution", ++ "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:657:36 + 655| && (entry as { step_id?: string }).step_id === 'analyze-story', + 656| ) as { payload: { output: unknown; verification: unknown } } | und… + 657| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 658| output: { + 659| story_title: 'stub', + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields +AssertionError: expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "completionReason": "retries_exhausted", ++ "completionReason": "worker_error", + "output": null, + "verification": Object { +- "gate": "json_schema", ++ "gate": "execution", + "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:752:36 + 750| // its verification record names the json_schema rejection. The re… + 751| // parsed value is nulled before the completion is persisted. + 752| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 753| completionReason: 'retries_exhausted', + 754| output: null, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text +AssertionError: expected null not to be null + ❯ tests/live-kernel.test.ts:823:24 + 821| // here (parseJsonOutput returned null on non-JSON stdout) and + 822| // these assertions would all fail. + 823| expect(output).not.toBeNull(); + | ^ + 824| expect(output.exit_code).toBe(0); + 825| expect(output.stdout_tail).toContain('looked at the story'); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:891:42 + 889| ) as { payload: { output: { story_title: string; reasoning: string… + 890| expect(stepCompleted).toBeDefined(); + 891| expect(stepCompleted!.payload.output.story_title).toBe(`echoed:${s… + | ^ + 892| expect(stepCompleted!.payload.output.reasoning).toContain(String(s… + 893| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) +TypeError: Cannot read properties of null (reading 'env_present') + ❯ tests/live-kernel.test.ts:958:38 + 956| ) as { payload: { output: { env_present: boolean } } } | undefined; + 957| expect(completed).toBeDefined(); + 958| expect(completed!.payload.output.env_present).toBe(false); + | ^ + 959| + 960| delete process.env.RELAYFLOW_WAKE_CONTEXT; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:1194:38 + 1192| expect(completed).toBeDefined(); + 1193| // UNSET, not EMPTY and not the leaked parent value. + 1194| expect(completed!.payload.output.story_title).toBe('model:UNSET'); + | ^ + 1195| + 1196| delete process.env.RELAYFLOW_MODEL; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[7/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +Error: LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ❯ tests/live-kernel.test.ts:1223:15 + 1221| const notice = `LIVE_ANALYZER_UNAVAILABLE: ${readiness.detail}`; + 1222| if (process.env['RELAYFLOWS_ALLOW_ANALYZER_SKIP'] !== '1') { + 1223| throw new Error( + | ^ + 1224| `${notice} — failing because gate-2 acceptance requires the … + 1225| + 'Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is … + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[8/31]⎯ + + FAIL tests/live-kernel.test.ts > a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant +AssertionError: expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +- Expected: +Object { + "lag_ms": 43000, + "schedule_id": "heartbeat-1m", + "scheduled_for_ms": 1764000000000, + "slot": 29400000, +} + ++ Received: +null + + ❯ tests/live-kernel.test.ts:1665:39 + 1663| // The bound: the run reports the grid instant and its own lag, so… + 1664| // backfilled run can tell it is running for a slot from the past. + 1665| expect(completed!.payload.output).toEqual({ + | ^ + 1666| schedule_id: 'heartbeat-1m', + 1667| slot: 29_400_000, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[9/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: c649fe14/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[10/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: nope!/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[11/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an empty batch +AssertionError: expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/needs runIds/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses a batch too large for the edge step lease +AssertionError: expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/exceeds the 8 that fit/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound +AssertionError: promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + ❯ tests/stuck-run-triage.test.ts:62:40 + 60| it('accepts eight ids — the incident batch is inside the bound', asy… + 61| const ids = Array.from({ length: 8 }, (_, i) => `${ID_A.slice(0, -… + 62| await expect(drive({ runIds: ids })).resolves.toBeDefined(); + | ^ + 63| }); + 64| }); + +Caused by: TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + ❯ tests/stuck-run-triage.test.ts:62:18 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[14/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin +AssertionError: expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + +- Expected: +/refusing to send the Cloud bearer token to https:\/\/evil\.example/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[15/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses a non-URL apiUrl +AssertionError: expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/is not a URL/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[16/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > allows an approved origin and uses it in the curl +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:77:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[17/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > defaults to production Cloud +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:82:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[18/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > never publishes a run record the fetch did not produce +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:89:34 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[19/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > names the Worker on every wrangler invocation +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:98:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[20/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:107:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[21/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:115:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[22/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:123:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[23/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > records wrangler's own exit status rather than head's +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:129:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[24/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > parses under both sh and bash +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:137:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[25/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:157:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[26/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > declares read-only permissions on every agent +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:176:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[27/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > tells the forensics agents their evidence is untrusted +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:182:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[28/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file +AssertionError: expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate runIds: c649fe14-0c2e-4e51-9a6a-4f0d1b0f77aa/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[29/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason +AssertionError: expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate workers: w-one/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[30/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > bounds ids x workers, not just ids +AssertionError: expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/24 concurrent tails, over the 16/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[31/31]⎯ + + Test Files 3 failed | 155 passed | 1 skipped (159) + Tests 30 failed | 2393 passed | 17 skipped (2440) + Start at 17:48:56 + Duration 194.32s (transform 2.90s, setup 0ms, collect 35.01s, tests 501.26s, environment 18ms, prepare 5.84s) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/mutation-restored.txt b/evidence/511-named-gate-diagnostics/mutation-restored.txt new file mode 100644 index 00000000..eea77cad --- /dev/null +++ b/evidence/511-named-gate-diagnostics/mutation-restored.txt @@ -0,0 +1,28 @@ +$ git checkout HEAD -- src/named-gate-lowering.ts # restore +$ git hash-object src/named-gate-lowering.ts # after restore +8a4ac50391d95c95693eb20cdbc69869a5d02060 +$ git rev-parse HEAD:packages/sdk/src/named-gate-lowering.ts +8a4ac50391d95c95693eb20cdbc69869a5d02060 +$ git status --porcelain -- src tests tsconfig.tests.json # empty = worktree identical to HEAD +(status exit 0) +$ git diff --stat HEAD -- . + +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ✓ tests/named-gate-diagnostics.test.ts (17 tests) 685ms + ✓ tests/named-gate-journal.test.ts (5 tests) 1929ms + ✓ a lowered subprocess_gate journals the gate command's streams > persists what the gate printed before a timeout killed it 832ms + ✓ a gate on an agent step > journals the command's streams and surfaces them in the authored failure 793ms + + Test Files 2 passed (2) + Tests 22 passed (22) + Start at 17:41:12 + Duration 3.34s (transform 768ms, setup 0ms, collect 1.65s, tests 2.61s, environment 0ms, prepare 75ms) + +EXIT=0 diff --git a/evidence/511-named-gate-diagnostics/mutation-reverted.txt b/evidence/511-named-gate-diagnostics/mutation-reverted.txt new file mode 100644 index 00000000..5dc6a4e7 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/mutation-reverted.txt @@ -0,0 +1,295 @@ +$ git hash-object src/named-gate-lowering.ts # before revert +8a4ac50391d95c95693eb20cdbc69869a5d02060 +$ git rev-parse HEAD:packages/sdk/src/named-gate-lowering.ts +8a4ac50391d95c95693eb20cdbc69869a5d02060 + +$ git checkout e21caad -- src/named-gate-lowering.ts # revert production file to the parent revision +$ git diff --stat HEAD -- src/named-gate-lowering.ts + packages/sdk/src/named-gate-lowering.ts | 40 +++++---------------------------- + 1 file changed, 5 insertions(+), 35 deletions(-) +$ git hash-object src/named-gate-lowering.ts # after revert +479136c28de4f97c5434800577999bbbb1eca59c +$ git rev-parse e21caad:packages/sdk/src/named-gate-lowering.ts +479136c28de4f97c5434800577999bbbb1eca59c + +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ❯ tests/named-gate-diagnostics.test.ts (17 tests | 12 failed) 626ms + × a gate that fails before the command runs says so > names the selection that missed, and does not run the command 33ms + → expected '' to contain 'subprocess_gate' + × a gate that fails before the command runs says so > reports an index into an object and a key into an array as selection failures 23ms + → expected '' to contain 'from_output [0]' + × a gate that fails before the command runs says so > reports a selected value that is not text 34ms + → expected '' to contain 'could not be read as text' + × a gate that fails before the command runs says so > reports a NUL byte in the selected text 23ms + → expected '' to contain 'NUL byte' + × a gate that fails before the command runs says so > keeps the diagnostic to one line so it cannot be mistaken for command output 21ms + → expected '' to contain 'from_output' + × a gate command that never exits normally is not reported as a plain failure > names the signal that killed it 26ms + → expected '' to contain 'SIGKILL' + × a gate command that never exits normally is not reported as a plain failure > names the spawn error and the input size when the child cannot start 26ms + → expected '' to contain 'E2BIG' + × diagnostics never reach the stdout a gate verdict is read from > leaves references_input stdout empty when the selection misses 77ms + → expected '' to contain 'in_output_at ["missing"]' + × diagnostics never reach the stdout a gate verdict is read from > leaves regex_match stdout empty when the selection misses 50ms + → expected '' to contain 'regex_match' + × word_count_bounds reports why its own child failed > reports a nonzero exit with what wc said 45ms + → expected '' to contain 'exited 2' + × word_count_bounds reports why its own child failed > reports output that is not a count 40ms + → expected '' to contain 'not a number' + × word_count_bounds reports why its own child failed > reports a signal rather than an empty exit 1 29ms + → expected '' to contain 'SIGKILL' + ❯ tests/named-gate-journal.test.ts (5 tests | 1 failed) 1808ms + × a lowered subprocess_gate journals the gate command's streams > journals the diagnostic when the gate never reached the command 93ms + → expected '' to contain 'subprocess_gate: from_output ["review…' + ✓ a lowered subprocess_gate journals the gate command's streams > persists what the gate printed before a timeout killed it 823ms + ✓ a gate on an agent step > journals the command's streams and surfaces them in the authored failure 713ms + +⎯⎯⎯⎯⎯⎯ Failed Tests 13 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > names the selection that missed, and does not run the command +AssertionError: expected '' to contain 'subprocess_gate' + +- Expected ++ Received + +- subprocess_gate + + ❯ tests/named-gate-diagnostics.test.ts:117:28 + 115| expect(capture.status).toBe(1); + 116| expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + 117| expect(capture.stderr).toContain('subprocess_gate'); + | ^ + 118| expect(capture.stderr).toContain('from_output ["review","verdict"]… + 119| expect(capture.stderr).toContain('"verdict"'); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > reports an index into an object and a key into an array as selection failures +AssertionError: expected '' to contain 'from_output [0]' + +- Expected ++ Received + +- from_output [0] + + ❯ tests/named-gate-diagnostics.test.ts:124:74 + 122| it('reports an index into an object and a key into an array as selec… + 123| const indexed = gateCommand({ type: 'subprocess_gate', command: se… + 124| expect(runGate(indexed, { output: { '0': 'not an array' } }).stder… + | ^ + 125| + 126| const keyed = gateCommand({ type: 'subprocess_gate', command: sent… + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > reports a selected value that is not text +AssertionError: expected '' to contain 'could not be read as text' + +- Expected ++ Received + +- could not be read as text + + ❯ tests/named-gate-diagnostics.test.ts:138:28 + 136| expect(capture.status).toBe(1); + 137| expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + 138| expect(capture.stderr).toContain('could not be read as text'); + | ^ + 139| }); + 140| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > reports a NUL byte in the selected text +AssertionError: expected '' to contain 'NUL byte' + +- Expected ++ Received + +- NUL byte + + ❯ tests/named-gate-diagnostics.test.ts:147:28 + 145| expect(capture.status).toBe(1); + 146| expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + 147| expect(capture.stderr).toContain('NUL byte'); + | ^ + 148| }); + 149| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > keeps the diagnostic to one line so it cannot be mistaken for command output +AssertionError: expected '' to contain 'from_output' + +- Expected ++ Received + +- from_output + + ❯ tests/named-gate-diagnostics.test.ts:163:28 + 161| const capture = runGate(command, { output: { other: 1 } }); + 162| + 163| expect(capture.stderr).toContain('from_output'); + | ^ + 164| expect(capture.stderr.trim().split('\n')).toHaveLength(1); + 165| }); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate command that never exits normally is not reported as a plain failure > names the signal that killed it +AssertionError: expected '' to contain 'SIGKILL' + +- Expected ++ Received + +- SIGKILL + + ❯ tests/named-gate-diagnostics.test.ts:179:28 + 177| // kernel's pipe; buffering them for a post-wait flush would lose … + 178| expect(capture.stdout).toContain('partial'); + 179| expect(capture.stderr).toContain('SIGKILL'); + | ^ + 180| }); + 181| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate command that never exits normally is not reported as a plain failure > names the spawn error and the input size when the child cannot start +AssertionError: expected '' to contain 'E2BIG' + +- Expected ++ Received + +- E2BIG + + ❯ tests/named-gate-diagnostics.test.ts:208:28 + 206| + 207| expect(capture.status).toBe(1); + 208| expect(capture.stderr).toContain('E2BIG'); + | ^ + 209| expect(capture.stderr).toContain('10 bytes'); + 210| }); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[7/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > diagnostics never reach the stdout a gate verdict is read from > leaves references_input stdout empty when the selection misses +AssertionError: expected '' to contain 'in_output_at ["missing"]' + +- Expected ++ Received + +- in_output_at ["missing"] + + ❯ tests/named-gate-diagnostics.test.ts:220:28 + 218| expect(capture.status).toBe(1); + 219| expect(capture.stdout).toBe(''); + 220| expect(capture.stderr).toContain('in_output_at ["missing"]'); + | ^ + 221| // The receipt `output_contains` looks for must never be forgeable… + 222| // diagnostic that merely names the gate. + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[8/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > diagnostics never reach the stdout a gate verdict is read from > leaves regex_match stdout empty when the selection misses +AssertionError: expected '' to contain 'regex_match' + +- Expected ++ Received + +- regex_match + + ❯ tests/named-gate-diagnostics.test.ts:232:28 + 230| expect(capture.status).toBe(1); + 231| expect(capture.stdout).toBe(''); + 232| expect(capture.stderr).toContain('regex_match'); + | ^ + 233| }); + 234| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[9/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > word_count_bounds reports why its own child failed > reports a nonzero exit with what wc said +AssertionError: expected '' to contain 'exited 2' + +- Expected ++ Received + +- exited 2 + + ❯ tests/named-gate-diagnostics.test.ts:262:28 + 260| expect(capture.status).toBe(1); + 261| expect(capture.stdout).toBe(''); + 262| expect(capture.stderr).toContain('exited 2'); + | ^ + 263| expect(capture.stderr).toContain('wc: read error'); + 264| }); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[10/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > word_count_bounds reports why its own child failed > reports output that is not a count +AssertionError: expected '' to contain 'not a number' + +- Expected ++ Received + +- not a number + + ❯ tests/named-gate-diagnostics.test.ts:272:28 + 270| expect(capture.status).toBe(1); + 271| expect(capture.stdout).toBe(''); + 272| expect(capture.stderr).toContain('not a number'); + | ^ + 273| }); + 274| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[11/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > word_count_bounds reports why its own child failed > reports a signal rather than an empty exit 1 +AssertionError: expected '' to contain 'SIGKILL' + +- Expected ++ Received + +- SIGKILL + + ❯ tests/named-gate-diagnostics.test.ts:280:28 + 278| + 279| expect(capture.status).toBe(1); + 280| expect(capture.stderr).toContain('SIGKILL'); + | ^ + 281| }); + 282| }); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/13]⎯ + + FAIL tests/named-gate-journal.test.ts > a lowered subprocess_gate journals the gate command's streams > journals the diagnostic when the gate never reached the command +AssertionError: expected '' to contain 'subprocess_gate: from_output ["review…' + +- Expected ++ Received + +- subprocess_gate: from_output ["review","verdict"] + + ❯ tests/named-gate-journal.test.ts:103:38 + 101| expect(gate.output?.exit_code).toBe(1); + 102| expect(gate.output?.stdout_tail).not.toContain('THE-COMMAND-RAN'); + 103| expect(gate.output?.stderr_tail).toContain('subprocess_gate: from_… + | ^ + 104| }, 30_000); + 105| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/13]⎯ + + Test Files 2 failed (2) + Tests 13 failed | 9 passed (22) + Start at 17:40:58 + Duration 3.11s (transform 706ms, setup 0ms, collect 1.51s, tests 2.43s, environment 0ms, prepare 77ms) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/restore-identity.txt b/evidence/511-named-gate-diagnostics/restore-identity.txt new file mode 100644 index 00000000..fe30ca42 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/restore-identity.txt @@ -0,0 +1,17 @@ +# Restoring the working tree to HEAD after the in-place baseline. +$ git checkout HEAD -- packages/sdk +$ git status --porcelain -- packages/sdk # empty = tree and index identical to HEAD +$ git diff --stat HEAD -- packages/sdk +$ for f in src/named-gate-lowering.ts tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts tsconfig.tests.json; do git hash-object packages/sdk/$f; git rev-parse HEAD:packages/sdk/$f; done +# src/named-gate-lowering.ts +8a4ac50391d95c95693eb20cdbc69869a5d02060 +8a4ac50391d95c95693eb20cdbc69869a5d02060 +# tests/named-gate-diagnostics.test.ts +b35c60eb24f577a7e8cee998ddf6fb117e9261c5 +b35c60eb24f577a7e8cee998ddf6fb117e9261c5 +# tests/named-gate-journal.test.ts +2a9f695b88b0f3261fd71bcf5f0b94d6564b1b03 +2a9f695b88b0f3261fd71bcf5f0b94d6564b1b03 +# tsconfig.tests.json +78b1d3d637f77a82013c2d64e4c863d14c44652b +78b1d3d637f77a82013c2d64e4c863d14c44652b diff --git a/evidence/511-named-gate-diagnostics/unrelated-failure-cause.txt b/evidence/511-named-gate-diagnostics/unrelated-failure-cause.txt new file mode 100644 index 00000000..995c43a6 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/unrelated-failure-cause.txt @@ -0,0 +1,13 @@ +# Why the 22 stuck-run-triage failures happen in this sandbox: two installed copies +# of @relayflows/surface are reachable, so the test and the workflow under test get +# different module instances and the WeakMap brand check in getFlowDefinition +# (packages/surface/src/flow.ts:137-143) misses. + +$ node -e "const{createRequire}=require(\"node:module\");for(const from of [\"packages/sdk/tests/x.js\",\"workflows/x.js\"]){const r=createRequire(process.cwd()+\"/\"+from);console.log(from,\"->\",r.resolve(\"@relayflows/surface/runtime\"))}" +packages/sdk/tests/x.js -> /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/@relayflows/surface/dist/runtime.js +workflows/x.js -> /home/daytona/.relayflow-v2-supervisor/durable/node_modules/@relayflows/surface/dist/runtime.js + +$ ls -d /home/daytona/.relayflow-v2-supervisor/durable/node_modules/@relayflows/* # the ancestor install, outside the checkout +/home/daytona/.relayflow-v2-supervisor/durable/node_modules/@relayflows/surface +$ ls -d node_modules 2>&1 # the repository root has no node_modules of its own +ls: cannot access 'node_modules': No such file or directory diff --git a/packages/sdk/src/named-gate-lowering.ts b/packages/sdk/src/named-gate-lowering.ts index 479136c2..8a4ac503 100644 --- a/packages/sdk/src/named-gate-lowering.ts +++ b/packages/sdk/src/named-gate-lowering.ts @@ -71,19 +71,33 @@ process.exit(artifacts.includes(${JSON.stringify(gate.path)})?0:1);`)}`; } const path = gate.type === 'subprocess_gate' ? gate.from_output : gate.type === 'word_count_bounds' ? undefined : gate.in_output_at; + const pathKey = gate.type === 'subprocess_gate' ? 'from_output' : 'in_output_at'; // Only compiler-owned code is serialized. Author strings are JSON literals; // upstream output travels exclusively through FLOWS_INPUT, never shell text. + // + // `fail` answers a gate that exits 1 having said nothing. Every exit that is + // NOT the author's own predicate or command verdict goes through it: a + // selection that missed, input the child cannot carry, a child that never + // started, a child killed by a signal. It writes ONE bounded line to fd 2 + // and nothing to fd 1 — `references_input` and `word_count_bounds` verify + // against stdout, so a stray byte there would change a verdict. + // + // `writeSync` rather than `process.stderr.write`: on a pipe the latter is + // asynchronous, and the `process.exit` on the same line would drop the + // diagnostic exactly when it is the only account of the failure there is. const setup = `const cp=require('node:child_process'); +const fail=m=>{require('node:fs').writeSync(2,${JSON.stringify(`${gate.type}: `)}+m.replace(/[\\r\\n]+/g,' ').slice(0,400)+'\\n');process.exit(1);}; const input=JSON.parse(process.env.FLOWS_INPUT); let value=input.output; const path=${JSON.stringify(path ?? null)}; if(path!==null){for(const key of path){ if(value===null||typeof value!=='object'||!Object.hasOwn(value,key)|| -(typeof key==='number'?!Array.isArray(value):Array.isArray(value)))process.exit(1); +(typeof key==='number'?!Array.isArray(value):Array.isArray(value))) +fail(${JSON.stringify(`${pathKey} ${JSON.stringify(path ?? null)} selected nothing: no `)}+JSON.stringify(key)+' at that position in the producer output'); value=value[key]; }}else if(${deterministic})value=value.stdout_tail; const text=typeof value==='string'?value:JSON.stringify(value); -if(typeof text!=='string')process.exit(1); +if(typeof text!=='string')fail('the selected value could not be read as text'); `; let body: string; switch (gate.type) { @@ -93,16 +107,32 @@ if(typeof reference!=='string'||reference.length===0||!text.includes(reference)) process.stdout.write('references_input:pass');`; break; case 'subprocess_gate': - body = `if(text.includes('\\0'))process.exit(1); + // `stdio: 'inherit'` hands the child the gate step's own stdout/stderr, + // which ARE the kernel's capture pipes (relayflowd/src/exec_det.rs:78), + // so the command's output is journaled as it is produced — including + // whatever it wrote before a timeout killed the process group. Buffering + // it for a post-wait flush would lose exactly that. What was missing is + // below: `status` is null for every outcome that is not an exit, and all + // of them used to collapse into an indistinguishable bare `exit 1`. + body = `if(text.includes('\\0'))fail('the selected text contains a NUL byte and cannot be passed to the gate command'); const result=cp.spawnSync('/bin/sh',['-c',${JSON.stringify(gate.command)}],{ env:{...process.env,INPUT:text},stdio:'inherit'}); +if(result.error)fail('could not run the gate command: '+(result.error.code||result.error.message)+' (the selected input was '+Buffer.byteLength(text)+' bytes)'); +if(result.signal)fail('the gate command was terminated by '+result.signal); process.exit(result.status===0?0:1);`; break; case 'word_count_bounds': + // The other gate that spawns a child. Its `wc` stderr was piped and then + // discarded, so a broken or absent `wc` was reported as a word count out + // of bounds. A bounded suffix of what it said travels with the cause. body = `const result=cp.spawnSync('wc',['-w'],{input:text,encoding:'utf8',env:{...process.env,LC_ALL:'C'}}); -if(result.status!==0)process.exit(1); +const noise=(result.stderr||'').trim().slice(-200); +const said=noise===''?'':': '+noise; +if(result.error)fail('could not run wc -w: '+(result.error.code||result.error.message)+said); +if(result.signal)fail('wc -w was terminated by '+result.signal+said); +if(result.status!==0)fail('wc -w exited '+result.status+said); const count=result.stdout.trim(); -if(!/^[0-9]+$/.test(count))process.exit(1); +if(!/^[0-9]+$/.test(count))fail('wc -w printed '+JSON.stringify(count.slice(0,80))+' instead of a word count'+said); process.stdout.write(BigInt(count).toString());`; break; case 'regex_match': diff --git a/packages/sdk/tests/named-gate-diagnostics.test.ts b/packages/sdk/tests/named-gate-diagnostics.test.ts new file mode 100644 index 00000000..b35c60eb --- /dev/null +++ b/packages/sdk/tests/named-gate-diagnostics.test.ts @@ -0,0 +1,282 @@ +import { spawnSync } from 'node:child_process'; +import { chmodSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, describe, expect, it } from 'vitest'; +import { compileSpec } from '../src/compile.js'; +import { lowerNamedGates } from '../src/named-gate-lowering.js'; +import type { NamedDataGate, StepSpec } from '../src/spec.js'; + +/** + * These cases run the REAL lowered command the way the kernel runs it — piped + * stdout and stderr, `FLOWS_INPUT` in the environment (exec_det.rs:72-78) — and + * assert on the captured streams. They establish stream capture and the + * diagnostics; the journal itself is asserted against a live daemon in + * `named-gate-journal.test.ts`, because a captured pipe is not a journal write. + */ + +const directories: string[] = []; +afterEach(() => { for (const directory of directories.splice(0)) rmSync(directory, { recursive: true, force: true }); }); + +function temporaryDirectory(): string { + const directory = mkdtempSync(join(tmpdir(), 'gate-diagnostics-')); + directories.push(directory); + return directory; +} + +/** The command the compiler actually emits for `.gate`. */ +function gateCommand(gate: NamedDataGate, producer: 'deterministic' | 'agent' = 'deterministic'): string { + // `references_input` names a declared input binding, so the producer needs a + // real upstream to bind to; every other gate reads the producer's output. + const referencing = gate.type === 'references_input'; + const source = { + id: 'source', type: 'deterministic', command: 'true', + verification: { type: 'json_schema', schema: { type: 'object', properties: { text: { type: 'string' } } } }, + }; + const step = producer === 'deterministic' + ? { id: 'produce', type: 'deterministic', command: 'true', verification: gate } + : { id: 'produce', type: 'agent', instruction: 'work', cli: 'stub', verification: gate }; + const spec = compileSpec({ + version: '0.1.0', name: 'gated', + steps: (referencing + ? [source, { ...step, input: { reference: { step: 'source', path: ['text'] } } }] + : [step]) as StepSpec[], + }); + const barrier = lowerNamedGates(spec.steps).find(candidate => candidate.id === 'produce.gate'); + const command = (barrier as { command?: string } | undefined)?.command; + if (command === undefined) throw new Error('no gate step was lowered'); + return command; +} + +interface Capture { status: number | null; stdout: string; stderr: string } + +/** + * `input` is the resolved binding the kernel puts in FLOWS_INPUT — the gate's + * own `input` is `{ output: }` (+ `reference` for + * references_input), never the raw envelope. + */ +function runGate(command: string, input: Record, options: { path?: string } = {}): Capture { + const result = spawnSync('/bin/sh', ['-c', command], { + encoding: 'utf8', + env: { + ...process.env, + FLOWS_INPUT: JSON.stringify(input), + ...(options.path === undefined ? {} : { PATH: `${options.path}:${process.env['PATH'] ?? ''}` }), + }, + }); + if (result.error !== undefined) throw result.error; + return { status: result.status, stdout: result.stdout, stderr: result.stderr }; +} + +/** A deterministic producer's journaled envelope. */ +const deterministicEnvelope = (stdout: string) => ({ exit_code: 0, stdout_tail: stdout, stderr_tail: '' }); + +describe('a subprocess_gate keeps the gate command\'s own streams', () => { + const gate = (command: string): NamedDataGate => + ({ type: 'subprocess_gate', command, from_output: ['stdout_tail'] }); + + it('captures stdout and stderr when the command fails', () => { + const command = gateCommand(gate("printf 'GATE_FAILED %s' \"$INPUT\" >&2; printf 'on stdout'; exit 1")); + const capture = runGate(command, { output: deterministicEnvelope('the reviewed text') }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toContain('on stdout'); + expect(capture.stderr).toContain('GATE_FAILED the reviewed text'); + }); + + it('captures stdout and stderr when the command passes', () => { + const command = gateCommand(gate("printf 'GATE_PASSED detail'; printf 'a warning' >&2")); + const capture = runGate(command, { output: deterministicEnvelope('the reviewed text') }); + + expect(capture.status).toBe(0); + expect(capture.stdout).toContain('GATE_PASSED detail'); + expect(capture.stderr).toContain('a warning'); + }); + + it('selects the whole envelope for a non-deterministic producer', () => { + // An agent's envelope has no implicit `stdout_tail` selection, so a gate + // without `from_output` receives the serialized envelope — the shape the + // reported incident's gate was handed. + const command = gateCommand({ type: 'subprocess_gate', command: 'printf %s "$INPUT"; exit 1' }, 'agent'); + const capture = runGate(command, { output: { exit_code: 0, stdout_tail: 'drafted', artifacts: ['a.md'] } }); + + expect(capture.status).toBe(1); + expect(JSON.parse(capture.stdout)).toEqual({ exit_code: 0, stdout_tail: 'drafted', artifacts: ['a.md'] }); + }); +}); + +describe('a gate that fails before the command runs says so', () => { + const sentinel = "printf 'THE-COMMAND-RAN'"; + + it('names the selection that missed, and does not run the command', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['review', 'verdict'] }); + const capture = runGate(command, { output: { review: { note: 'no verdict here' } } }); + + expect(capture.status).toBe(1); + expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + expect(capture.stderr).toContain('subprocess_gate'); + expect(capture.stderr).toContain('from_output ["review","verdict"]'); + expect(capture.stderr).toContain('"verdict"'); + }); + + it('reports an index into an object and a key into an array as selection failures', () => { + const indexed = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: [0] }); + expect(runGate(indexed, { output: { '0': 'not an array' } }).stderr).toContain('from_output [0]'); + + const keyed = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['name'] }); + expect(runGate(keyed, { output: ['an', 'array'] }).stderr).toContain('from_output ["name"]'); + }); + + it('reports a selected value that is not text', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel }); + // A deterministic producer whose envelope has no `stdout_tail` at all: + // `JSON.stringify(undefined)` is `undefined`, not a string. + const capture = runGate(command, { output: { exit_code: 0 } }); + + expect(capture.status).toBe(1); + expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + expect(capture.stderr).toContain('could not be read as text'); + }); + + it('reports a NUL byte in the selected text', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['stdout_tail'] }); + const capture = runGate(command, { output: deterministicEnvelope('before\0after') }); + + expect(capture.status).toBe(1); + expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + expect(capture.stderr).toContain('NUL byte'); + }); + + it('still runs the command for the literal characters backslash and zero', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['stdout_tail'] }); + const capture = runGate(command, { output: deterministicEnvelope('a literal \\0 sequence') }); + + expect(capture.status).toBe(0); + expect(capture.stdout).toContain('THE-COMMAND-RAN'); + expect(capture.stderr).toBe(''); + }); + + it('keeps the diagnostic to one line so it cannot be mistaken for command output', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['a\nb'] }); + const capture = runGate(command, { output: { other: 1 } }); + + expect(capture.stderr).toContain('from_output'); + expect(capture.stderr.trim().split('\n')).toHaveLength(1); + }); +}); + +describe('a gate command that never exits normally is not reported as a plain failure', () => { + it('names the signal that killed it', () => { + const command = gateCommand({ + type: 'subprocess_gate', command: 'printf partial; kill -9 $$', from_output: ['stdout_tail'], + }); + const capture = runGate(command, { output: deterministicEnvelope('text') }); + + expect(capture.status).toBe(1); + // `inherit` means the bytes written before the kill are already in the + // kernel's pipe; buffering them for a post-wait flush would lose them. + expect(capture.stdout).toContain('partial'); + expect(capture.stderr).toContain('SIGKILL'); + }); + + it('names the spawn error and the input size when the child cannot start', () => { + // The real generated program, with `node:child_process` stubbed to the + // E2BIG shape (`status: null`, `error.code`) an oversized environment + // produces. A nonexistent command would exit 127 through /bin/sh and + // never reach this branch. + const directory = temporaryDirectory(); + const preload = join(directory, 'stub-spawn.cjs'); + writeFileSync(preload, `const Module = require('node:module'); +const load = Module._load; +Module._load = function (request, ...rest) { + if (request !== 'node:child_process') return load.call(this, request, ...rest); + return { spawnSync: () => ({ + error: Object.assign(new Error('spawnSync /bin/sh E2BIG'), { code: 'E2BIG' }), + status: null, signal: null, stdout: null, stderr: null, + }) }; +}; +`); + writeFileSync(join(directory, 'node'), `#!/bin/sh +exec ${JSON.stringify(process.execPath)} --require ${JSON.stringify(preload)} "$@" +`); + chmodSync(join(directory, 'node'), 0o755); + + const command = gateCommand({ type: 'subprocess_gate', command: 'true', from_output: ['stdout_tail'] }); + const capture = runGate(command, { output: deterministicEnvelope('0123456789') }, { path: directory }); + + expect(capture.status).toBe(1); + expect(capture.stderr).toContain('E2BIG'); + expect(capture.stderr).toContain('10 bytes'); + }); +}); + +describe('diagnostics never reach the stdout a gate verdict is read from', () => { + it('leaves references_input stdout empty when the selection misses', () => { + const command = gateCommand({ type: 'references_input', input_key: 'reference', in_output_at: ['missing'] }); + const capture = runGate(command, { output: { present: 'x' }, reference: 'a required substring' }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toBe(''); + expect(capture.stderr).toContain('in_output_at ["missing"]'); + // The receipt `output_contains` looks for must never be forgeable by a + // diagnostic that merely names the gate. + expect(capture.stdout + capture.stderr).not.toContain('references_input:pass'); + }); + + it('leaves regex_match stdout empty when the selection misses', () => { + const command = gateCommand({ type: 'regex_match', pattern: 'ok', in_output_at: ['missing'] }); + const capture = runGate(command, { output: { present: 'x' } }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toBe(''); + expect(capture.stderr).toContain('regex_match'); + }); + + it('keeps word_count_bounds stdout a bare decimal on success', () => { + const command = gateCommand({ type: 'word_count_bounds', min: 1, max: 5 }); + const capture = runGate(command, { output: deterministicEnvelope('one two three') }); + + expect(capture.status).toBe(0); + expect(capture.stdout).toBe('3'); + expect(capture.stderr).toBe(''); + }); +}); + +describe('word_count_bounds reports why its own child failed', () => { + /** A `wc` earlier on PATH than the real one, behaving as the test needs. */ + function stubWordCount(script: string): string { + const directory = temporaryDirectory(); + writeFileSync(join(directory, 'wc'), `#!/bin/sh\n${script}\n`); + chmodSync(join(directory, 'wc'), 0o755); + return directory; + } + + const command = () => gateCommand({ type: 'word_count_bounds', min: 1, max: 5 }); + + it('reports a nonzero exit with what wc said', () => { + const path = stubWordCount("printf 'wc: read error' >&2; exit 2"); + const capture = runGate(command(), { output: deterministicEnvelope('one two') }, { path }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toBe(''); + expect(capture.stderr).toContain('exited 2'); + expect(capture.stderr).toContain('wc: read error'); + }); + + it('reports output that is not a count', () => { + const path = stubWordCount("printf 'not a number'"); + const capture = runGate(command(), { output: deterministicEnvelope('one two') }, { path }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toBe(''); + expect(capture.stderr).toContain('not a number'); + }); + + it('reports a signal rather than an empty exit 1', () => { + const path = stubWordCount('kill -9 $$'); + const capture = runGate(command(), { output: deterministicEnvelope('one two') }, { path }); + + expect(capture.status).toBe(1); + expect(capture.stderr).toContain('SIGKILL'); + }); +}); diff --git a/packages/sdk/tests/named-gate-journal.test.ts b/packages/sdk/tests/named-gate-journal.test.ts new file mode 100644 index 00000000..2a9f695b --- /dev/null +++ b/packages/sdk/tests/named-gate-journal.test.ts @@ -0,0 +1,165 @@ +import { flow } from '@relayflows/surface'; +import { afterEach, describe, expect, it } from 'vitest'; +import { executeAuthoredFlow } from '../src/authored-flow-executor.js'; +import { compileSpec, toKernelSpec } from '../src/compile.js'; +import type { JournalClient } from '../src/journal-client.js'; +import { attachLocalAgent } from '../src/local-agent.js'; +import type { FlowSpec } from '../src/spec.js'; +import { chainFixture } from './flow-chain-fixture.js'; + +/** + * The acceptance the ticket asks for: what a lowered named gate leaves in the + * JOURNAL, read back from a real daemon. `named-gate-diagnostics.test.ts` + * proves the generated program writes the bytes; only these cases prove they + * are persisted as the gate step's `output.stdout_tail` / `output.stderr_tail` + * and survive a failed completion. + */ + +const cleanup: Array<() => Promise> = []; +afterEach(async () => { for (const close of cleanup.splice(0).reverse()) await close(); }); + +interface Envelope { exit_code?: number; stdout_tail?: string; stderr_tail?: string } +interface Completion { completionReason: string; output: Envelope | null; verification?: { verdict?: string } } + +/** The `step.completed` the kernel appended for one step, once it exists. */ +async function completionOf( + journal: JournalClient, runId: string, stepId: string, timeoutMs = 20_000, +): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + const { entries } = await journal.journalRead(runId, 1, 1000); + const found = (entries as Array<{ entry_type: string; step_id?: string; payload: Completion }>) + .find(entry => entry.entry_type === 'step.completed' && entry.step_id === stepId); + if (found !== undefined) return found.payload; + await new Promise(resolve => setTimeout(resolve, 25)); + } + throw new Error(`step.completed for "${stepId}" never arrived in ${runId}`); +} + +function gatedSpec(command: string, name: string): FlowSpec { + return compileSpec({ + version: '0.1.0', name, + steps: [{ + id: 'produce', type: 'deterministic', command: "printf 'the reviewed text'", + verification: { type: 'subprocess_gate', command, from_output: ['stdout_tail'] }, + }], + }); +} + +describe('a lowered subprocess_gate journals the gate command\'s streams', () => { + it('keeps both tails on a FAILING gate, the case the operator has nothing else for', async () => { + const fixture = chainFixture(); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + + const started = await journal.runStart(toKernelSpec(gatedSpec( + "printf 'GATE_FAILED tree moved under me'>&2; printf 'GATE_STDOUT %s' \"$INPUT\"; exit 1", + 'gate-fails', + ))); + const gate = await completionOf(journal, started.run_id, 'produce.gate'); + + expect(gate.completionReason).toBe('retries_exhausted'); + expect(gate.output?.exit_code).toBe(1); + expect(gate.output?.stderr_tail).toContain('GATE_FAILED tree moved under me'); + expect(gate.output?.stdout_tail).toContain('GATE_STDOUT the reviewed text'); + }, 30_000); + + it('keeps both tails on a PASSING gate, so a gate that passed for the wrong reason is inspectable', async () => { + const fixture = chainFixture(); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + + const started = await journal.runStart(toKernelSpec(gatedSpec( + "printf 'GATE_PASSED %s' \"$INPUT\"; printf 'a note nobody reads on green'>&2", + 'gate-passes', + ))); + const gate = await completionOf(journal, started.run_id, 'produce.gate'); + + expect(gate.completionReason).toBe('success'); + expect(gate.output?.exit_code).toBe(0); + expect(gate.output?.stdout_tail).toContain('GATE_PASSED the reviewed text'); + expect(gate.output?.stderr_tail).toContain('a note nobody reads on green'); + }, 30_000); + + it('journals the diagnostic when the gate never reached the command', async () => { + const fixture = chainFixture(); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + + const started = await journal.runStart(toKernelSpec(compileSpec({ + version: '0.1.0', name: 'gate-selection-missed', + steps: [{ + id: 'produce', type: 'deterministic', command: "printf 'the reviewed text'", + verification: { + type: 'subprocess_gate', command: "printf 'THE-COMMAND-RAN'", + from_output: ['review', 'verdict'], + }, + }], + }))); + const gate = await completionOf(journal, started.run_id, 'produce.gate'); + + expect(gate.output?.exit_code).toBe(1); + expect(gate.output?.stdout_tail).not.toContain('THE-COMMAND-RAN'); + expect(gate.output?.stderr_tail).toContain('subprocess_gate: from_output ["review","verdict"]'); + }, 30_000); + + it('persists what the gate printed before a timeout killed it', async () => { + const fixture = chainFixture(); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + + // A timeout is not propagated from the producer to the barrier, so it is + // set on the compiled gate step directly. The point is the kernel's: + // `inherit` puts the command's bytes in the capture pipe as they are + // written, so SIGKILL to the process group cannot erase them. + const spec = toKernelSpec(gatedSpec("printf 'GATE_PARTIAL before the wait'; sleep 30", 'gate-times-out')); + const barrier = spec.steps.find(step => step.id === 'produce.gate'); + if (barrier?.type !== 'deterministic') throw new Error('the lowered gate must be a deterministic step'); + barrier.timeout_ms = 750; + + const started = await journal.runStart(spec); + const gate = await completionOf(journal, started.run_id, 'produce.gate'); + + expect(gate.completionReason).toMatch(/timeout|retries_exhausted/u); + expect(gate.output?.stdout_tail).toContain('GATE_PARTIAL before the wait'); + }, 30_000); +}); + +describe('a gate on an agent step', () => { + it('journals the command\'s streams and surfaces them in the authored failure', async () => { + // The reported incident's shape: the gate follows an agent, so the + // envelope it selects from is the worker's, not a deterministic step's. + const fixture = chainFixture(JSON.stringify({ summary: 'drafted the section' })); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + const agent = await attachLocalAgent(journal); + cleanup.push(() => agent.close()); + + const handle = flow('agent-then-gate', async (f) => { + await f.agent('draft', { task: 'Draft the section.' }) + .gate({ + type: 'subprocess_gate', + command: "printf 'GATE_FAILED missing signoff'>&2; printf 'GATE_SAW %s' \"$INPUT\"; exit 1", + }); + f.done('success'); + }); + + const failure = await executeAuthoredFlow(handle, journal, undefined, { + flowPath: fixture.flowPath, localAgentStream: agent.stream, dataDir: fixture.data, + }).then(() => undefined, (error: Error & { runId?: string }) => error); + + expect(failure, 'the failing gate must fail the flow').toBeDefined(); + // The failure report is where an operator meets this first: it must carry + // the command's own account, not just "a gate failed". + expect(failure!.message).toContain('GATE_FAILED missing signoff'); + + const gate = await completionOf(journal, failure!.runId!, 'agent-1.gate'); + expect(gate.output?.exit_code).toBe(1); + expect(gate.output?.stderr_tail).toContain('GATE_FAILED missing signoff'); + // No `from_output`: a non-deterministic producer's envelope is selected + // whole, so the command is handed the serialized envelope rather than a + // `stdout_tail` it does not have. + expect(JSON.parse(gate.output!.stdout_tail!.replace('GATE_SAW ', ''))) + .toEqual({ summary: 'drafted the section' }); + }, 60_000); +}); diff --git a/packages/sdk/tsconfig.tests.json b/packages/sdk/tsconfig.tests.json index cc9298fa..78b1d3d6 100644 --- a/packages/sdk/tsconfig.tests.json +++ b/packages/sdk/tsconfig.tests.json @@ -33,6 +33,8 @@ "tests/direct-input.test.ts", "tests/fixtures/needs-human.flow.ts", "tests/named-gates.test.ts", + "tests/named-gate-diagnostics.test.ts", + "tests/named-gate-journal.test.ts", "tests/build-gate.test.ts", "tests/scope-preflight.test.ts", "tests/worker-cli-cwd.test.ts", diff --git a/summary.md b/summary.md new file mode 100644 index 00000000..b7f7621e --- /dev/null +++ b/summary.md @@ -0,0 +1,292 @@ +# A failing named gate now says why (#511) + +**Correction:** the ticket implemented here is **#511** ("A failing +`subprocess_gate` journals empty stdout/stderr"). The first commit on this +branch, `f308f95`, and the PR title and body cite **#507**, which is a +different open issue (`f.gitlab` is comment-only while `f.github` has full +writeback). That commit is already pushed and is not amended here; the number +is corrected in this report and in `evidence/511-named-gate-diagnostics/`. The +PR body needs the same correction, which is a remote write this change does not +make. + +## What the ticket reported, and what the evidence actually shows + +The ticket proposed replacing `stdio: 'inherit'` in `named-gate-lowering.ts` +with `spawnSync` buffering, on the theory that the daemon captures the gate +command's stdio nowhere. + +**That is not what the code does, and the replacement would make things +worse.** `kernel/relayflowd/src/exec_det.rs:78` creates `Stdio::piped()` for +the deterministic step's stdout and stderr and drains both on reader threads. +The lowered gate *is* a deterministic step, so `inherit` hands the author's +command those very pipes. Its bytes reach the journal as they are written. + +This is not an argument from reading alone. The new +`tests/named-gate-journal.test.ts` starts a real `relayflowd`, runs a lowered +`subprocess_gate`, and reads `step.completed.payload.output` back for the +generated `produce.gate` step. Both tails are there, on a failing gate and on +a passing one. Those two cases **passed against unmodified production code**. +The ticket's stated root cause does not reproduce here, and the reported +macOS incident has not been replayed; nothing below claims it has. + +Buffering would also have cost something real: the kernel SIGKILLs the whole +process group on timeout, so output held for a post-wait flush is destroyed +exactly when it is the only account of the failure. `stdio: 'inherit'` is kept +and now has a regression test that pins it — a gate printing a marker and then +sleeping past a 750 ms timeout still journals the marker. + +## The defect that is real: exits that say nothing + +The gate program had five bare `process.exit(1)` sites, and `word_count_bounds` +four more. Each produced precisely the shape the ticket describes — `exit_code: +1`, empty `stdout_tail`, empty `stderr_tail` — but for reasons that have +nothing to do with the author's command: + +- `from_output` / `in_output_at` selected a path the producer output does not + have (the command never ran at all); +- the selected value could not be read as text; +- the text contained a NUL byte; +- `spawnSync` returned an `error` and never started the child (this is where + an `E2BIG` would land); +- the child was killed by a signal, so `status` is `null`. + +A `references_input` gate whose binding has drifted, and a `subprocess_gate` +whose producer changed shape, both report as "the gate failed" with nothing +attached. That is the undiagnosable failure, and it is now fixed. + +## The change + +`packages/sdk/src/named-gate-lowering.ts` only. + +A `fail(message)` helper in the shared preamble writes **one bounded line to +fd 2** and exits 1. Every non-verdict exit routes through it. Specifics: + +- **`writeSync(2, ...)`, not `process.stderr.write`.** On a pipe the latter is + asynchronous and the `process.exit` on the same line would drop the + diagnostic — losing the message precisely when it is the only evidence. +- **stderr only, never stdout.** `references_input` verifies via + `output_contains` on stdout and `word_count_bounds` via an anchored decimal + pattern, so one stray byte on fd 1 would change a verdict. Three tests assert + stdout stays empty (or stays the bare count) when a diagnostic fires. +- **Bounded and single-line.** 400 characters, with `\r\n` collapsed to spaces, + so an author-controlled label cannot forge extra log lines. +- **No input, environment, or raw error object is dumped.** Spawn failures + carry the error *code* and the input's byte count, which is what makes an + argument/environment-size failure diagnosable without printing the payload. + +`word_count_bounds` additionally stops discarding its `wc` stderr: `error`, +`signal`, nonzero status and malformed output each get their own cause with a +bounded 200-byte suffix of what `wc` said. Previously a missing or broken `wc` +was indistinguishable from a word count out of bounds. + +Deliberately **unchanged**, per the reviewed plan: `stdio: 'inherit'`, the +`FLOWS_INPUT` transport, exit normalization, verification predicates, the +journal schema, retry policy, and the NUL guard's two-layer escaping. Ordinary +predicate mismatches in `references_input` / `regex_match` / `artifact_exists` +keep their bare `exit 1` — a predicate that simply did not match is not a +capture failure and needs no new output. + +## Tests + +**`tests/named-gate-diagnostics.test.ts`** (17 cases) runs the *actual command +the compiler emits* — obtained through `compileSpec` + `lowerNamedGates`, not a +copied helper — under piped stdio. It covers stream capture on pass and fail, +each selection failure with a `THE-COMMAND-RAN` sentinel asserted **absent**, +NUL rejection (with a literal backslash-zero case proving legitimate input +still runs), SIGKILL, stdout non-contamination for every affected gate, and the +`wc` failure modes via a `PATH`-ordered shim. + +The spawn-error case deserves a note. An oversized outer `FLOWS_INPUT` would +stop the *test's own* gate process from starting and prove nothing about the +inner spawn; a nonexistent command starts `/bin/sh` fine and exits 127. So the +test puts a `node` shim first on `PATH` that execs the real interpreter with +`--require`, and the preload hooks `Module._load` to return a +`node:child_process` whose `spawnSync` reports `E2BIG` with a null status. The +real serialized program runs against a stubbed syscall. No production injection +hook was added. + +**`tests/named-gate-journal.test.ts`** (5 cases) is the acceptance evidence: a +live daemon, assertions on the persisted `step.completed` output of the +generated gate step. It covers a failing gate, a passing gate, a selection +diagnostic, partial tails surviving a timeout, and a gate on an **agent** step +— the incident's shape, where the envelope is selected whole rather than as +`stdout_tail`. That case also asserts the failing gate's stderr reaches the +authored failure report's message. + +## Acceptance + +| Ticket requirement | Status | +| --- | --- | +| Failing gate journals both tails | Met — `named-gate-journal.test.ts`, live daemon. Already true before this change; now pinned. | +| Passing gate journals both tails | Met — same file. Already true before; now pinned. | +| Failure report shows the cause | Met — agent-gate case asserts the marker in the authored failure message. | +| Test asserts journal capture of both streams | Met — committed assertions on persisted `output`. | +| Other lowered gates checked | Met — `word_count_bounds` was the other child-spawning gate and was blind to its own child's failure; fixed. `artifact_exists`, `regex_match`, `references_input` spawn nothing and need no change. | +| **`flows logs ` shows the failure** | **PENDING — not closed by this PR.** | + +### Why `flows logs` is left open + +`flows logs` is Cloud-only. `packages/sdk/src/cloud-read.ts:504` issues `GET +/api/v1/workflows/runs//logs`; there is no local-journal fallback, and +this repository contains **no producer** for that log — no write side of the +route exists here. `docs/CLOUD.md:449-452` states it directly: "what these +routes serve is Cloud's own record of the run, not the kernel journal." + +A test could mock a log body containing these markers, but that would prove +only that the renderer prints what it is given — it could not show the gate's +stderr was ever published. The missing integration is the **Cloud runner / +harness that writes runner logs**, which lives outside this repository. I have +not substituted `flows replay` (local-journal only) or added a new local logs +command; both would be scope the ticket did not ask for. + +**Tracking, stated precisely.** The work item is: *the Cloud runner (or the +harness that wraps it) must publish a deterministic step's `stdout_tail` and +`stderr_tail` into the body served by `GET +/api/v1/workflows/runs//logs`.* Closing it requires end-to-end evidence +from that producer — a real Cloud run whose lowered gate fails, and the +`flows logs ` output showing the gate's stderr — which cannot be +produced from this repository, because no code here writes that route. +Until that evidence exists, **#511 is not fully closed by this PR**; merging +it leaves that one acceptance bullet open. No follow-up issue was filed: +filing one is a remote write, and this change makes none. A maintainer should +open it against the Cloud repository and link it from #511. + +## Validation + +Every run below is in `evidence/511-named-gate-diagnostics/`, as the captured +output of the command that heads the file, ending in its exit status. The +earlier revision of this report quoted excerpts without the transcripts, the +full daemon path, or a baseline invocation; that is what these runs replace. +The runs are re-executions, not the earlier session's logs. + +All commands run from `packages/sdk` with the same explicit environment — +`test:prep` exports `RELAYFLOWD_BIN` inside a subshell that never reaches +vitest, and the kernel target lives outside the tree because `ops/cargo.sh` +redirects `CARGO_TARGET_DIR`: + +```sh +RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +``` + +### The required package command — [head-package-test.txt](evidence/511-named-gate-diagnostics/head-package-test.txt) + +```sh +RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm test +``` + +Kernel build, typecheck, build, test typecheck, vitest. Exit 1: + +``` + ✓ tests/named-gate-diagnostics.test.ts (17 tests) 532ms + ✓ tests/named-gate-journal.test.ts (5 tests) 1959ms + Test Files 3 failed | 155 passed | 1 skipped (159) + Tests 30 failed | 2393 passed | 17 skipped (2440) +``` + +**The package is not green in this sandbox.** The transcript is complete — +every failure's assertion and stack is in it, not summarized. + +### Mutation verification — [mutation-reverted.txt](evidence/511-named-gate-diagnostics/mutation-reverted.txt), [mutation-restored.txt](evidence/511-named-gate-diagnostics/mutation-restored.txt) + +Red half: revert the only production file to the parent revision, rebuild, +re-run both new files. + +```sh +git checkout e21caad -- src/named-gate-lowering.ts +git hash-object src/named-gate-lowering.ts # 479136c2… = e21caad's blob +npm run build && npx vitest run tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts +``` + +``` + ❯ tests/named-gate-diagnostics.test.ts (17 tests | 12 failed) 626ms + ❯ tests/named-gate-journal.test.ts (5 tests | 1 failed) 1808ms + Test Files 2 failed (2) + Tests 13 failed | 9 passed (22) +EXIT=1 +``` + +The 9 that pass are the honest baseline: stream capture already worked before +this change. Green half, with the restore identity checked rather than +asserted: + +```sh +git checkout HEAD -- src/named-gate-lowering.ts +git hash-object src/named-gate-lowering.ts # 8a4ac50391d95c95693eb20cdbc69869a5d02060 +git rev-parse HEAD:packages/sdk/src/named-gate-lowering.ts + # 8a4ac50391d95c95693eb20cdbc69869a5d02060 +git status --porcelain -- src tests tsconfig.tests.json # empty +npm run build && npx vitest run tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts +``` + +``` + ✓ tests/named-gate-diagnostics.test.ts (17 tests) 685ms + ✓ tests/named-gate-journal.test.ts (5 tests) 1929ms + Test Files 2 passed (2) + Tests 22 passed (22) +EXIT=0 +``` + +Equal hashes and an empty `git status` are what makes "byte-for-byte" a +checkable statement rather than a claim. + +### The 30 failures, with a baseline — [head-isolated-three-files.txt](evidence/511-named-gate-diagnostics/head-isolated-three-files.txt), [baseline-in-place-e21caad.txt](evidence/511-named-gate-diagnostics/baseline-in-place-e21caad.txt) + +The three failing files, run alone at head and then again with `packages/sdk` +reverted to `e21caad` **in the same working directory**, so the revision is the +only variable: + +```sh +# head (f308f95) +npx vitest run tests/live-kernel.test.ts tests/stuck-run-triage.test.ts tests/authored-node-runtime.test.ts +# Test Files 3 failed (3) +# Tests 30 failed | 23 passed | 14 skipped (67) + +# baseline (e21caad), same directory: +git checkout e21caad -- packages/sdk +rm packages/sdk/tests/named-gate-diagnostics.test.ts packages/sdk/tests/named-gate-journal.test.ts +git diff --stat e21caad -- packages/sdk # empty +npm run build && npx vitest run tests/live-kernel.test.ts tests/stuck-run-triage.test.ts tests/authored-node-runtime.test.ts +# Test Files 3 failed (3) +# Tests 30 failed | 23 passed | 14 skipped (67) +``` + +Same counts, same cases, same assertion line numbers, at both revisions. That +is the evidence for "pre-existing"; the tree was then restored and the identity +re-checked ([restore-identity.txt](evidence/511-named-gate-diagnostics/restore-identity.txt): +all four changed blobs hash-equal to `HEAD`, `git status --porcelain` empty). + +Why each fails here: + +- `tests/stuck-run-triage.test.ts` — 22 cases throwing "expected an + `@relayflows/surface` flow handle". Two installs of that package are + reachable from this checkout, and node resolves the test and the workflow + under test to different ones, so the handle is absent from the `WeakMap` the + test's copy of `getFlowDefinition` consults + (`packages/surface/src/flow.ts:137-143`). Literal resolution output is in + [unrelated-failure-cause.txt](evidence/511-named-gate-diagnostics/unrelated-failure-cause.txt): + the test resolves inside the checkout, the workflow resolves to + `/home/daytona/.relayflow-v2-supervisor/durable/node_modules`, an ancestor of + it. A sandbox layout defect. +- `tests/live-kernel.test.ts` — 8 cases needing agent CLIs unavailable here. +- `tests/authored-node-runtime.test.ts` — suite-level failure at line 18, + `expected '1.3.6' to be '1.4.0'`: this sandbox's bun is older than the + version the test pins, so all 14 cases skip. This is the file that exercises + `word_count_bounds` end-to-end, so it was checked first and specifically; it + does not run here for want of a prerequisite. + +A third run, [baseline-e21caad.txt](evidence/511-named-gate-diagnostics/baseline-e21caad.txt), +ran the same three files at `e21caad` in a `/tmp` worktree and saw only 1 +failure. It is reported but **not** used as the baseline: `/tmp` has no +ancestor `@relayflows` install, so it changes the resolution above and does not +hold the environment fixed. + +### Not run + +The Rust suite: no kernel code changed +(`git diff --stat e21caad HEAD -- kernel` is empty). Any Cloud runner-log +publication: no producer exists here, as recorded above. The reported macOS +incident: not replayed, on Linux or anywhere. + +`tsconfig.tests.json` gains the two new files so `typecheck:tests` covers them; +it passes inside the `npm test` transcript. No file under `docs/evidence` and +no generated file was edited.