feat(ENG-CUDAGRAPH-BREAK): W4 — the persistent device input path becomes a seam capability, and it does not close the decline (#1307, #1163, #1305) by localai-bot · Pull Request #1310 · mudler/vllm.cpp

localai-bot · 2026-08-19T07:01:53Z

W4 of ENG-CUDAGRAPH-BREAK makes the persistent device input path a seam
capability, migrates the two Qwen3.5 decode drivers onto the break-point capture
seam, and settles the async decline and #1305 with evidence rather than with a
stage assignment.

StepDevInputs appeared on 41 lines of src/vllm/model_executor/models/qwen3_5.cpp
and ZERO lines of qwen3_moe.cpp, qwen3.cpp, deepseek_v2.cpp, voxtral.cpp
and qwen3_dflash.cpp. That asymmetry is measured damage, not tidiness: the four
drivers without it replay against HOST vectors, and qwen3.cpp:1106 DECLINES its
decode graph outright whenever the asynchronous device-token mirror is live, on
its own battery — depth-1 graph ON PASS 78/78, depth-2 graph OFF PASS 82/82,
depth-2 graph ON FAIL, slots 1-3 degenerate (#323, #1179). A shipped model lost
graphed decode to a capability written once and unavailable to it.

Coverage and correctness, never speed. No throughput is claimed and none was
measured. GB10 measured prefill idle between launches at 3.8% with GPU-busy above
96%, and the 27B prefill gap at 92.5% non-GEMM glue GPU work; decode is already
captured and already banked its launch-overhead win.

What lands

vt::PersistentStepInput — one capture-stable per-step device input. It
BINDS a destination the driver owns and refreshes it IN PLACE from either a
pinned host staging block or a DEVICE source. What it owns is the part every
driver re-derived: the address-stability rule as a REFUSAL rather than a comment,
the pinned staging block that makes the upload a true asynchronous DMA, and the
refreshing ARM as an observable (last_source(), vt::StepInputStats) rather
than an inference from which line the driver happened to call. What it
deliberately does NOT own is the device allocation — Qwen3_5DecodeGraph draws
its retained inputs from a DEDICATED DevicePool so they never pop a block the
captured forward's own scratch then needs, and a seam that took that over would
silently move nine drivers onto one pool.

The arm has to be observable, and W3 is why. That stage shipped a
kFull/kPiecewise distinction no gate could see, and flipping one token left a
whole driver gate GREEN at 226/226. The host and device arms here are the same
shape of difference: both leave the destination holding bytes, both leave every
pointer unmoved, and only one is correct while the mirror is live.

Both Qwen3.5 drivers open a vt::GraphCaptureScope over a per-slot
vt::BreakableGraph in kFull and replay through BreakableGraph::Replay, and
their PinnedStepInputs/StageStepInputs staging runs through the capability.
Six of the nine drivers are on the seam, and
grep -rn 'std::getenv("VLLM_CPP_CUDAGRAPH")' src/ returns exactly ONE line,
src/vt/breakable_graph.cpp:61 — one switch that turns capture off, at last.

The finding that changes this row's own story

This spec, the engine-matrix row and qwen3.cpp's own comment all said the fix
the decline names already existed as StepDevInputs. It does not.
StepDevInputs has NO token-id member. Its pinned sibling
PinnedStepInputs::token_ids was allocated at capture, filled every step, zeroed
by the poison hook, and never uploaded and never read. The embed runs OUTSIDE
the captured region from the HOST vector (EmbedInto) in every batched driver,
so the decode graph carries no token ids to the device in ANY driver —
including the one this row cited as having the capability. The dead block is
removed.

So making the persistent device input path a seam capability was never, by
itself, going to close the decline: the input an asynchronous mirror patches is
precisely the one the path does not carry. What lands is the capability that
makes writing that destination a per-driver two-liner instead of a tenth
hand-rolled copy, and RefreshFromDevice is the arm it needs.

The decline decision: it STANDS, and `qwen3.cpp` is untouched

Two grounds, not one.

The device token-id destination the fix needs does not exist anywhere, per
above.
The decline's recorded CAUSE does not survive contact with the tree.
338cbbfd1 (Mistral / InternLM2: concurrent async requests past slot 0 produce degenerate repetition #323) records "the registry-level DeviceTokenIdsScope
(60e71a0e) did not close it: this path returns BEFORE the eager forward ever
runs". At that commit's own parent the scope was constructed at
qwen3_dense.cpp:96 BEFORE DenseDecodeGraphForward at :103, and
EmbedInto consumed it through ApplyDeviceTokenIdsOverride on all three of
the driver's arms (qwen3.cpp:610,621,644 @ 338cbbfd1^). The override WAS
live on the graph path. The measured failure is real; the mechanism recorded
beside it is not the one that produced it, and nobody has since identified
what did.

A mitigation whose failure mode is unexplained is not retired by a refactor that
addresses an explanation nobody has confirmed.

The battery was NOT run, stated plainly rather than implied.
tests/parity/test_qwen3_dense_async_serving.cpp needs a GPU and the real
Qwen3-0.6B/4B snapshots, and its header restricts it to dgx.casa. dgx:gpu0
was held by another session's job for this branch's whole window; the lease
obtained was thor:gpu0, which carries no such checkpoint. The owed work is now
TWO runs and not one — the battery as it stands, and the battery with the decline
deleted, because only the second can fail.

#1305's disposition: it STAYS OPEN, and W4 did not add a decline either. W3
declined to trade a shipped default-ON capability for an unmeasured hazard; W4
adds the finding that makes the trade worse. The hazard #1305 describes is the
SAME one qwen3.cpp mitigates, and that mitigation's recorded cause is falsified
and its real cause unidentified. Copying a mitigation whose mechanism nobody can
name into three more registrations spreads an unexplained behaviour rather than
closing a defect.

Gates

G1, on a leased GPU. thor:gpu0, NVIDIA Thor sm_110, driver 595.78, nvcc
13.0.88, source 4ea38eccb, CUDA-ON build for arch 110, 32 .cu.o objects, the
binary resolving libcudart.so.13 and libcublasLt.so.13. 5 cases, 2066
assertions, 0 failed, exit 0 — the assertion count carries the claim, because
with no CUDA backend the same file prints Status: SUCCESS! over assertions: 0.

G1 Qwen3MoeDecodeGraph     on CUDA: 5 steps x 100 logits, 0 differing, 4 replays
G1 VoxtralDecodeGraph      on CUDA: 5 steps x 100 logits, 0 differing, 4 replays
G1 DeepseekV2DecodeGraph   on CUDA: 5 steps x 100 logits, 0 differing, 4 replays
G1 Qwen3_5DecodeGraph      on CUDA: 5 steps x  40 logits, 0 differing, 4 replays
G1 Qwen3_5DenseDecodeGraph on CUDA: 5 steps x  40 logits, 0 differing, 4 replays

W2's Qwen3DenseDecodeGraph is still covered by argument rather than by
measurement, and ## Gates G1 says so.

Red first, everywhere.

gate	red	green
`test_persistent_step_input` (vs a stub with the API and no guarantees)	9 cases, 0 passed, 59 assertions, 32 failed, exit 1	9/9, 59/59
`test_persistent_step_input` zero-byte case (review repair, vs the unrepaired source)	10 cases, 9 passed, 66 assertions, 6 failed, exit 1	10/10, 66/66
`test_qwen3_5_decode_graph_seam` (before the migration)	3 cases, 1 failed, 62 assertions, 5 failed, exit 1	7/7, 129/129

Negative mutations, each with its diff size, its compile status and its exit
status, each detected, each restored and verified by sha256:

mutation	changed	compiled	result
delete the address-stability refusal	5 lines	clean	reds only the capacity case (8/9, 54/59)
null device source becomes a silent no-op	6 lines	clean	reds only the null case (8/9, 58/59)
the host arm stops going through staging	2 lines	clean	reds only the stale-versus-fresh case (8/9, 57/59)
restore the pre-W4 `qwen3_5.cpp` wholesale (G2)	296 lines	clean	reds BOTH drivers' G2 cases, 10 of 124
the MoE driver replays the backend directly (G2, per driver)	7 lines	clean	reds ONLY the MoE G2 case
the MoE driver's `kFull` flipped to `kPiecewise`	3 lines	clean	reds ONLY its mode counters
delete the `StageStepInputs` production call site (reachability)	5 lines	clean	reds ONLY the reachability case

Every one of them left test_qwen3_decode_graph_seam 231/231,
test_qwen3_moe_decode_graph_seam 228/228 and test_breakable_graph 229/229
green, so each gate is scoped to what it names.

Reachability, which the capability could easily have failed. A unit test that
constructs the type by hand proves the class works and never that anything
reaches it. A case enters through Qwen3_5DecodeGraph::Step and asserts
vt::GetStepInputStats(): binds=6 after the capture step, host_refreshes >= 5
after the replay step. Deleting the call site reds that case ALONE while
test_persistent_step_input stays 59/59 green — the difference itself. Two
limits stated rather than hidden: the case sets VT_ASYNC_EXECUTOR=1, because
the persistent device input path sits behind that lever (default OFF) plus the
speculative-decode arm, and that lever also turns on the 2-slot parity ring, so
slot 0 captures on step THREE.

A harness was lying and is fixed rather than worked around.
RecordingCaptureBackend::Copy performed its memcpy immediately even mid-capture,
so a capture appeared to have already moved bytes a real cudaMemcpyAsync would
only move on replay. It now files the copy against the open segment, capturing
the two ADDRESSES and not the bytes — a harness that snapshotted the bytes would
make the fresh and stale arms indistinguishable. test_breakable_graph is
unchanged at 29 cases, 229 assertions.

Two bugs found on the way, both fixed in flow

#1319, an out-of-bounds write in the poison hook.
MaybePoisonStagedInputs zeroed four pinned staging blocks over pin.S —
TOKENS — while seq_lens is allocated with pin.R, REQUESTS.
PinnedStepInputs keeps those apart deliberately, because SPEC-DSPARK W8 (#442)
separated them for a speculative verify with S = R * (1 + k). On a pure-decode
step the fill was exact; on a spec step it wrote (S - R) int32s past a
cudaHostAlloc'd block. Bounded — the hook runs only under the test-only
VT_ASYNC_EXECUTOR_POISON=1, whose one consumer is the deterministic RED arm of
tests/parity/test_qwen36_async_serving.cpp — and that bound is the interesting
part: the corruption landed inside the instrument, in the arm whose job is to
fail loudly for one specific reason. Fixed as a consequence of the adoption:
each cell is zeroed over its own capacity(), so the count and the allocation
are the same fact. Present at this branch's base 5c8671c50, so found here and
not caused here.

A fragile ordering the adoption introduced, fixed before it could bite, at
BOTH sites. The shape-change path dropped s.dev — which owns the device
destinations — before unbinding the cells that name them. Nothing was wrong at
runtime, because Unbind() does not dereference the destination; the order was
correct by the implementation's silence rather than by anything stated, which is
one edit away from a use-after-free on the rarest path this driver has. The fresh
review then found the CAPTURE path with the same shape and no such statement:
s.dev = std::make_unique<StepDevInputs>(...) destroys the previous one while
s.pin's cells still name its pool blocks, rebinding four statements later, and
the aux_taps branch reaches it with the cells bound because it resets the graph
without calling s.pin.Free(). Both drivers now free first, with the reason at
the site. No new gate, matching the commit that made the same class of reorder on
the other path.

What LANDS UNREACHED, named rather than implied

vt::PersistentStepInput::RefreshFromDevice has no production caller.
grep -rn RefreshFromDevice src/ include/ returns its definition alone, and
last_source()/StepInputSource have no production reader either. This is the
staged slice AGENTS.md admits: the arm is unreachable until a driver holds a
device token-id DESTINATION, and no driver does — which is the same finding that
keeps the decline standing. Writing a caller before that destination exists would
be the tenth hand-rolled copy this row removes. Owner: row
ENG-CUDAGRAPH-BREAK, the stage that gets a dgx window WITH the
checkpoints; listed under ## Owed in the spec, at the declaration in
include/vt/persistent_step_input.h, and here; tracked by #1179 and #323.

The HOST arm IS reached, and its reach is bounded rather than claimed whole:
StageStepInputs routes both Qwen3.5 decode drivers through it when dbuf is
set, which is VT_ASYNC_EXECUTOR=1 (default OFF, docs/ENVIRONMENT.md) or a
speculative verify step, and test_qwen3_5_decode_graph_seam holds that call site
with a mutation that reds only the reachability case.

Fresh-review repairs

A fresh review returned FAIL with one MEDIUM, two LOW and one INFO. Every mutation
it ran proved the gates detect what they claim; the repairs are records and two
small behaviours.

MEDIUM. The decline's own comment was left standing verbatim, still
asserting the DeviceTokenIdsScope mechanism this branch falsified, still
naming StepDevInputs as the fix at the stale anchor qwen3_5.cpp:3894, and
still reading Owner: W4 — a stage this branch marks DONE. The spec,
docs/STATUS.md, the engine-matrix row, the header and this body all carried
the correction; the one place a reader lands did not. Rewritten, re-owned to row
ENG-CUDAGRAPH-BREAK, anchors repointed onto the FUNCTION so the next rewrite
cannot stale them. The decline itself is unchanged. The spec's preamble
carried the same falsified premise and now says so where a reader meets it.
LOW. The capture-site lifetime order, above.
LOW. The unreached arm's owner was a description; it is a row ID now.
INFO. A zero-byte refresh was a silent no-op — the early return sat before
the counters, so host_refreshes under-counted and last_source_ reported the
previous arm, or kUnset, over a step that did choose one. The COPY is skipped
and the decision is counted. Red first, and the mutation restoring the early
return compiles clean and reds only that case.

The public page the gate caught

check-doc-checkpoint red on the SQUASHED shape, because W4 edits
CMakeLists.txt — a member of both USER_USAGE_FILES and LANDING_SOURCE_FILES
— to add src/vt/persistent_step_input.cpp to the core library, and the first
class DEMANDS docs/USAGE.md. Adding an internal .cpp to an existing target
changes no command, config key or install step, so the tempting reading was that
the classifier is too broad here and an argued exception is the honest answer.

Reading the section it points at settles it the other way. docs/USAGE.md
"Turning CUDA graph capture off, including the break seam" carried three claims
this branch makes false:

VLLM_CPP_CUDAGRAPH=0 "always did" reach "the six batched decode drivers that
each read it". After W4 none of them reads it:
grep -rn 'getenv("VLLM_CPP_CUDAGRAPH")' src/ returns exactly one line,
src/vt/breakable_graph.cpp:61.
"No production step opens a capture scope yet." Six do, at six sites in five
translation units. The sentence was written at W1, when it was true; W2
falsified it and W3 widened the gap, and neither rewrote it, because neither
touched a USER_USAGE_FILES path and the gate never asked. W4 touches
CMakeLists.txt for the same reason W1 did, so the gate asks again — the
classifier working three waves late, not spuriously.
The build paragraph named src/vt/breakable_graph.cpp as the one seam file in
the core library on every platform. src/vt/persistent_step_input.cpp now sits
beside it in the same unconditional add_library(vllm STATIC ...) list.

The three paragraphs are CORRECTED, not appended to, and the second says outright
that it asserted the opposite until now, so a reader who remembers the old text
lands on the correction. No claim about the switch's MEANING changes, because
none of it changed: same name, same parse
(e == nullptr || std::string(e) != "0"), same default. What changed is where it
is read and what it reaches.

No checker is edited. Excluding CMakeLists.txt from a path set to clear a red
would widen a gate's scope to silence it, and on this evidence the gate was
right. Verified on the squashed shape in a detached worktree at origin/main
f06b9e93d: pre-fix squash 2aa849efc exit 1 with the reported error, post-fix
squash 680746640 exit 0, "public documents match the claims this change makes."

What W4 did NOT do

The async battery, above. It needs dgx WITH checkpoints. It now owes TWO runs:
as it stands, and with the decline deleted, because only the second can fail.
The auxiliary-stream auto-join (spec D10). qwen3_5.cpp:6254-6255,6384 forks,
but under kFull there is no segment close inside the fork window, so the rule
is still unexercised. It becomes live for the first PIECEWISE production
capture.
The D1 INPUT half (pool reuse between segments) and the caught-exception
residual, both of which are properties of a between-segments window a kFull
capture does not have.
The PIECEWISE arm still has no production driver.

Closes #1307 and #1319. Parent #1163 stays open: W5 takes the three single-shape drivers
and G5's ROCm/Tenstorrent arm, W6 moves the eligibility predicate off
pure_decode. #1305 and #1179 stay open with their disposition recorded above.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

… seam capability, not one driver's private code (#1307, #1163) W4 of `ENG-CUDAGRAPH-BREAK`. `StepDevInputs` appears on 41 lines of `src/vllm/model_executor/models/qwen3_5.cpp` and ZERO lines of `qwen3_moe.cpp`, `qwen3.cpp`, `deepseek_v2.cpp`, `voxtral.cpp` and `qwen3_dflash.cpp`. That asymmetry is not tidiness. The four drivers without it replay against HOST vectors, and one of them declines its decode graph outright whenever the asynchronous device-token mirror is live (`qwen3.cpp:1106`), on a measured battery: `depth-1 graph ON PASS 78/78`, `depth-2 graph OFF PASS 82/82`, `depth-2 graph ON FAIL, slots 1-3 degenerate` (#323, #1179). A shipped model lost graphed decode because a capability written once was unavailable to it, and #1305's three registrations admit the same asynchronous step with no decline at all. An eighth hand-rolled copy would be the defect, so the capability moves to the seam. `vt::PersistentStepInput` is one capture-stable per-step device input. It BINDS a destination the driver owns and refreshes it IN PLACE from either a pinned host staging block or a DEVICE source. WHAT IT DELIBERATELY DOES NOT OWN, because a seam that owned it would become a second dispatcher with a per-model branch (spec `## Risks/decisions` D3): the device allocation, and which inputs a model has. `Qwen3_5DecodeGraph` draws its retained inputs from a DEDICATED `DevicePool` precisely so they never pop a block the captured forward's own scratch then needs, and a seam that took that allocation over would silently move nine drivers onto one pool. WHAT IT DOES OWN is the part every driver re-derives: the address-stability rule as a REFUSAL rather than a comment, the pinned staging block that makes the upload a true asynchronous DMA, and the refreshing ARM as an OBSERVABLE rather than an inference from which line the driver happened to call. THE ARM HAS TO BE OBSERVABLE, and W3 is why. That stage shipped a `kFull`/`kPiecewise` distinction no gate could see, and flipping one token left a whole driver gate GREEN at 226/226. The host and device arms here are the same shape of difference: both leave the destination holding bytes, both leave every pointer unmoved, and only one of them is correct while the mirror is live. So the arm is counted where it is decided, as `last_source()` and as `vt::StepInputStats`. RED FIRST. The gate was written against a stub carrying the declared API and none of its guarantees: `tests/vt/test_persistent_step_input.cpp` ran 9 cases, 0 passed, 59 assertions, 32 failed, exit 1. GREEN after the implementation: 9/9 cases, 59/59 assertions, exit 0. THE CASE THE FILE EXISTS FOR is that a DEVICE refresh recorded inside a capture re-reads its source on every replay while a HOST one replays the bytes its staging block held at capture time — stale versus fresh, on the same captured graph, in the same process, over THREE replays because one cannot distinguish a correct read from one that finds a buffer nothing has overwritten yet. That is the decline's own wording for the fix ("read the identifiers at REPLAY time from a stable device buffer") made executable. THE SHARED RECORDING BACKEND WAS LYING ABOUT COPIES, and this change fixes it rather than working around it. `RecordingCaptureBackend::Copy` performed its memcpy immediately even mid-capture, so a capture appeared to have already moved bytes a real `cudaMemcpyAsync` would only move on replay. It now files the copy against the open segment like any other captured operation, capturing the two ADDRESSES and not the bytes — because a real capture bakes the pointers, and a harness that snapshotted the bytes would make the fresh and stale arms indistinguishable. `test_breakable_graph` is unchanged at 29 cases, 229 assertions, exit 0, the same numbers #1291 recorded. NEGATIVE MUTATIONS, each with its diff size, its compile status and its exit status, each detected, each restored and verified by sha256: | mutation | changed | compiled | result | |---|---|---|---| | delete the address-stability refusal | 5 lines | clean | 8/9, 54/59, exit 1 — reds only the capacity case | | null device source becomes a silent no-op | 6 lines | clean | 8/9, 58/59, exit 1 — reds only the null case | | the host arm stops going through staging, collapsing the two arms | 2 lines | clean | 8/9, 57/59, exit 1 — reds only the stale-versus-fresh case | No throughput is claimed and none was measured. GB10 measured prefill idle between launches at 3.8% with GPU-busy above 96%, and the 27B prefill gap at 92.5% non-GEMM glue GPU work; decode is already captured and already banked its launch-overhead win. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…o the seam, retiring the last two VLLM_CPP_CUDAGRAPH reads (#1307, #1163) W4 of `ENG-CUDAGRAPH-BREAK` migrates `Qwen3_5DecodeGraph` (35B GDN-hybrid MoE) and `Qwen3_5DenseDecodeGraph` (27B dense) onto the break-point capture seam. Each `Step` opens a `vt::GraphCaptureScope` over a per-slot `vt::BreakableGraph` in `kFull` and replays through `BreakableGraph::Replay`. What is gone from both: the hand-rolled `BeginCapture`/`EndCaptureGraph` pair, the raw `void*` handle, the `bool captured` flag, the `DestroyGraph` loop, the hand-rolled `try { EndCaptureGraph(); } catch (...) {}` drain, and the driver's own `VLLM_CPP_CUDAGRAPH` read. SIX OF THE NINE DRIVERS ARE NOW ON THE SEAM, and the six batched-driver reads of `VLLM_CPP_CUDAGRAPH` the spec's `## Our baseline` item 1 counted are down to ZERO. `grep -rn 'std::getenv("VLLM_CPP_CUDAGRAPH")' src/` now returns exactly ONE line, `src/vt/breakable_graph.cpp:61`, and that one is the point: there is finally a single switch that turns capture off. The three single-shape drivers still carry their own invented switches (`VT_V4_DECODE_GRAPH`, `VT_DFLASH_GRAPH`, `VT_LAGUNA_DECODE_GRAPH`); they are W5's. kFULL, INHERITED FROM W2 AND NOT RE-ARGUED. vLLM's v1 default `FULL_AND_PIECEWISE` (`vllm/config/compilation.py:63` @ pin `5559679229`) is documented at `:630-632` as a FULL graph for DECODE batches and a piecewise one for prefill and mixed batches, and `decode_mode()` (`:65-66`) returns the full half. No throughput is claimed here and none was measured. ## Each driver owes its own gate, and they share one harness Nothing else can see the difference: a driver that kept its raw pair produces identical logits, an identical backend log and an identical `replay_count()`. `segments_captured`, `full_scopes` and `replays` are the only observables that separate "captured a graph" from "captured a graph THROUGH THE SEAM". `tests/vllm/models/test_qwen3_5_decode_graph_seam.cpp` carries both drivers and includes `decode_graph_seam_harness.h`. Both, not one file each, because a fifth copy of the harness would reproduce inside `tests/` the duplication this row removes from `src/`, and two copies diverge invisibly — both stay green while measuring different things. Each driver gets its own cases: G2 over cold, capture, replay and a second replay; G4 bit-exactness of the capture step against the driver's own eager forward; and the capture-failure case. RED FIRST, on the MoE driver's five seam assertions before the migration: 3 cases, 1 failed, 62 assertions, 5 failed, exit 1. GREEN after, both drivers: 6 cases, 124 assertions, exit 0, with G4 reading `40 values, 0 differing` on each. ## G2 reachability mutations | mutation | changed | compiled | result | |---|---|---|---| | restore the pre-W4 `qwen3_5.cpp` wholesale | 296 lines | clean | reds BOTH drivers' G2 cases, 10 of 124, exit 1 | | the MoE driver replays `Backend::ReplayGraph` directly, bypassing the container | 7 lines | clean | reds ONLY the MoE G2 case, 1 of 124, exit 1 | | the MoE driver's `kFull` flipped to `kPiecewise` | 3 lines | clean | reds ONLY the MoE G2 case on `full_scopes`/`piecewise_scopes`, 2 of 124, exit 1 | Every one left `test_qwen3_decode_graph_seam` 231/231, `test_qwen3_moe_decode_graph_seam` 228/228, `test_breakable_graph` 229/229 and `test_persistent_step_input` 59/59 GREEN, so each gate is scoped to its own driver. Each file restored and verified by sha256. THE MODE MUTATION IS THE ONE W3 COULD NOT DETECT before it added the counters, and it is re-measured here rather than assumed: neither Qwen3.5 driver registers a `vt::GraphBreak`, so `breaks_registered == 0` holds in both modes for them too, and only `full_scopes`/`piecewise_scopes` move. ## One accessor was added to the container, narrowly `vt::BreakableGraph::segment(size_t)` returns one segment's opaque handle. Its single caller is the `VT_BENCH_PROFILE_CONTROL` replay marker in the dense driver, a bench-only build that must hand the handle to a profiler. It is not an invitation to interpret it: spec `## Risks/decisions` D4 still requires every acquisition and release to go through `Backend::EndCaptureGraph` and `Backend::DestroyGraph` so #1162 can interpose at the backend, and it does. ## Gates run Focused: `test_qwen3_5_decode_graph_seam` 6/6 (124), `test_breakable_graph` 29/29 (229), `test_persistent_step_input` 9/9 (59), `test_qwen3_decode_graph_seam` 4/4 (231), `test_qwen3_moe_decode_graph_seam` 3/3 (228). `ctest -R 'qwen3_5|qwen35|qwen27|breakable|persistent_step|decode_graph|qwen3_moe|qwen3_forward|deepseek|voxtral|mtp|runner'` 65/65 passed, 0 failed. Full CPU build exit 0. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…hrough the seam capability, and drop a pinned block nothing uploaded (#1307, #1163) The capability landed one commit ago with a unit gate and no production caller, which is precisely the shape `.agents/reachability.md` exists to catch: a unit test that constructs the type by hand proves the class works and never that anything reaches it. This commit routes `Qwen3_5DecodeGraph`'s and `Qwen3_5DenseDecodeGraph`'s per-step input staging through `vt::PersistentStepInput`, so the capability is entered from `ModelRegistry::Forward` -> the registered Qwen3.5 forward -> `Step`. WHAT CHANGES. `PinnedStepInputs` held six bare pinned pointers it allocated, and `StageStepInputs` issued the matching six `Backend::Copy` calls ninety lines away. Nothing checked that the staged byte count and the destination's size agreed, and nothing stopped a refresh from being longer than the destination the captured graph baked. Each field is now one `vt::PersistentStepInput` binding the device destination and its pinned staging block together, and the seam refuses a refresh longer than the bound capacity. IT BINDS RATHER THAN ALLOCATES, deliberately. The device side is drawn from a DEDICATED `DevicePool` so a retained input never pops a block the captured forward's own scratch then needs — a `cudaMalloc` mid-capture aborts the capture — and a seam that took that decision over would silently move nine drivers onto one pool. ## The pinned token-id block was dead, and its absence is a finding `PinnedStepInputs::token_ids` was allocated at capture, filled by `StageStepInputs` every step, zeroed by the poison hook, and NEVER uploaded and never read. `StepDevInputs` has no token-id member at all, because the embed runs OUTSIDE the captured region from the HOST vector (`EmbedInto`). So THE DECODE GRAPH CARRIES NO TOKEN IDS TO THE DEVICE, in the one driver that has the persistent device input path. That matters well beyond a wasted allocation: it means "make `StepDevInputs` a seam capability" was never going to close the `qwen3.cpp` async decline by itself, because the input that an asynchronous mirror patches is exactly the one the path does not carry. The fix that decline names — read the identifiers at REPLAY time from a stable device buffer — needs a DEVICE token-id destination that NO driver has. What this row lands is the capability that makes writing one a per-driver two-liner instead of a tenth hand-rolled copy. Recorded in the spec's `## Owed`. Removing it cannot change behaviour: a block nothing uploads cannot poison anything, which is why it also leaves the `VT_ASYNC_EXECUTOR_POISON` hook covering the three inputs that are actually staged. ## Reachability, gated and mutated `tests/vllm/models/test_qwen3_5_decode_graph_seam.cpp` gains a case that enters through `Qwen3_5DecodeGraph::Step` and asserts `vt::GetStepInputStats()`, which moves only inside `vt::PersistentStepInput`: `binds=6` after the capture step and `host_refreshes >= 5` after the replay step. TWO LIMITS, stated rather than hidden. The case sets `VT_ASYNC_EXECUTOR=1`, because the persistent device input path sits behind that lever (default OFF) plus the speculative-decode arm — on a default text-decode step the driver holds no `StepDevInputs` at all. And the lever also turns on the 2-slot parity ring, so slot 0 captures on step THREE rather than step two, which is why the walk is longer than the other cases'. NEGATIVE MUTATION: delete the `StageStepInputs(d, s)` production call site (5 changed lines, compiled clean, exit 1). It reds the reachability case and ONLY that assertion, and leaves `test_persistent_step_input` 59/59 GREEN — which is the distinction itself, a class that still works over a capability nothing reaches. `test_qwen3_decode_graph_seam` 231/231, `test_qwen3_moe_decode_graph_seam` 228/228 and `test_breakable_graph` 229/229 were unaffected. Restored and verified by sha256. ## Gates run `test_qwen3_5_decode_graph_seam` 7/7 (129), and `ctest -R 'qwen3_5|qwen35|qwen27|breakable|persistent_step|decode_graph|qwen3_moe|qwen3_forward|deepseek|voxtral|mtp|runner|dspark|expert_stream'` 77/77 passed, 0 failed. Full CPU build exit 0. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

….5 drivers (#1307, #1163) `tests/vllm/models/test_decode_graph_seam_g1_cuda.cpp` gained `Qwen3_5DecodeGraph` and `Qwen3_5DenseDecodeGraph`, so all five drivers on the seam that have a CPU-buildable tiny model are compared bit for bit against their own eager arm over COLD, CAPTURE and THREE consecutive replays. They need a cache pool the three W3 drivers did not: `CudaGdnCachePool` allocates the RECURRENT ssm and conv state on device beside the paged KV, and each arm gets its own. That is not tidiness — the GDN recurrence advances its state every step, so two arms sharing one state would step each other's recurrence and the agreement would measure nothing. The eager arm is still selected without an environment variable: `max_num_reqs == 0` makes `PadToCaptureSize` return -1 and every migrated `Step` falls out to its plain forward, so both arms are the same binary on the same device. READ THE COUNT, NOT THE STATUS. On this box the file now reports 5 cases passed over `assertions: 0`, which is the skip wearing a pass its own header names. The device result is recorded in the spec's `## Gates` G1. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

… this row had asserted three times (#1307, #1163, #1305) The records the W4 change made stale: the spec's `## Work breakdown` W4, `## Now` and three `## Owed` entries, the `ENG-CUDAGRAPH-BREAK` row in the engine matrix, `docs/STATUS.md`, and a new claim file. WHAT W4 DELIVERED. `vt::PersistentStepInput` as the seam capability, both Qwen3.5 drivers migrated, six of the nine drivers on the seam, and exactly one `std::getenv("VLLM_CPP_CUDAGRAPH")` left in `src/` — the seam's own. WHAT W4 FALSIFIED, which is why these edits are not a status bump. This spec, the engine-matrix row and `qwen3.cpp`'s own comment all said the fix the async decline names — read the identifiers at REPLAY time from a stable device buffer — already existed in the sibling driver as `StepDevInputs`. It does not. `StepDevInputs` has no token-id member, and the pinned block that did was allocated at capture, filled every step and NEVER uploaded or read. The embed runs OUTSIDE the captured region from the HOST vector in every batched driver, so the decode graph carries no token ids to the device in ANY driver. So the decline was never one refactor away from removable, and #1305 is the same hazard rather than a different one. THE DECLINE STANDS, on two grounds and not one. The missing destination above, and a second finding: the decline's own recorded CAUSE does not survive contact with the tree. `338cbbfd1` says the registry-level `DeviceTokenIdsScope` "did not close it: this path returns BEFORE the eager forward ever runs", but at that commit's parent the scope was constructed at `qwen3_dense.cpp:96` BEFORE the graph call at `:103`, and `EmbedInto` consumed it on all three of the driver's arms (`qwen3.cpp:610,621,644`). The override WAS live on the graph path. The measured battery is real; the mechanism recorded beside it is not the one that produced it, and nobody has since identified what did. A mitigation whose failure mode is unexplained is not retired by a refactor addressing an explanation nobody confirmed. THE BATTERY WAS NOT RUN, stated plainly rather than implied. `tests/parity/test_qwen3_dense_async_serving.cpp` needs a GPU AND the real Qwen3-0.6B/4B snapshots, and its header restricts it to `dgx.casa`. `dgx:gpu0` was held by another session for W4's whole window; W4's lease was `thor:gpu0`, which carries no such checkpoint. The owed work is now TWO runs and not one — the battery as it stands, and the battery with the decline deleted, because only the second can fail. `docs/STATUS.md` also loses a sentence that had gone false: "two remain" of the per-driver capture-switch reads. None remain. Every added paragraph is inside the 700-character cap. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

Brings #851's record-checker range change under this branch's gates before the W4 pull request is opened, so the checkers this branch reports run against the version that will judge it. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…rs, measured on thor:gpu0 (#1307, #1163) The W4 G1 run, on `thor:gpu0` through an `rc` lease. NVIDIA Thor sm_110, driver 595.78, nvcc 13.0.88, source `4ea38eccbf82bbe6b8a227753fe64463fb840b78`, CUDA-ON build for arch 110, 32 `.cu.o` objects, the 46.3 MB binary resolving `libcudart.so.13` and `libcublasLt.so.13` out of `/usr/local/cuda-13.0/targets/sbsa-linux/lib`. `tests/vllm/models/test_decode_graph_seam_g1_cuda.cpp`: 5 cases, 2066 assertions, 0 failed, exit 0. ``` G1 Qwen3MoeDecodeGraph on CUDA: 5 steps x 100 logits, 0 differing, 4 replays G1 VoxtralDecodeGraph on CUDA: 5 steps x 100 logits, 0 differing, 4 replays G1 DeepseekV2DecodeGraph on CUDA: 5 steps x 100 logits, 0 differing, 4 replays G1 Qwen3_5DecodeGraph on CUDA: 5 steps x 40 logits, 0 differing, 4 replays G1 Qwen3_5DenseDecodeGraph on CUDA: 5 steps x 40 logits, 0 differing, 4 replays ``` THE ASSERTION COUNT CARRIES THE CLAIM AND THE STATUS LINE DOES NOT. With no CUDA backend every case skips and the same file prints `Status: SUCCESS!` over `assertions: 0`, which its own header names as a skip wearing a pass. 2066 is that count, against W3's 1600 for three drivers. Five of the six migrated drivers are now covered by MEASUREMENT. W2's `Qwen3DenseDecodeGraph` still is not: it shares the seam and the shape, which is an argument, and `## Gates` G1 says so rather than letting the sentence "G1 is met" imply otherwise. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…e destination they name is released (#1307, #1163) The shape-change path dropped `s.dev` — the `StepDevInputs` whose `DBuf`s own the device destinations — BEFORE calling `s.pin.Free()`, which unbinds the `vt::PersistentStepInput` cells that name them. Between the two lines the cells held pointers into pool blocks that had already gone back on the free list. Nothing was wrong at runtime, and that is the point: `Unbind()` frees its own pinned staging block and nulls the destination without dereferencing it, so the old order was correct by the implementation's SILENCE rather than by anything stated. A single edit inside `Unbind()` — a `Memset` of the destination, a debug assertion that reads it — would have turned it into a use-after-free that only shows up when a block-table column count changes mid-run, which is the rarest path this driver has. Both drivers reordered, with the reason at the site. Focused gate `test_qwen3_5_decode_graph_seam` 7/7 (129 assertions), and `ctest -R 'qwen3_5|qwen35|qwen27|breakable|persistent_step|decode_graph|qwen3_moe|qwen3_forward|deepseek|voxtral|mtp|runner|dspark|expert_stream'` 77/77 passed, 0 failed. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…block over TOKENS, and it is sized in REQUESTS (#1319, #1307) Found while migrating the two Qwen3.5 drivers onto the capture seam and not caused by it: the shape is present at this branch's base commit `5c8671c50`. `MaybePoisonStagedInputs` zeroed four pinned staging blocks over `pin.S` elements each. `seq_lens` is allocated with `pin.R`. `PinnedStepInputs` keeps those two counts apart deliberately — SPEC-DSPARK W8 (#442) separated them because a speculative verify arrives with `S = R * (1 + k)` — so on a pure-decode step the fill was exact and on a SPEC step it wrote `(S - R)` int32s past the end of a `cudaHostAlloc`'d block. BOUNDED, and the bound is the interesting part rather than an excuse. The hook runs only under `VT_ASYNC_EXECUTOR_POISON=1`, a test-only escape hatch never set in production (`scripts/env-doc-allowlist.txt:6`), whose single consumer is the deterministic RED arm of `tests/parity/test_qwen36_async_serving.cpp:170-173`. So the corruption landed inside the instrument, in the arm whose whole job is to fail loudly for one specific reason — the failure mode most likely to be read as the defect it was meant to detect. Already fixed by the previous commit's adoption, as a consequence rather than as an intention: each cell is now zeroed over its own `capacity()`, so the count and the allocation are the same fact and cannot disagree. This commit adds the reason at the site and the index row, so the next reader learns it from the code instead of re-deriving it. Two of the original four fills are also gone: `block_table` and `qsl` were never poisoned in the first place, and `token_ids` has no pinned block any more because nothing ever uploaded it. Closes #1319. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…scharge, instead of leaving a finished stage on them (#1307, #1163) Four `## Owed` entries still read `Owner: W4` after W4 landed, and W4 discharged none of them. A finished stage left standing as an owner is the failure this protocol's record surfaces exist to prevent: the item looks assigned and nobody is coming. All four are properties of a between-segments window, and both drivers W4 migrated open `kFull`, which has exactly one segment. So none of them was within reach of this stage, and each now says that rather than implying the opposite: - **The auxiliary-stream auto-join (D10).** `qwen3_5.cpp:6254-6255,6384` forks, but under `kFull` no segment CLOSES inside the fork window for the rule to govern, so the machinery would have landed unexercised. Moves to W5, whose Laguna fork is inside the captured region by construction, and to the first stage that captures piecewise. - **The PIECEWISE arm has no production driver.** This entry said the fix is persistent driver-owned storage, "which is the capability `StepDevInputs` already is". W4 landed the storage PRIMITIVE — `vt::PersistentStepInput` — so what is missing is no longer a primitive: it is a driver that holds its layer's inputs there and a break closure that reads them instead of `RunLayer`'s frame. Moves to W6. - **The caught-exception residual** and **D1's INPUT half.** Both become live for the first piecewise capture, and both now record that W4 did not close them for the identical reason W2 did not. Move to W6. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

Second merge of `main` under this branch, taken for the issue index rather than for the code. `#876` and `#1297` touch nothing this branch touches except `.agents/issue-index.md`, where both sides APPEND. `git merge-tree` calls that clean because the path carries `merge=union` in `.gitattributes`. GitHub does not apply the union driver, so an append-only file that merges cleanly here can still read as conflicting there. Resolving it on this side is what keeps the pull request mergeable, and it costs one merge commit rather than a debugging session on the forge. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

Third merge of `main`, and like the second it is taken for one append-only file rather than for the code. `#1318` touches nineteen paths and this branch shares exactly one of them: `.agents/issue-index.md`, where both sides APPEND a row. That path carries `merge=union` in `.gitattributes`, so `git merge-tree` reports it clean and a local merge resolves it without a marker. GitHub does not apply the union driver, so it reported the pull request CONFLICTING against a base it differs from only by two appended lines. Merging on this side is the resolution available to a branch that may not rebase. Nothing in the product diff changed and nothing was re-gated by this merge beyond re-running the affected suites, because the two sides do not touch a shared line of code. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…ir instrument and their date (#1307, #1163) The header cited "41 lines there, and ZERO lines of ... `qwen3.cpp` ...", copied from the spec's `## Our baseline`. Two of those numbers no longer hold at this commit and one never quite did. `grep -c StepDevInputs src/vllm/model_executor/models/qwen3_5.cpp` returns 44 now, because this row added comments naming the symbol. And it returns 1 for `qwen3.cpp`, not 0 — the W2 decline comment names `StepDevInputs` as the fix the driver does not have. That is not a defect in the record's conclusion; it is `grep -c` counting matching LINES, prose included, so the instrument can only bound the answer from above. The header now gives the numbers, the command that produced them, the date they belong to, and what the command cannot distinguish. A number quoted often enough starts being treated as measured, and this one is quoted in four records. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…echanism this row falsified (#1307, #1163) W4 established that the async device-token decline's recorded CAUSE is not the one that produced its measured failure, and re-owned four `## Owed` entries off a finished stage for that reason. The spec, `docs/STATUS.md`, the engine-matrix row, the new header and the pull request body all carry the correction. The one place a reader lands when they open the decline did not. `qwen3.cpp`'s `DenseDecodeGraphForward` still said, verbatim, that "the registry-level DeviceTokenIdsScope (60e71a0) did not close it: this path returns BEFORE the eager forward ever runs". Re-derived at `338cbbfd1^` rather than inherited: the scope is constructed at `qwen3_dense.cpp:96-97`, BEFORE `DenseDecodeGraphForward` at `:102`; all three of the driver's arms (`qwen3.cpp:610,621,644`) call `EmbedInto`, which applies the override through `ApplyDeviceTokenIdsOverride`; and `60e71a0e`'s own three registry sites (`mistral_registry.cpp:89`, `internlm2_registry.cpp:92`, `llama_registry.cpp:101`) have that same order, two of them on the models the battery reproduced on. The override was LIVE on the graph path. The same comment named `StepDevInputs` as the fix, at the stale anchor `qwen3_5.cpp:3894`, and read `Owner: W4` — a stage this branch marks DONE without doing it. W4 measured that no driver carries token ids to the device at all, so that fix exists nowhere; `vt::PersistentStepInput::RefreshFromDevice` is the arm it needs and the DESTINATION is still owed. THE DECLINE IS UNCHANGED, and the rewrite says why it stands rather than why it should go: the battery is real, its recorded mechanism is not what produced it, and nobody has identified what did. A mitigation whose failure mode is unexplained is not retired by a refactor addressing an explanation no one confirmed. Re-owned to row `ENG-CUDAGRAPH-BREAK`, the stage that gets a `dgx` window WITH the checkpoints, matching what the spec's `## Owed` already said. The spec's own preamble carried the falsified `StepDevInputs` premise too and now says so where a reader meets it, not only in `## Now` a thousand lines later. Line anchors into the decline are repointed onto the FUNCTION, because this rewrite staled them and the next one would again. No behaviour change. `test_qwen3_decode_graph_seam` 4 cases / 231 assertions and `test_qwen3_break_point` 2 / 516, both exit 0. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…oo, and count a zero-byte refresh (#1307, #1163) Three fresh-review findings, all in the same capability. THE CAPTURE SITE HAS THE LIFETIME ORDER `6d1dd6800` FIXED ON THE OTHER BRANCH. That commit reordered `s.pin.Free(); s.dev.reset();` on the `cols_changed` path and documented it as "one edit away from a use-after-free". The capture path does `s.dev = std::make_unique<StepDevInputs>(...)`, which destroys the previous `StepDevInputs` and returns its pool blocks to the free list while `s.pin`'s cells still name them, and rebinds four statements later at `s.pin.Alloc`. It is reachable: the `aux_taps` branch resets the graph and clears `warm` WITHOUT calling `s.pin.Free()`, so the next warm step arrives with the cells bound. Harmless today for exactly the reason `6d1dd6800` refused to rely on — `Unbind()` does not dereference. Both drivers now free first, with the reason at the site, and it costs nothing: `Alloc` calls `Free()` first regardless. No new gate, matching `6d1dd6800`, which landed the same class of reorder as a reorder. A ZERO-BYTE REFRESH WAS A SILENT NO-OP. `if (bytes == 0) return;` sat before the counters, so such a call incremented nothing and left `last_source_` unchanged — `host_refreshes` under-counting, and `kUnset` reported over a step that did choose an arm. That observable is the whole reason the type distinguishes its two arms, and a guard nobody can read is a mute switch. Now the COPY is skipped and the decision is still counted. Not reachable with a harmful shape today: every driver byte count is a positive product and `gdn_state_idx` is guarded by `has_idx`, which is why this is a counter defect and not a numerics one. Red first: the new case fails on the unrepaired source, 10 cases / 9 passed / 66 assertions / 6 failed / exit 1. Green after at 10/10 and 66/66. The mutation restoring the early return compiles clean and reds ONLY that case, leaving `test_qwen3_5_decode_graph_seam` 129/129 and `test_qwen3_decode_graph_seam` 231/231 green; the file was restored and verified by sha256. `RefreshFromDevice` LANDS WITH NO PRODUCTION CALLER, and the owner is now a row ID rather than a description. It is a staged slice under "Nothing lands dead": the destination it would refresh does not exist in any driver, so writing a caller now would be the tenth hand-rolled copy this row removes. Named at the declaration, in the spec's `## Owed` and in the pull request, owned by row `ENG-CUDAGRAPH-BREAK`. The HOST arm's reach is bounded rather than claimed whole: `StageStepInputs` routes both Qwen3.5 drivers through it when `dbuf` is set, which is `VT_ASYNC_EXECUTOR=1` (default OFF) or a speculative verify step. `ctest -R 'qwen3_5|qwen35|qwen27|breakable|persistent_step|decode_graph|qwen3_moe|qwen3_forward|deepseek|voxtral|mtp|runner|dspark|expert_stream'` 77/77 passed, 0 failed, exit 0. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

Brings in the CI concurrency-group fix (#1323) and the DFlash2 scoping spec (#1321). Neither touches the decode-graph seam, `vt::PersistentStepInput` or the async decline this branch repairs. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…tion step opens a capture scope (#1307, #1163) `check-doc-checkpoint` reds the squashed shape of this branch, because W4 edits `CMakeLists.txt` — a member of both `USER_USAGE_FILES` and `LANDING_SOURCE_FILES` — to add `src/vt/persistent_step_input.cpp` to the core library. The first class DEMANDS `docs/USAGE.md`. Adding an internal `.cpp` to an existing target changes no command, no config key and no install step, so the tempting reading is that the classifier is too broad here and the honest resolution is an argued exception. It is not. The classifier is pointing at a section this branch makes wrong, and reading it settles the question: * `docs/USAGE.md` says `VLLM_CPP_CUDAGRAPH=0` "always did" reach "the six batched decode drivers that each read it". After W4 not one of them reads it. `grep -rn 'getenv("VLLM_CPP_CUDAGRAPH")' src/` returns exactly one line, `src/vt/breakable_graph.cpp:61`. * The same section says "No production step opens a capture scope yet — that arrives when the decode drivers migrate onto the seam". Six do, at six sites in five translation units. That sentence was written at W1, when it was true; W2 falsified it and did not rewrite it, because W2 touched no `USER_USAGE_FILES` path and the gate never asked. W3 the same. W4 touches `CMakeLists.txt` for the same reason W1 did, and the gate asks again — which is the classifier working, three waves late rather than spuriously. * The build paragraph names `src/vt/breakable_graph.cpp` as the one seam file in the core library on every platform. `src/vt/persistent_step_input.cpp` now sits beside it in the same unconditional `add_library(vllm STATIC ...)` list. That sentence is exactly what `CMakeLists.txt`'s membership in `USER_USAGE_FILES` exists to keep current. So the three paragraphs are corrected rather than appended to, and the second one says outright that it asserted the opposite until now — a reader who remembers the old text should land on the correction rather than wonder which page is stale. No claim about the switch's MEANING changes, because none of it changed: the same name, the same parse (`e == nullptr || std::string(e) != "0"`), the same default. What changed is where it is read and what it reaches. Nothing else in the branch is touched. No checker is edited: excluding `CMakeLists.txt` from a path set to clear a red would widen a gate's scope to silence it, and on this evidence the gate was right. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

mudler added 17 commits August 19, 2026 06:29

localai-bot merged commit 3b76ccb into main Aug 19, 2026
1 of 3 checks passed

localai-bot deleted the row/ENG-CUDAGRAPH-BREAK-W4 branch August 19, 2026 08:44

localai-bot mentioned this pull request Aug 19, 2026

qwen3_5 decode-graph poison hook wrote past the end of the pinned seq_lens block on a spec step #1319

Closed

Navigation Menu

feat(ENG-CUDAGRAPH-BREAK): W4 — the persistent device input path becomes a seam capability, and it does not close the decline (#1307, #1163, #1305) - #1310

feat(ENG-CUDAGRAPH-BREAK): W4 — the persistent device input path becomes a seam capability, and it does not close the decline (#1307, #1163, #1305)#1310
localai-bot merged 17 commits into
mainfrom
row/ENG-CUDAGRAPH-BREAK-W4

localai-bot commented Aug 19, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

localai-bot commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What lands

The finding that changes this row's own story

The decline decision: it STANDS, and qwen3.cpp is untouched

Gates

Two bugs found on the way, both fixed in flow

What LANDS UNREACHED, named rather than implied

Fresh-review repairs

The public page the gate caught

What W4 did NOT do

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

localai-bot commented Aug 19, 2026 •

edited

Loading

The decline decision: it STANDS, and `qwen3.cpp` is untouched