feat(ENG-CUDAGRAPH-BREAK): migrate the three single-shape drivers onto the seam, and join an outstanding aux-stream fork before a segment closes (#1335, #1163)

[localai-bot]

W5 is the LAST migration stage of ENG-CUDAGRAPH-BREAK. It moves the three
SINGLE-SHAPE drivers onto the break-point capture seam and discharges D10, the
auxiliary-stream fork/join, which every earlier stage was structurally unable to
exercise.

NINE OF NINE DRIVERS ARE NOW ON THE SEAM AND THE MIGRATION IS COMPLETE. A
call-shaped grep over src/vllm/ for BeginCapture, EndCaptureGraph,
ReplayGraph and DestroyGraph, with comment lines excluded, returns NOTHING.
The unqualified word-grep still matches 23 lines, every one of them prose about
what was removed — which is why the qualifier is stated rather than a
cleaner-sounding sentence.

Gone with them: three raw void* handles, two hand-written destructors that
released them, and the DFlash driver's private try/drain.

The three drivers

Each Step opens a vt::GraphCaptureScope over its own vt::BreakableGraph in
kFull and replays through BreakableGraph::Replay.

• DFlash draft graph (qwen3_dflash.cpp) — the (1+k) paged draft step of
  the speculator.
• DeepSeek V4 decode graph (deepseek_v4.cpp) — the T=1 resident decode step.
• Laguna decode graph (laguna.cpp) — whose own note at :2116-2119 asked
  for this seam by name and named V4's driver as the sibling that moves with it.
  Both moved here, and that note now says what is actually left device-coupled:
  the captured CHAIN is VT_MARLIN_NVFP4 CUDA code, not the capture.

kFULL, inherited from W2 and not re-argued: vLLM's v1 default
FULL_AND_PIECEWISE (vllm/config/compilation.py:63 @ pin 5559679229) is
documented at :630-632 as a FULL graph for DECODE batches, and decode_mode()
(:65-66) returns the full half. W2 measured what the alternative costs — a
kPiecewise scope over a decode driver does not merely slow the step down, it
FAULTS on the first replay.

The three per-model rollback switches STAY (VT_V4_DECODE_GRAPH,
VT_DFLASH_GRAPH, VT_LAGUNA_DECODE_GRAPH): each is a same-binary A/B lever for
exactly one driver. What changes for a user is that VLLM_CPP_CUDAGRAPH reaches
all three for the first time — before this, none of them read it, so no single
setting turned capture off everywhere. docs/USAGE.md says so.

A FAILED capture now propagates in all three instead of returning its buffer.
Under stream capture nothing between BeginCapture and a throwing
EndCaptureGraph executed — the kernels were RECORDED, not run — so those
buffers hold whatever the allocator last left there. On the DFlash driver that is
especially invisible, because a draft the target rejects looks exactly like a bad
draft.

D10, and why W5 is the first stage that could land it

GraphCaptureScope owns the set of side queues forked since the current segment
opened and joins every outstanding one before Backend::EndCaptureGraph — the
port of _end_current_segment (breakable_cuda_graph.py:353-361 @ SGLang pin
f63458b5be) plus the wait_stream hook (:101-153) whose only purpose is to
populate that set.

W1 registered its break point on a model that forks no auxiliary queue. W2, W3
and W4 all opened kFull, which has exactly ONE segment and therefore no segment
CLOSE inside a fork window for the rule to govern; landing the machinery then
would have landed it unexercised. W5 owns laguna.cpp:2572-2576,2612, the only
fork inside a captured region by construction, and that driver is the rule's
production caller through vt::GraphNoteFork and vt::GraphNoteJoin. No
monkey-patch is needed, because our fork is an explicit RecordEvent /
QueueWaitEvent pair rather than an implicit torch call — the model tells the
scope.

The rule is gated as a COUNTER and an ORDER, because W3 proved prose is not a
gate: it measured a mode guard that stayed green at 226/226 under the exact
mutation it named. So vt::GraphBreakStats gains forks_tracked and
forks_auto_joined, and the claim is an ORDER — closing a capture with an
unjoined fork FAILS at cudaStreamEndCapture, so "the join happened BEFORE the
close" is the whole thing, asserted out of ONE backend trace
(Begin RecordEvent QueueWaitEvent EndCaptureGraph Begin EndCaptureGraph).

Gates

Test 15 of ## Tests to port, five arms — the rule, the CONTROL where the model
joins first and the scope must do nothing, kFull, the inert scope, and
re-registering one queue. Two mutations, both compiled clean, both with the
diff stat recorded: deleting JoinOutstandingForks() reds ONLY the new case (5
assertions, exit 1, 29 of 30 cases still passing), and making NoteJoin fail to
retire the entry reds it on 8 — so neither the rule nor its control is
decoration. test_breakable_graph 30 cases / 265 assertions / exit 0.

tests/vllm/models/test_qwen3_dflash_decode_graph_seam.cpp, on the shared
harness. RED FIRST against the unmigrated driver: 3 cases, 0 passed, 16
assertions, 7 failed, exit 1. GREEN after: 3 cases, 18 assertions, exit 0. G2:
replacing the scope and Replay with the pre-W5 raw pair (compiled clean, 79
insertions / 29 deletions) reds ONLY that file, while test_breakable_graph
(265), test_qwen3_decode_graph_seam (231), test_qwen3_moe_decode_graph_seam
(228), test_voxtral_decode_graph_seam (230),
test_deepseek_v2_decode_graph_seam (230), test_qwen3_5_decode_graph_seam
(129) and the driver's OWN test_dflash_propose (31) all stay GREEN. That last
one is the argument restated as a measurement: a driver's bit-exactness suite
cannot see which capture machinery ran. G4 holds in the same file at 24
values, 0 differing.

What was measured on a GPU

Two results from one rc lease on thor:gpu0 — NVIDIA Thor, sm_110, driver
595.78, nvcc 13.0.88, -DVLLM_CPP_CUDA=ON -DVLLM_CPP_CUDA_ARCHITECTURES=110, 32
.cu.o objects, the same provenance W3 and W4 used.

G1 was RE-RUN, and it is a re-run rather than a new case. D10 puts a
JoinOutstandingForks() call on the path of EVERY segment close, so the seam
changed UNDERNEATH the five drivers W3 and W4 measured; carrying an older number
forward would have been asserting a measurement of different code.
test_decode_graph_seam_g1_cuda ran 5 cases, 2066 assertions, 0 failed, exit 0,
every driver again reading 0 differing, 4 replays, and test_breakable_graph
ran 265 assertions on the same device.

And the one thing a green build could NOT have told us was measured
separately. Laguna's capture class is inside #ifdef VT_MARLIN_NVFP4, so "the
CUDA build compiled laguna.cpp" is satisfied just as well by a build that
compiled the migrated region OUT — success and failure are the SAME OBSERVATION.
CMake reported CUDA feature marlin-nvfp4: ENABLED for [110] and
-DVT_MARLIN_NVFP4=1 appears on laguna.cpp's OWN compile command in
compile_commands.json. Then the region itself was MUTATED: an undeclared
identifier inserted immediately after
vt::GraphCaptureScope scope(b, q, graph, kFull) failed the object build under
-Werror at laguna.cpp:2735, against a baseline object build of rc 0, with the
tree restoring to an empty git diff. The identical mutation on DeepSeek V4
failed at deepseek_v4.cpp:1921. Both migrated regions are compiled.

A defect this migration CAUSED, found and fixed in the same flow

#1352. W5 routes the DFlash capture through the seam, and the seam reads
VLLM_CPP_CUDAGRAPH itself — so the driver's admission predicate gained a state
it did not know about. With that switch set to 0 on a capture-capable backend
the driver still chose the CAPTURE lane, ran its eager warm pass, opened an INERT
scope, and ran the whole forward a SECOND time inside it: two full draft forwards
per propose, forever. Correct output, pure waste, on a non-default lane — which
is exactly why nothing would have found it later.

Both versions emit IDENTICAL logits and both leave segments_captured at 0, so
no ordinary gate separates them. The assertion is LANE IDENTITY against a live
control: a backend that CANNOT capture takes the eager path by construction, so a
capture-capable backend with the switch OFF must do exactly the same work. Run in
a child process, because the switch is read once per process.

The first instrument was WRONG in the direction that fails the CORRECT
program, and the harness records why. Counting Backend::Alloc read
control=11, switch-off=0, and would have read those two numbers whichever lane
ran second, because the DevicePool serves the second of two identical forwards
out of its free list without calling Alloc at all. It reported the ORDER of the
runs, not their shape. Copies are per-operation work the pool cannot absorb.
Red-first on the unfixed driver: DIFFERENT-LANE, control copies=4 against
switch-off copies=5, exit 1 — and the same run reports the logits IDENTICAL.

What is OWED, stated per driver because the obstacles differ

• DFlash owes only G1, for the reason every CPU-gated driver in this row owes
  it: a CPU "replay" recomputes nothing.
• DeepSeek V4 owes G1 AND G2. CanRunResidentDecode refuses a CPU queue
  outright and refuses again without the four V4 kernel families registered under
  kCUDA. The harness swaps a backend and a platform; it cannot manufacture a
  device type, a CUDA op registry and a whole vt kernel layer, and a stub that
  did would measure the stub. This migration is covered by CONSTRUCTION, which is
  an argument and not a gate.
• Laguna owes both for a BUILD reason: its capture class is inside
  #ifdef VT_MARLIN_NVFP4, so on a host with no nvcc the migrated region is in
  no reachable binary. Its two green CPU suites verify the FILE, not the region.

One half of 2 and 3 is now CLOSED: the mutation above proves both regions are
really compiled, so "the code might not even be built" is gone. What remains is
behavioural — that the routing reaches the seam, and that a REPLAYED segment
reproduces the eager forward — and both need each model's own device kernels
rather than only a compiler.

G5's ROCm and Tenstorrent arms move from W5 to the ROW. This is the second stage
to inherit the item and hit the identical wall — rc devices lists dgx:gpu0,
orin:gpu0 and thor:gpu0, all NVIDIA — and no ordering of the remaining work
produces the hardware.

Framing

Coverage and correctness, never speed. No throughput is claimed and none was
measured. Prefill has 3.8% host idle at above 96% GPU-busy, the 27B prefill gap
is 92.5% non-GEMM glue, decode is already captured, and the sibling row
ENG-CUDAGRAPH-DEDUP closed its benefit question negative after measurement.
kFull keeps every migrated step's shape the one it already had.

One more record defect, found and corrected in flow

#1361. The spec's G5 entry credited the W1 exit criterion to "sm_121a on GB10",
while the SAME FILE records it as measured on orin:gpu0. A Jetson AGX Orin is
neither a GB10 nor sm_121a, so the sentence named a device the measurement did
not run on and an architecture nothing in this row has measured the criterion
against. The two-architecture claim survives and only the attribution was wrong:
the criterion ran on orin:gpu0, G1 and the unit suite on thor:gpu0 at sm_110.

sm_121a on GB10 is OWED and now has its own entry, carrying the reason it
cannot be closed cheaply: re-running G1 on dgx:gpu0 would not discharge it,
because every migrated driver opens kFull and so nothing in this tree re-begins
a capture mid-forward. It needs the W1 probe itself, or the first PIECEWISE
driver, which is W6.

Closes #1335. Closes #1352. Closes #1361. Parent #1163 stays open: W6 moves the eligibility predicate at
runner.cpp:1341 off pure_decode and unblocks the PIECEWISE arm, and the row
still owns the async decline (#1179, #323), #1305, and G5's non-CUDA arm.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]