Skip to content

record(ENG-CUDAGRAPH-DEDUP): the fold engages at the shipped bucket set, and the saving does not survive its own null control (#1162, #1303) - #1304

Merged
localai-bot merged 1 commit into
mainfrom
row/ENG-CUDAGRAPH-DEDUP-BYTES-RESULT
Aug 19, 2026
Merged

record(ENG-CUDAGRAPH-DEDUP): the fold engages at the shipped bucket set, and the saving does not survive its own null control (#1162, #1303)#1304
localai-bot merged 1 commit into
mainfrom
row/ENG-CUDAGRAPH-DEDUP-BYTES-RESULT

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

W6 is the device-byte measurement W4 and W5 both recorded as owed. Every number
before it counted executables; this row's stated saving is a memory saving, and
nobody had measured bytes. It ran on origin/main 2c8f53d -- PR #1232 merged --
so the "nothing on main folds today" caveat every earlier record carried is
retired and this measures a configuration that ships.

The fold is real at the bucket set that ships. vllm-bench sets max_num_seqs =
concurrency, so W32 captured 7 of 7 and W64 captured 11 of 11, exactly
decode_graph_sizes.h:32-41, against the 2-3 buckets every earlier conclusion was
drawn from. The coarse key folds 7 graphs to 3 execs and 11 to 5; the exact key
folds nothing at probes=0, reproducing W4 at four times the bucket count. Token
ids are byte-identical across every cell of a workload, including both OFF/OFF
controls.

The saving is not real. Two instruments agree. nvidia-smi per-process tail
median shows W64 identical to the megabyte in all five cells and W32's coarse arm
reading higher than its OFF arm. A cudaMemGetInfo shim gives a nominal 13.83 MiB
at 7 buckets, 0.42% of a 3.25 GiB process, and nothing at 11. That nominal effect
fails four independent controls: EXACT is a true null and disagrees with OFF by
10.6-13.1 MiB against a 13.83 MiB candidate; the W64 OFF/OFF pair disagrees with
itself by 18.2 MiB; one instantiate recorded a negative delta; and
cudaGraphExecDestroy reclaimed zero in every cell. These are pool-granular
readings, and the coarse arm's throwaway probes grow that pool exactly like
retained execs do.

The mechanism inverts this row's premise. The driver refuses 43% of probes at 7
buckets and 73% at 11, and every refusal gives one reason:
cudaGraphExecUpdateErrorTopologyChanged. The shim's node counts say why false
candidates form at all -- the decode graphs are two topologies, 376 and 404
nodes, mixed across the buckets. Every refusal is about topology, never a
parameter, so a coarser key produces more false hits rather than more folds. W5's
refused=0 was an artefact of workloads whose buckets only ever shrank, so exactly
one pair was ever presented.

The verdict is delivered and negative: both flags stay OFF, now on measurement
rather than on silence, and the coarse key alone is a no-op rather than merely
unsupported because GraphDedupCoarseKeyEnabled() is reached only from Register
under GraphDedupEnabled() at src/vt/cuda/cuda_backend.cu:237. No ceiling is
declared: three traceable things would change it, and the first is now #1303.

The clock pin was refused inside the lease, so no time-based figure is
attributable and none is recorded. nvcc was 13.0.88 here against 13.3.73 for the
W5 fold run, result=2 is one driver on one GB10, only the Qwen3 dense decode
driver was exercised, VT_ASYNC_RUNNER=0 throughout so the feature is still
unreachable on the default serving path, and cudaMemGetInfo cannot separate an
executable's own cost from the pool chunk that satisfied it. Every one of those
is in the record.

On the row state. The DONE case is real and was argued rather than dismissed: the
measurement obligations are discharged, the decision the row existed to inform is
delivered, and an ## Outcome recording what was measured and rejected is what
AGENTS.md asks of a DONE row. The row stays ACTIVE for three reasons, none of
them a checker technicality. The feature is unreachable on the default serving
path, which is the "nothing lands dead" half of this row and is owned by
ENG-CUDAGRAPH-BREAK (#1179). Two items still sit under #1162 itself: the
device-tier signature stability and discrimination tests, and probing
group.current_raw instead of raws.front() to retire the transitivity assumption
in Replay. And the DONE record surface owes a .agents/parity-ledger.md entry, a
closing-commit owner in place of CLAIM-ENG-CUDAGRAPH-DEDUP, an exact test anchor,
and the release of that claim -- an operator act, on files this record-only
branch does not own. Measured, not read: flipping the row to DONE here reds
check-agent-record.py with "lacks exact parity-ledger link" and "active claim
CLAIM-ENG-CUDAGRAPH-DEDUP references ENG-CUDAGRAPH-DEDUP in state DONE". It is
not PARTIAL, because nothing upstream is omitted. It is not BLOCKED, because
nothing external stops the next step.

The issue index gained one row, for #1303. #1226 is closed and #1162 already has
a row, and check-agent-record.py refuses a duplicate because under merge=union a
duplicate is what two branches appending the same issue look like. #1303 is new:
it owns the 376/404 topology split, which is a hypothesis about the capture path
rather than about the key, and the live residue of the closed #1226 is re-homed
onto it and onto #1162 in the spec's ## Owed.
Should #1162 close? Not with this pull request. It is the row's own issue, it
still owns the two engineering items named above, and closing it would leave them
without an owner while the row is still ACTIVE. What #1162 loses is its open
QUESTION: the benefit is measured, the default is decided, and no further attempt
at the same flip is owed. Close it when the row reaches DONE, together with the
parity-ledger entry, the closing-commit owner and the release of
CLAIM-ENG-CUDAGRAPH-DEDUP that the flip requires. #1226 is already closed, and its
live residue is re-homed here onto #1303 and #1162.

Record-only. No engine code changes, and no gate was leased or re-run: the
measurement was taken on 2026-08-19 and this pull request writes it down.
Checkers run green on this head: check-agent-record.py,
check-public-doc-tables.py, check-gate-commands.py,
check-issue-index-append-only.py --base origin/main --head HEAD,
check-commit-trailers.py and check-commit-style.py --range origin/main..HEAD.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

…et, and the saving does not survive its own null control (#1162, #1303)

W6 is the device-byte measurement W4 and W5 both recorded as owed. Every number
before it counted executables; this row's stated saving is a memory saving, and
nobody had measured bytes. It ran on origin/main 2c8f53d -- PR #1232 merged --
so the "nothing on main folds today" caveat every earlier record carried is
retired and this measures a configuration that ships.

The fold is real at the bucket set that ships. vllm-bench sets max_num_seqs =
concurrency, so W32 captured 7 of 7 and W64 captured 11 of 11, exactly
decode_graph_sizes.h:32-41, against the 2-3 buckets every earlier conclusion was
drawn from. The coarse key folds 7 graphs to 3 execs and 11 to 5; the exact key
folds nothing at probes=0, reproducing W4 at four times the bucket count. Token
ids are byte-identical across every cell of a workload, including both OFF/OFF
controls.

The saving is not real. Two instruments agree. nvidia-smi per-process tail
median shows W64 identical to the megabyte in all five cells and W32's coarse arm
reading higher than its OFF arm. A cudaMemGetInfo shim gives a nominal 13.83 MiB
at 7 buckets, 0.42% of a 3.25 GiB process, and nothing at 11. That nominal effect
fails four independent controls: EXACT is a true null and disagrees with OFF by
10.6-13.1 MiB against a 13.83 MiB candidate; the W64 OFF/OFF pair disagrees with
itself by 18.2 MiB; one instantiate recorded a negative delta; and
cudaGraphExecDestroy reclaimed zero in every cell. These are pool-granular
readings, and the coarse arm's throwaway probes grow that pool exactly like
retained execs do.

The mechanism inverts this row's premise. The driver refuses 43% of probes at 7
buckets and 73% at 11, and every refusal gives one reason:
cudaGraphExecUpdateErrorTopologyChanged. The shim's node counts say why false
candidates form at all -- the decode graphs are two topologies, 376 and 404
nodes, mixed across the buckets. Every refusal is about topology, never a
parameter, so a coarser key produces more false hits rather than more folds. W5's
refused=0 was an artefact of workloads whose buckets only ever shrank, so exactly
one pair was ever presented.

The verdict is delivered and negative: both flags stay OFF, now on measurement
rather than on silence, and the coarse key alone is a no-op rather than merely
unsupported because GraphDedupCoarseKeyEnabled() is reached only from Register
under GraphDedupEnabled() at src/vt/cuda/cuda_backend.cu:237. No ceiling is
declared: three traceable things would change it, and the first is now #1303.

The clock pin was refused inside the lease, so no time-based figure is
attributable and none is recorded. nvcc was 13.0.88 here against 13.3.73 for the
W5 fold run, result=2 is one driver on one GB10, only the Qwen3 dense decode
driver was exercised, VT_ASYNC_RUNNER=0 throughout so the feature is still
unreachable on the default serving path, and cudaMemGetInfo cannot separate an
executable's own cost from the pool chunk that satisfied it. Every one of those
is in the record.

On the row state. The DONE case is real and was argued rather than dismissed: the
measurement obligations are discharged, the decision the row existed to inform is
delivered, and an ## Outcome recording what was measured and rejected is what
AGENTS.md asks of a DONE row. The row stays ACTIVE for three reasons, none of
them a checker technicality. The feature is unreachable on the default serving
path, which is the "nothing lands dead" half of this row and is owned by
ENG-CUDAGRAPH-BREAK (#1179). Two items still sit under #1162 itself: the
device-tier signature stability and discrimination tests, and probing
group.current_raw instead of raws.front() to retire the transitivity assumption
in Replay. And the DONE record surface owes a .agents/parity-ledger.md entry, a
closing-commit owner in place of CLAIM-ENG-CUDAGRAPH-DEDUP, an exact test anchor,
and the release of that claim -- an operator act, on files this record-only
branch does not own. Measured, not read: flipping the row to DONE here reds
check-agent-record.py with "lacks exact parity-ledger link" and "active claim
CLAIM-ENG-CUDAGRAPH-DEDUP references ENG-CUDAGRAPH-DEDUP in state DONE". It is
not PARTIAL, because nothing upstream is omitted. It is not BLOCKED, because
nothing external stops the next step.

The issue index gained one row, for #1303. #1226 is closed and #1162 already has
a row, and check-agent-record.py refuses a duplicate because under merge=union a
duplicate is what two branches appending the same issue look like. #1303 is new:
it owns the 376/404 topology split, which is a hypothesis about the capture path
rather than about the key, and the live residue of the closed #1226 is re-homed
onto it and onto #1162 in the spec's ## Owed.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
@localai-bot
localai-bot merged commit 5d9fe33 into main Aug 19, 2026
0 of 15 checks passed
@localai-bot
localai-bot deleted the row/ENG-CUDAGRAPH-DEDUP-BYTES-RESULT branch August 19, 2026 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Graph dedup never folds: the signature keys grid dims and memcpy extents, so two padded decode buckets can never group

2 participants