fix(VT-ACT-ROUND-POLARITY): narrow act(gate) to the input dtype, which upstream pins bit-exactly (#1322) by localai-bot · Pull Request #1347 · mudler/vllm.cpp

localai-bot · 2026-08-19T11:32:55Z

vt's gated activations computed the whole gated expression in f32 and rounded
once on the store. Upstream narrows act(gate) to the INPUT dtype before the
multiply, on every accelerator path it ships, and asserts that its two
implementations agree EXACTLY. Read at the parity pin 555967922:

csrc/libtorch_stable/activation_kernels.cu:158 — silu_kernel returns (T)(((float)x) / (1.0f + expf((float)-x * alpha)))
csrc/libtorch_stable/activation_kernels.cu:36 — compute returns (scalar_t)(ACT_FN(gate, alpha) * ((float)up + beta)), so ACT_FN has already narrowed
vllm/model_executor/layers/activation.py:143 — F.silu(x[..., :d]) * x[..., d:], and F.silu on a bf16 tensor yields bf16
tests/kernels/core/test_activation.py:108 — torch.testing.assert_close(out, ref_out, atol=0.0, rtol=0.0)

The premise was re-verified before any code was written

The RmsNorm half of #1322 was refuted at exactly this step by an upstream
revert, so the same search was owed here and it came back the other way. There
is no revert: the (T)(...) narrowing is continuous across v0.15.0 through
v0.27.2rc0, including tags NEWER than the pin. The reference checkout is
SHALLOW — its walk cannot reach anything older than 2026-06-10 — so the search
was run on tree content across releases rather than on git log -S alone, which
is also why two of the three shas the spike cites for the RmsNorm revert report
as non-ancestors here.

The committed oracle golden settles it without reading any kernel source.
Recomputing tests/parity/goldens/silu_and_mul_bf16_8x256/ from its own x.npy
reproduces out.npy bit-exactly under bf16(bf16(silu(g)) * up) and under
neither f32 form, so the capture came from a vLLM that rounds:

expression	max abs err vs golden	bit-exact
`silu_f32(g) * up` (before)	1.434994e-02	no
`bf16(silu_f32(g)) * up` (after)	7.812500e-03	no
`bf16(bf16(silu_f32(g)) * up)` (upstream's full chain)	0.000000e+00	yes

Against the harness's own atol + rtol*|want|, the worst element margin moves
0.5364 -> 0.2989. No golden gets worse and none was ever over tolerance.
Over 2^20 bf16 pairs the effect is 27.51% of outputs differing, every one by
exactly 1 bf16 ULP.

Why this is safe

The narrowing target is x.dtype, not out.dtype. Upstream never has the two
differ; this seam permits an f32 input with a bf16 output. Keying on the input
leaves every f32-in path bit-identical, which is why the existing f32 activation
goldens and both byte-exact composite contracts do not move — the gate/up
tensors in test_ops_moe_grouped_bf16_gate_up_silu and the silu_mul_fp4_quant
byte-exact case are f32. That property is asserted directly rather than inferred
from values.

RoundThrough is reused; kRmsNormGatedGroup already used it for this exact
purpose, so no new mechanism was needed.

Evidence

Red-first on a clean tree with the kernel unchanged: test_ops_activation gave
24 cases / 2 failed / Status: FAILURE!, both bf16 exactness cases failing at
REQUIRE(out[...] == want), while the f32-input case already passed. After the
kernel change: 24 cases / 24 passed / 285996 assertions / Status: SUCCESS!.

Fourteen CPU suites were run before and after with the binary sha256 recorded on
each side. All are unchanged-green, and the CPU golden pass ran 46 cases on both
sides. test_qwen3_load and test_gemma3_load report assertions: 0 on both
sides and are recorded as skips, not as passes.

Scope, and what this does NOT claim

CPU only. CUDA, ROCm, Metal and Tenstorrent have no toolchain on this box, so
those kernels are deliberately NOT edited blind. Vulkan is a second-order case:
its GLSL compiles ahead of time into a committed SPIR-V blob and
scripts/gen-vulkan-spirv.py --check exits 1 here with "no GLSL->SPIR-V
compiler found", so editing the shader without regenerating would ship a source
that disagrees with the executed SPIR-V — #1342's defect in a worse form.
Tenstorrent needs no change: ttnn::silu already materializes a bf16 tile
before ttnn::multiply, so it already has the upstream polarity.

Those arms are named under ## Owed in the spec, each with the resource its
gate is pending on. This change claims no provider parity it did not test.

Unrelated and pre-existing, found while gating: test_minimax_music3_e2e_real
is registered unconditionally but links against ApiServer, so the CPU build
breaks under -DVLLM_CPP_SERVER=OFF — the configuration build-test-cpu-arm64
and cuda-fat-build both use. Not touched here.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…the pin, and upstream reverted the edit we were asked to make (#1322, #1342, #1343) FOLLOWING_AGENTS_PROTOCOL #1322 files one defect over `vt::RmsNorm` and `vt::SiluAndMul` keeping f32 where the reference rounds. Read against the primary oracle at the recorded pin the two ops separate, and only one of them is a divergence. `vt::RmsNorm` matches what upstream executes. `layernorm_kernels.cu:: rms_norm_kernel` computes `static_cast<scalar_t>(x * s_variance * w)` and `csrc/cpu/layernorm.cpp::rms_norm_impl` computes `fp32_x * fp32_s_variance * fp32_w` -- f32 across the weight multiply, one rounding, on both backends. Only the Python `ir.ops` reference rounds first, and `vllm_c.py:: rms_no_var_size` reaches it only when `weight.dtype != x.dtype`, which is GemmaRMSNorm, whose weight is already f32 so the cast-back rounds nothing. Upstream then made exactly the proposed edit in `124fac10c` (vllm#42379, "Fix RMSNorm kernels to multiply in weight's native dtype") and reverted it in `225936a1d` (vllm#46070) to unbreak its own CI. Both are ancestors of the pin and the reverted-to form is what the pin carries, so the change would have moved us away from the oracle rather than toward it. The activation half is real and upstream pins it bit-exactly: `silu_kernel` returns `(T)(...)` and `compute` returns `(scalar_t)(ACT_FN(gate, alpha) * ((float)up + beta))`, so `act(gate)` narrows to the input dtype before the multiply, and `test_act_and_mul` asserts `atol=0.0, rtol=0.0` between the kernel and `forward_native`. Measured over 2^20 bf16 pairs, the missing narrowing moves 27.47% of outputs by exactly 1 bf16 ULP. No product code changes here. The fix spans six providers and every bf16 gated multilayer perceptron in the tree, and four of those providers cannot be executed from the shared checkout, so the implementing wave is its own row rather than something to land half-gated. This commit records the ground truth so that row does not re-derive it. Two sibling defects found during the inventory and filed rather than fixed in flow: #1342 (Vulkan and Metal silu shaders document a 1:1 port of the CPU spelling and emit a different one) and #1343 (`vt::RmsNormPlusAdd` rounds between the norm and the add on its composed arm and not on its ROCm arm). Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [Claude Code]

…branch The row branched at 7c48010 and main has moved 32 commits since. The only overlapping record is `.agents/issue-index.md`, which is append-only: main's 459 rows are all present and this branch's three rows (#1322, #1342, #1343) are the only additions, so the union is 462 rows with nothing removed. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…h upstream pins bit-exactly (#1322) The gated activations computed the whole expression in f32 and rounded once on the store. Upstream narrows act(gate) to the INPUT dtype first, on every accelerator path it ships, and asserts the two implementations agree exactly: activation_kernels.cu:158 `silu_kernel` returns `(T)(...)`, :36 `compute` returns `(scalar_t)(ACT_FN(gate, alpha) * ((float)up + beta))`, activation.py:143 is `F.silu(x[..., :d]) * x[..., d:]`, and test_activation.py:108 compares them at `atol=0.0, rtol=0.0`. All four read at the parity pin 555967922. The premise was re-verified before any code was written, because the RmsNorm half of #1322 was refuted at this same step by an upstream revert. There is no such revert here: the narrowing is continuous across v0.15.0 through v0.27.2rc0, including tags NEWER than the pin. `${VLLM_SOURCE}` is a shallow checkout whose walk cannot reach a revert older than 2026-06-10, so the search was run on tree content across releases rather than on `git log -S` alone. The committed oracle golden settles it independently of any source reading. Recomputing `tests/parity/goldens/silu_and_mul_bf16_8x256/` from its own `x.npy` reproduces `out.npy` BIT-EXACTLY under `bf16(bf16(silu(g)) * up)` and not under either f32 form, so the capture came from a vLLM that rounds. Against the harness's `atol + rtol*|want|`, the worst element margin moves 0.5364 -> 0.2989. No golden gets worse. Rounding through `x.dtype` rather than `out.dtype` is what makes this safe: it leaves every f32-input path bit-identical, which is why the existing f32 activation goldens and both byte-exact composite contracts (whose gate/up tensors are f32) do not move. That property is asserted directly rather than inferred from values. Reuses `RoundThrough`, which `kRmsNormGatedGroup` already uses for this exact purpose; no new mechanism was needed. CPU ONLY. CUDA, ROCm, Metal and Tenstorrent have no toolchain on this box and are deliberately not edited blind; Vulkan additionally needs a GLSL compiler to regenerate its committed SPIR-V, which `gen-vulkan-spirv.py --check` confirms is absent here. Tenstorrent already has the upstream polarity. Those arms are named under `## Owed` in the spec with the resource each one is pending on, and this commit claims no parity it did not test. Also drops the duplicate #1322 index row this branch would otherwise merge into main, which `test_agent_record` catches as a union-merge duplicate. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…branch Main advanced again while the CPU arm was being measured. The only overlapping record is the append-only issue index, whose union is verified by set-compare: every issue on main is present, and this branch adds only #1342 and #1343. #1322 appears exactly once, main's landed row, since this branch's duplicate was dropped in the previous commit. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5-1m [claude-code]

…SHA, and ASan explains why the bad read looks in-bounds (#1403) (#1457) `test_qwen3_5_decode_graph_seam` SIGSEGVs on `main` is a real report of a red that `main` had already stopped carrying. #1403 names `96ed8346f`; the repair `7dec1d990` (PR #1393) is not an ancestor of it. At `b537a5344` the file passes five runs of five, CPU-only Release x86_64, 8 cases, 138 assertions, `Status: SUCCESS!`, exit 0. No code was owed, so none is written here. What is written here is the part the re-derivation produced that the original repair could not state about itself, because after its fixture change its own gate no longer reaches the guard it added. ## Both halves of `7dec1d990` are load-bearing Measured by reverting each half to its `96ed8346f` bytes in this binary, one variable at a time. `git diff HEAD --stat` and a `sha256sum` were taken on every arm, and the compile exit code beside each. | arm | build | exit | doctest | |---|---|---:|---| | neither reverted (`b537a5344`) | Release | 0, five of five | 8 cases, 8 passed, 138 assertions, `SUCCESS!` | | `SpecAttnMeta` fixture reverted | Release | 1 | 7 passed, 1 failed; W6 THROWS the bound by name | | fixture and kernel bound reverted | Release | 139, three of three | crash at `test_qwen3_5_decode_graph_seam.cpp:800` | | neither reverted | Debug + ASan/UBSan | 0 | 8 cases, 8 passed, 138 assertions, zero sanitizer findings | | fixture and kernel bound reverted | Debug + ASan/UBSan | 1 | `AddressSanitizer: SEGV` | The refusal the middle arm reaches is `src/vt/cpu/cpu_paged_attn.cpp:152`: `paged_attention: the block table is shorter than the sequence it must address`. So the fixture change alone would have left a silent contract violation, and the bound alone would have turned the crash into a refusal without making the gate pass. Neither is redundant. ## The sanitizer result, and the trap in it Debug plus `VLLM_CPP_SANITIZE=address,undefined` — NDEBUG **off**, so asserts are live — on the fully reverted tree: ``` ==1788353==ERROR: AddressSanitizer: SEGV on unknown address 0x5045f5f84900 ==1788353==The signal is caused by a READ memory access. #0 KvElem<KvKind::K> src/vt/cpu/cpu_paged_attn.cpp:59 #1 operator() src/vt/cpu/cpu_paged_attn.cpp:224 ... vt::cpu::Threadpool::ComputeThread, worker T1 SUMMARY: AddressSanitizer: SEGV src/vt/cpu/cpu_paged_attn.cpp:59 AddressSanitizer can not provide additional info. ``` **There is no free site, because this is not a use-after-free**, and `AddressSanitizer can not provide additional info` is the tell. It is not a `heap-buffer-overflow` either. The out-of-bounds block-table read is IN-BOUNDS as far as ASan is concerned: the pool places the tables, so the read lands inside the neighbouring live allocation and crosses no redzone. It is the **value** read that becomes a wild block index, which `KvElem` multiplies by the KV block stride and dereferences — hence a fault at an unmapped address rather than a poisoned one. The consequence for the next reader is the part worth keeping: **a clean ASan report at that read is not evidence the read is in bounds** for this defect class. Only the bound at `:152` knows where the table's own last column is. It is also why the crash is order-dependent rather than deterministic, which is what #1407 measured from the other side. The `Thread T1 created by T0 here` stack in the raw report is the thread *creation* site — `Threadpool::Global()` from `EmbeddingKernel` — and not an allocation or free site. It is easy to misread as one. ## Two smaller corrections The issue's `5f68e60df` pointer is right about the commit and wrong about the half. `git log -S'SpecAttnMeta'` and `git log -S'W6: two spec shapes of EQUAL S'` each return that commit alone, and both hits sit in the TEST fixture, not in the graph eligibility it widened. A reader who starts at the eligibility predicate is reading production code that was correct. #1405's truncated assertion count is sharper under a worker-thread fault than filed. Doctest printed its summary TWICE with different totals, `135` then `141`, because the main thread ran on after T1 died. A gate pinning a total sees count drift; a gate grepping one summary line can read whichever it reaches first. The exit code remains the authority. ## Gates - `tests/test_qwen3_5_decode_graph_seam`, CPU-only Release x86_64: exit 0, three runs of three at this head; five of five at `b537a5344`; and exit 0 under Debug + ASan/UBSan with zero sanitizer findings. The red arms are in the table above. - Full `ctest -j4`, Release, at `b537a5344`: **567 tests, 564 passed, 3 failed**, 752 s. This branch adds no code, so that is a measurement of `main`'s suite and not of this change. Every red is accounted for and none is this row's: - `test_serve_low_tools` — **passes serially**, 26.50 s. A load artifact: `uptime` read `load average: 91.81` with a concurrent agent building. - `test_ltx2_video` — **OOM-killed**, not a code verdict. `Out of memory: Killed process (test_ltx2_video) ... anon-rss:35991348kB` in `dmesg`, with `free -g` showing 2 GiB free on an 84 GiB box; it took SIGTERM at 562 s having passed 84 of 85 cases. - `test_nemotron_h_paged_forward` — the known attention-backend red, and `9ecaf1bb3` on `main` repairs it. At this head it passes: exit 0, 24.02 s. - `test_engine_core_proc`, the other red I was warned about, PASSED under `-j4`. - A second full run at the rebased head was abandoned rather than reported: the box reached `load average: 240` with 1 GiB free while a sibling agent ran its own suite, and a run in that state measures contention. CI is the clean lane for a full run at this head. What that partial run did surface is #1458 below, which was then confirmed serially. - `check-issue-index-append-only.py`, `check-agent-record.py`, `check-commit-style.py`, `check-commit-trailers.py`: exit 0 against `origin/main..HEAD`, run from this worktree's own `scripts/`. - `scripts/agent-preflight.sh --staged`: exactly one red, `test_cpu_x86_llamacpp_floor`, which is #618's load-dependent harness leg — it returns `NO_QUIET_WINDOW` at `load=133.03` rather than a verdict. - Every build log was checked with `grep -c 'No space left on device'` = 0 and `df -h` before each; free space swung between 15G and 74G and never ran out. - Instrument notes, since three misfired here. `git diff --stat` reads EMPTY after `git checkout <sha> -- <path>`, because that stages the change — a mutation that HAD applied looked like one that had not, until I switched to `git diff HEAD --stat` plus `sha256sum`. `gh issue list --search` returned nothing for a test name that certainly has an issue, so its empty result is not evidence of absence. And `ninja` aborts the whole invocation on an unknown target, so a ctest name that is not a build target left three binaries stale and their re-run looked like a verdict; the compile rc caught it. ## A second, unrelated red on `main`, filed in flow — #1458 The run at the rebased head turned up four suites that were green three commits earlier: `test_ltx2_text_encoder`, `test_muse_glimmer_text`, `test_muse_glimmer_text_fallback` and `test_minimax_music3_ar`. They are deterministic, not contention — first seen under `-j4`, then re-measured serially. `4712dac40` (`VT-ACT-ROUND-POLARITY`, #1347) is the cause, attributed by mutation in both directions rather than inferred from the commit range: reverting `src/vt/cpu/cpu_ops.cpp` alone turns all four green, and restoring it to a re-matched `sha256` turns them red again, compile rc 0 on every arm. It is filed rather than repaired because the in-flow rule covers a small and clear fix and this is a numerics decision — either the bf16 error floors were calibrated against the rounding polarity that commit corrected, or the narrowing is wider than upstream's. The second commit here carries only its index row, because an issue filed without a fix has to name an owner. ## What this does not do No `src/` or `tests/` change. Adding a second guard on the same call chain is what #1407 explicitly withdrew, and re-doing the repair would be worse than the duplicate report it answers. The remaining real gap is #1406 — the same unbounded index in 12 CUDA and 7 ROCm sites — which already has an owner and is untouched here. Closes #1403 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…carries a red this branch did not cause `origin/main` moved four times during this repair -- `9ecaf1bb3`, `f07f96e1c`, `aeba0de6f`, `01854663c` -- and each time `git merge-base --is-ancestor` answered 1, which is the arm that SKIPS `commit-trailers` and `commit-style` without failing. A skip is not a green, so the base is taken again. The third sync brought `4712dac40` (`VT-ACT-ROUND-POLARITY`, #1322 via #1347), which reds four CPU suites: `test_minimax_music3_ar`, `test_ltx2_text_encoder`, `test_muse_glimmer_text` and `test_muse_glimmer_text_fallback`. That is [#1458](#1458), already filed from another flow, and this branch inherits it rather than causing it -- proven both ways rather than argued. Reverting THIS row's two source files to their pre-repair `7a3909187` content leaves all three binaries red at `run_rc 1`; reverting `src/vt/cpu/cpu_ops.cpp` alone to `4712dac40^` turns all three green at `run_rc 0`, with every file restored to an identical sha256 afterwards. This sync itself is CUDA and documents, so it changes nothing the CPU gate runs. The five auto-merged files -- `.agents/issue-index.md` and the four `docs/` projections -- were each verified by key: this branch's delta across the merge is byte-identical to its delta from the merge base. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [ClaudeCode]

…he four reds it brought are attributed both ways `origin/main` moved four times while the device-leakage repair was gated. The third sync carried `4712dac40` (`VT-ACT-ROUND-POLARITY`, #1322 via #1347) and with it four red CPU suites: `test_minimax_music3_ar`, `test_ltx2_text_encoder`, `test_muse_glimmer_text` and `test_muse_glimmer_text_fallback`. They are #1458, filed from another flow before this control was run. The spec now records both `ctest` runs -- `567/567` green on the pre-`4712dac40` base and `rc 8` with those four on the post base -- and attributes them by mutation rather than by argument. Reverting this row's two source files to their pre-repair content leaves all three binaries red; reverting `src/vt/cpu/cpu_ops.cpp` alone to `4712dac40^` turns all three green. Every file was restored to an identical sha256. Neither control repairs anything, and that is deliberate: #1458 needs `VT-ACT-ROUND-POLARITY` to decide whether its kernel or four bf16 error floors that were never re-derived are the wrong side, which is that row's oracle work rather than a small and clear in-flow fix. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [ClaudeCode]

… handing out an address-keyed cache (#1415) Row `A2-Q2b`, issue [#810](#810), spec `.agents/specs/nemotron-h-a2q2b-realckpt-lmhead.md`. Measured first, built second. The row's premise was that NemotronH decode re-expands quantized weights on the host and that `lm_head` is a large share of it. That was arithmetic, and the brief that dispatched this row said so. It is now a count. **This description was rewritten after a fresh review returned FINDINGS.** The measurement and the seam design below survived review unchanged and were verified independently. Three claims that were in the previous description did not, and they are corrected in place rather than quietly dropped: the new test had never compiled, the routing checker was said to hold a branch it does not hold, and the Marlin arm was said to be unbuildable here when it builds in seconds. **A scoped re-review then found no correctness defect in the shipped code and verified all seven repairs, and a final round corrected what it did find — all of it evidence, records and process.** The evidence tables named a tree they were not measured on and now carry an `overlay` column with the un-overlaid baseline stated as its own row; the Marlin object figure is re-measured and anchored to a SHA; the shared process-static this row inherits is named; the #1421 index row's counts are corrected while the pull request is still unmerged and the row still editable; and `§4` of the spec stopped telling the next agent to take `flock` on a fleet device. No product code changed in that round. ## What the measurement says Tallied at `NemotronHOwned::DenseBf16` (`nemotron_h.cpp:395`), the single seam every host dequant in this architecture passes through, on the real 21 GB checkpoint (revision `29f2d174`) through the production ABI driver `examples/nemotron_h_gen`. One decode step, T=1, `top_k` 6, 23 MoE layers, 369 dequant calls: | group | shape | calls | elements | % | |---|---|---|---|---| | routed expert `up_proj` | `[1856, 2688]` | 138 | 688 472 064 | 22.36% | | routed expert `down_proj` | `[2688, 1856]` | 138 | 688 472 064 | 22.36% | | shared expert `down_proj` | `[2688, 3712]` | 23 | 229 490 688 | 7.45% | | shared expert `up_proj` | `[3712, 2688]` | 23 | 229 490 688 | 7.45% | | `lm_head` | `[131072, 2688]` | 1 | 352 321 536 | 11.44% | | mamba `out_proj` (FP8) | `[2688, 4096]` | 23 | 253 231 104 | 8.23% | | mamba `in_proj` (FP8) | `[10304, 2688]` | 23 | 637 034 496 | 20.69% | | **TOTAL** | | **369** | **3 078 512 640** | **100%** | `138 == 6 x 23` exactly, which is what identifies this as the decode shape rather than a prefill aggregate. Three findings, and the first two are corrections: 1. The dispatching estimate put `lm_head` at 131072 x 4096 = 537e6 elements and ~43% of the population. `hidden_size` is 2688. It is 352 321 536 elements and 28.35%. 2. The widely quoted "1.24e9 elements / 2.49 GB per token" names a REGIME, not a total. It is not the host arm's 3 078 512 640. It is `mamba + lm_head` = 1 242 587 136 = 2.485 GB, the residue after A2-Q2a moved the MoE arm to the device, matching to four significant figures. 3. `lm_head` is the last one. Against the A2-D1 discriminator on `dgx:gpu0` (device mamba ON 1.554 s/token and 108.2x vs vLLM, OFF 10.319 s/token and 718.1x), the mamba arm is worth 6.64x and `lm_head` is on the host in every leg. Once the mamba arm lands, `lm_head` is 352 321 536 of 352 321 536, i.e. everything that remains. It is also the largest single re-expansion in the model by 12.7x, so its 704.6 MB transient bf16 buffer is the one that matters on a unified-memory box that reboots rather than OOM-kills. So the estimate's share was wrong, its direction was right, and the case is stronger after the discriminator than before it. ## The residency decision, taken explicitly The spec's `## 5. Owed` required this row to choose rather than default: "A2-Q2a routed around it by never calling either `MarlinDenseResidentFor`; `lm_head` must do the same or say why not." That accessor keys its repack cache on the weight's ADDRESS ([#984](#984)), and NemotronH is exactly the second-consumer condition an address key cannot survive. Rather than fork the seam as `qwen3_5.cpp` did, or inherit the defect as routing through it plainly would, the seam is EXTENDED: `MatmulNvfp4W4A16D` and `MatmulNvfp4MarlinD` accept a caller-owned `MarlinDenseResident*`, defaulting to the existing cache so every current caller is byte-for-byte unchanged. NemotronH owns its resident in a `ResidentSlot` on the weights, which is the property [#237](#237) added `ResidentSlot` for and the same call A2-Q2a made for the MoE arena. [#984](#984) is left exactly as it was for every other caller, where its own two-engine red-before belongs. ## What moved `NemotronHPagedForward`, the production path, now returns a device-resident `ForwardLogits` on an NVFP4 checkpoint, which is every released one, and downloads `final_normed` only when a consumer needs it. `NemotronHDeviceForward` deliberately keeps the host projection. It is the operand the numeric gate compares against, it is what makes A2-R's token gate attributable, and it has no production caller. The spec required this row to disclose where that property ends: it ends in the paged forward. ## The allowlist entry is narrowed, not removed Its one clause is met. But `check-runner-routing-consistency.py` resolves a cross-TU delegate only for the `Class::ForwardDevice` shape, and NemotronH's device forward is a free function in another TU, so the model still classifies HOST although `NemotronHPagedForward` assigns both `fl.device_tensor` and `fl.device_storage`. Filed as [#1410](#1410) and left to its own row: it changes checker semantics, which `AGENTS.md` routes away from an in-flow fix, and widening a classification to turn a red gate green is precisely the move that section slows down. ## The review repair ### The new test had never compiled, so the red it claimed could not have existed `tests/vllm/models/test_nemotron_h_moe_device.cpp` used `NemotronHHostWeights` unqualified and never declared it. Measured on the same command: | tree | `compile_rc` | errors | |---|---|---| | PR head `29b1128e`, this file | 1 | 9 | | merge base, same file | 0 | 0 | | repaired, default arm | 0 | 0 | | repaired, `-DVT_MARLIN_NVFP4=1` | 0 | 0 | The consequence is larger than the declaration. The file had never compiled on any platform, so **the red-first result this row claimed for its synthetic numeric gate did not exist and could not have existed**. That gate is CUDA-only and still has never executed; the spec now records `NEVER RUN` rather than restating a red nobody saw. The test target is deliberately registered with no CUDA guard, and that is what surfaced this — a case that skips at run time still has to parse on every CPU build. ### The production source asserted a protection that does not exist A comment at the device branch claimed the allowlist entry was removed and that "the routing checker, not a comment, is what now holds this branch in place: put the host projection back and it goes red by name". All three claims were false, and they contradicted this description and the allowlist file in the same commit. Reproduced here on the repaired tree: baseline `check-runner-routing-consistency.py` exits 0 with "3 host-logits off-framework (3 allowlisted)"; after deleting the whole `if (DeviceLmHeadEligible(...)) { ... return fl; }` block (`git diff` moves by 1 insertion and 14 deletions, so it applied) the checker exits 0 with **byte-identical output**; the file was restored to an identical sha256. **Nothing automated holds that branch.** The checker is not widened to make it — that changes checker semantics and [#1410](#1410) owns it with its own red-before. The comment now says what is true, the allowlist entry's own text is corrected the same way, and the reachability deletion mutation is recorded PENDING a CUDA window rather than claimed. Every case that can enter the device branch is `MarlinW4A16Selects`-gated and therefore CUDA-only, so that mutation cannot run on a CPU box. ### A silent dequant fallback that no token gate can see `DeviceLmHeadEligible` restated the shared dispatcher's three selection clauses and dropped `MarlinW4A16Enabled()`. Under an explicit `VT_NVFP4_MARLIN=0` the model said eligible while `MatmulNvfp4W4A16D` took its naive redundant-dequant arm — the same logits, while re-uploading the whole `[131072, 2688]` operand on **every decode step**, because `LmHeadNvfp4View` hands out a stack temporary that `ResidentNvfp4`'s weight-keyed cache can never hit. The repair is structural rather than a patched clause. The three clauses now live once, in `dense_nvfp4::MarlinW4A16Selects`, and both the dispatcher and the model call it, so they cannot drift. `DeviceLmHeadD` additionally refuses BY NAME if that same predicate is false on the operand it is about to hand over, which catches an eligibility answer taken against a different queue or dtype. That is deliberately the predicate and not the seam's `fallback_gemms` counter: `MutableW4A16Stats()` is a plain non-atomic process-wide static, so a counter window in production would refuse a correct run whenever anything else took a fallback GEMM concurrently, and a false refusal is worse than the silence it replaces. The counter is the right instrument in a test, where single-threadedness is a property of the harness, and the synthetic gate asserts it there. It is demonstrably armed rather than assumed: `test_qwen3_forward.cpp:559` — the assertion itself, not the `TEST_CASE(` line at `:497` — already asserts on CPU that it reaches exactly `5 * num_hidden_layers` when the dispatcher does fall back. **And the counter is not the only process-wide static on this route, so this description should say what the row does NOT decline.** `dense_nvfp4::DenseMarlinWorkspace` (`dense_nvfp4_gemm.h:506`) is a process-static DEVICE allocation — `static void* ws` behind a `static std::mutex`, sized from `MarlinDeviceSms` and keyed on nothing, not even the device index — shared by every caller of `MatmulNvfp4MarlinD` (`:539`) and `GateUpFusedMarlinD` (`:700`). It has three consumers today (the shared dense route in `dense_attn_block.h`, MiniMax-H3, and the compressed-tensors NVFP4 scheme); NemotronH's `lm_head` becomes a fourth. It is PRE-EXISTING — it arrived at `80d1da096` and its definition is byte-identical at this row's merge base and at its head — and nothing here addresses it. In a change whose thesis is refusing to inherit the sibling process-static defect ([#984](#984)), saying nothing about the one it does inherit was an asymmetry a reviewer is entitled to see stated. The cost the fallback would carry is 198.18 MB per call — 176.16 MB of packed codes plus 22.02 MB of group scales — arithmetic that checks to the byte against the `[131072, 2688]` geometry. The behavioural red for this class needs CUDA and is PENDING with the rest. ### The Marlin arm builds here, and saying otherwise overstated the blocker The previous description said "this branch has not built the Marlin path at all". `include/vt/cuda/marlin_repack.h` includes only `<cstdint>`, `<cstddef>` and `<vector>`, so the host side needs no CUDA toolkit. On this box, which has no `nvcc`: ``` c++ -std=c++20 -I include -I src -isystem third_party -DVT_MARLIN_NVFP4=1 \ -Wall -Wextra -Werror -c -o nhd_marlin.o \ src/vllm/model_executor/models/nemotron_h_device.cpp -> rc 0, 0 errors, 0 warnings, a 1 268 552-byte object ``` **That byte count is anchored to the merge commit on this branch**, gcc 13.3.0, no `nvcc`, and compilation is deterministic here — two runs of the identical command, identical `sha256 42d670b6...`. The anchor is the correction. The figure this description carried until now, 1 269 696, was accurate when written and accurate at the reviewed head `29b1128e3`, but the translation unit moved twice afterwards — 1 272 808 at `fedf78d86`, 1 268 552 from `bff2b7b2f` on — and an evidence block naming no SHA cannot tell a reader which of the three it means. The measured series across seven commits is in the spec. The size is incidental either way; what the block asserts is `rc 0`, zero errors and zero warnings on a toolchain with no CUDA, and that holds at every one of those commits. Both arms are compiled and both are compiled `-Werror`. What genuinely needs `nvcc` is the Marlin **kernel** and every **execution** of the device path; those stay PENDING and are what the table below records. ### Reachability, and a CPU vehicle that already existed Both synthetic cases build `NemotronHHostWeights` by hand, which proves the class works and not that anything reaches it, and the deletion mutation for the device arm needs CUDA. But this row also refactored the part of `NemotronHPagedForward` that every CPU step runs, and `tests/vllm/models/test_nemotron_h_paged_forward.cpp` already drives that function through a real `GPUModelRunner`. It gains a case: three gathered rows against one request, so the returned row count must follow `logits_indices` and never `num_reqs`; the trace's copy of `final_normed` fed back through the production host entry point must reproduce the returned logits bit for bit; and the same logits must come back with the trace off. With `trace->capture` set, `DownloadF32(final_normed)` ran twice on the host arm. One download now serves both consumers on every path. That duplicate is not observable from outside the function — both copies are of the same unchanged buffer — so it is repaired structurally and no assertion pretends to catch it. ### Records `spec §4.3` did not exist; the requirement is under `## 5. Owed`. The allowlist's `## Gates` did not either; the heading is `## 3. The gate`. `docs/FEATURES.md` still listed the device `lm_head` as owed while `docs/USAGE.md` said it was device; both now say implemented and never executed, which is the honest state. The appended issue-index row for [#1410](#1410) cited `## Owed` where the heading is `## 5. Owed` — cosmetic, since the row passes on its owning-row disjunct, but the index is append-only and this is the only window to correct it. [#1421](#1421) is filed rather than fixed in flow: two NVFP4 W4A16 upstream anchors are wrong at the pin, verified against a checkout of `vllm-project/vllm` at `5559679229bc961848b121ccdeaa8fa5d79bec98`. `kernels/linear/__init__.py:879-881` is the tail of `init_wfp8_a16_linear_kernel`, an FP8 helper sharing the `a16` vocabulary, and the forced-Marlin line it quotes is at 922-924; `qwen3.py:271-274` is the `class Qwen3ForCausalLM(...)` line and `packed_modules_mapping` is at 275-278. Both predate the merge base, and both are cited from nine code and test sites over SEVEN files plus eight record sites over FIVE record files (`parity-ledger.md` carries three). The index row said "six files" while its own parenthetical enumerated seven, and "six record files" where there are five; `.agents/issue-index.md` is append-only and a landed row can never be edited, so an unmerged pull request is the only window to correct that, and this one does. The set still needs re-deriving at the pin rather than a sed. This branch relocates one of those citations and deliberately leaves its VALUE unchanged, so the issue owns every site uniformly instead of half of them looking reviewed. ## Gate status, stated as pending rather than as passed Every number here is from a run on a CPU box with no CUDA toolkit, `RelWithDebInfo`, `-Wall -Wextra -Werror`. **Read the `overlay` column first.** This tree cannot construct a `GPUModelRunner` at all — [#1371](#1371) throws there — so every green below was taken with [#1392](#1392 production fix applied to the working tree, never committed here and reverted byte-for-byte afterwards. The previous version of this table said "a run on this tree" and reported the §12 case at `run_rc 0 / SUCCESS!`, which is false of the tree it named: un-overlaid, that case is already red. | binary | overlay | `run_rc` | cases | assertions | verdict | |---|---|---|---|---|---| | `test_nemotron_h_moe_device` | none | 0 | 4, 4 passed | 4 | `SUCCESS!` — and **all four SKIP**, both A2-Q2b cases included | | `test_nemotron_h_paged_forward`, whole binary | **none** | 1 | 13, 2 passed, 11 failed | 18 | `FAILURE!` — [#1371](#1371), not this row | | `test_nemotron_h_paged_forward`, whole binary | **#1392** | 0 | 13, **13 passed** | 3269 | `SUCCESS!` | | the new §12 case alone | **none** | **1** | 1, **0 passed, 1 failed**, 12 skipped | **0** | **`FAILURE!` — throws #1371 before its first assertion** | | the new §12 case alone | **#1392** | 0 | 1, 1 passed, 12 skipped | 13 | `SUCCESS!` | **The `moe_device` row is not a pass, and it is the honest state of the synthetic gate.** The binary builds and exits 0, but every case takes the `TryCudaQueue` skip on a GPU-less box, so those 4 assertions are the skip notices themselves. The numeric gate examined nothing. **The paged-forward red is [#1371](#1371 All 11 failures throw the identical `No valid attention backend for device type 0 from {FLASH_ATTN: [head_size not supported]}` at `GPUModelRunner` construction, and 10 of them are cases this row never touched. Overlaying [#1392](#1392 production fix in the working tree — never committed here, reverted afterwards — turns the same binary green. Worth recording: doctest printed `assertions: 18 | 18 passed | 0 failed` while 11 cases were throwing, so the assertion line alone would have read as a pass. ### The red-first, on the cases that can run Each mutation went into a scratch copy of `nemotron_h_device.cpp`, was proven applied by `git diff --stat`, was built before being run (a mutation that fails to build proves nothing), and was restored to an identical sha256. **Every row below was also run with [#1392](#1392) overlaid**, and that is not a detail. Un-overlaid, the §12 case is already `run_rc=1` on the UNMUTATED tree, so an M1 or M2 red measured there would prove nothing whatever. The overlay is what lets the vehicle report a green in the first place; only then can a mutation take it away. | mutation | overlay | applied | `compile_rc` | `run_rc` | verdict | |---|---|---|---|---|---| | — (unmutated control) | **#1392** | — | 0 | 0 | **GREEN**, `SUCCESS!`, 13 assertions | | — (unmutated control) | **none** | — | 0 | **1** | already **RED** — #1371, which is why the rest is overlaid | | **M1** — `n_out` -> `R` at the host projection | **#1392** | 1 ins / 1 del | 0 | **1** | **RED**, `FAILURE!` | | **M2** — never fill `trace->final_normed` | **#1392** | 1 ins / 2 del | 0 | **1** | **RED**, `FAILURE!` | | **M4** — restore the exact pre-repair two-download shape | **#1392** | 3 ins / 8 del | 0 | 0 | **GREEN — reported, not hidden** | Re-derived rather than restated, on a clean tree at the merge commit, overlay and mutations reverted to byte-identical sources (`nemotron_h_device.cpp` back to `sha256 abf6e21f...`): §12 alone un-overlaid gives `run_rc=1`, `1 | 0 passed | 1 failed | 12 skipped`, `assertions: 0`, `THREW exception: No valid attention backend for device type 0 from {FLASH_ATTN: [head_size not supported]}`; overlaid, `run_rc=0`, 1 passed, 13 assertions; the whole binary overlaid, 13/13, 3269 assertions; M1 `compile_rc=0 run_rc=1` throwing `gathered row count does not match hidden_size` at `nemotron_h.cpp:1029`; M2 `compile_rc=0 run_rc=1` on `REQUIRE( 0 == 288 )`. M4's cell needs no separate attestation — un-overlaid every run of this case is red, so a GREEN is only reachable with the overlay and its verdict entails the column. M1 is the red the `n_out` rename exists for: with the request count substituted, the returned row count is 1 where the gather asked for 3. M2 arms the trace-operand assertion. Both showed the same trap — doctest printed `assertions: 2 | 2 passed | 0 failed` on M1 while the case was failing, because a `REQUIRE` throws rather than counting. M4 staying green is a result, not a gap. The duplicate `DownloadF32` copies the same unchanged buffer twice and produces identical bytes, so nothing observable from outside the function can distinguish it. It is repaired structurally and no assertion pretends to catch it. **The device arm's own red-first does not exist on a CPU box and is not claimed.** `MarlinW4A16Selects` is false on a CPU queue, so the device branch, the `fallback_gemms` assertion and the reachability deletion mutation are all unreachable here. ### A gate this row was already failing, and cannot repair in place `scripts/check-doc-checkpoint.py` is red on this branch, and it was red at the reviewed head `29b1128e` with the identical two errors (`--base 96ed834 --head 29b1128`): ``` ERROR: commit 1c62d99: changed user_usage but did not update docs/USAGE.md ERROR: commit 8fa900a: changed .agents/benchmark-record.md: measurement recorded but did not update docs/STATUS.md ``` The previous description said this gate had one real failure and that it was repaired. It was not, and `.github/workflows/ci.yml:519` runs the same `--base/--head` invocation, so the lane is red for this reason independently of [#1371](#1371). The second error names a real gap and it is now closed: this row moves a lifecycle state and records a measurement, and `docs/STATUS.md` said neither. It does now, in 192 characters, because that page carries a shrink-only ratchet on oversized cells and the first attempt took it from 44 to 45. The first error, and the historical form of the second, cannot be closed by a later commit. The checker iterates `commits_in_range` and judges each commit on its own contents, so the obligation belongs to `1c62d9974` and `8fa900a62`. Discharging it means rewriting commits that are already the reviewed base, which resets this pull request's CI approval and moves the head a fresh reviewer was asked to look at. **That is a scheduling decision, so it is recorded as owed and raised for the operator rather than taken by a repair pass.** Run over this repair's own commits (`origin/main..HEAD`), the checker reports these two and nothing else. ### The spec sent the next GPU window through the wrong mutex `§4` step 1 read "Take `$GPU_LOCK` with a blocking `flock` and wait". `AGENTS.md` requires a fleet device to be claimed through `rc`, and `dgx:gpu0` is one. The text predates that rule, but this branch is the wrong place to leave it standing: it ADDS `scripts/nemotron-h-a2q2b-gpu-gate.sh` for exactly that window, the script correctly takes no mutex of its own, and the contradicting instruction sat beside it in the same file — and every PENDING leg above is waiting on that window, so the next reader of `§4` is the person about to open it. Two mutexes that do not exclude each other are worse than one: the fleet cannot see `$GPU_LOCK`, so a `flock` over `ssh` does not exclude a concurrent `rc` holder and the controller reports the box free while somebody is on it. On 2026-08-17 that pair voided a whole speed axis (`minimax-music3.md` §13.10). Step 1 now claims through `rc run -d dgx:gpu0 --max-runtime <N>h --` with the gate script as the payload, and step 2's headroom check moves inside the lease and names the script's own PRECONDITION 1. ### The trailer gates had stopped running At the previous head, `origin/main` was not an ancestor, so `scripts/agent-preflight.sh --fail-on-skip` took its `TRAILER_BEHIND` arm and SKIPPED both `commit-trailers` and `commit-style`. A skip is not a green. **It then happened five more times during the device-leakage repair**, because `origin/main` moved under it at `9ecaf1bb3`, `f07f96e1c`, `aeba0de6f`, `01854663c` and `c8d926ea8`. The ref only moves on a fetch, so the fix is ordering: merge, then do not fetch again, then gate, then push. At the pushed head both gates RUN: `BASE_SHA=c8d926ea8` non-empty, `ANCESTRY_STATUS=0`, `RANGE_COUNT=22`, `RANGE_STATUS=0`, **zero `SKIP` tokens anywhere in the report**, 85 gates `ok`, and the only failure is `doc-checkpoint range` with the two commits above. The skip arms are falsified individually rather than read off a green line, because `agent-preflight.sh` prints a summary only on failure. Each is armed over exactly this range, proven by a detached scratch commit with a period-terminated subject and no trailer block, which turns `check-commit-style.py` and `check-commit-trailers.py` red with the specific messages. Both files that both sides of the merge touched are records, resolved by shape rather than by whatever the three-way merge produced. `.agents/issue-index.md` is a genuine append-only log carrying `merge=union`, and both sides append at the tail, so all four new rows survive — verified additive, 469 + 2 lines on each side to 471, no line removed or altered. `docs/USAGE.md` is a keyed record: the merged file is byte-identical to `origin/main`'s version with this branch's scoped edit re-applied, so every key neither side owns is unchanged. ### The DSR ratchet was red, and no review round ever saw it run `device-leakage` never COMPLETED while this pull request was under review, so its verdict was an input to none of the three passes. It completed after the third and failed: `vt_ifdef` **35 against a baseline of 32**, `rc 1`. Three `#ifdef VT_MARLIN_NVFP4` sites had been added to the device-agnostic shared layer, which is exactly what the ratchet exists to stop. They were located by running `scripts/check-device-leakage.py --report` at the failing head `7a3909187` and diffing the per-file table against `9ecaf1bb3`, rather than by reading the diff for guards. | # | site at `7a3909187` | what the guard decided | resolution | |---|---|---|---| | 1 | `include/vllm/model_executor/models/dense_nvfp4_gemm.h:768` — inside `MarlinW4A16Selects` | **nothing** | **removed** | | 2 | `src/vllm/model_executor/models/nemotron_h_device.cpp:883` — around `LmHeadNvfp4View` | **nothing** | **removed** | | 3 | `src/vllm/model_executor/models/nemotron_h_device.cpp:985` — `DeviceLmHeadD`'s body | `ResidentIn` and a complete `dense_nvfp4::MarlinDenseResident` | **`DSR-ALLOW(A2-Q2b)`** | **No baseline was changed.** `AGENTS.md` forbids making a red gate green by widening an assertion, and a baseline bump is that. `scripts/device-leakage-baseline.json` is untouched at 32, and the per-file table is now byte-identical to `origin/main`'s. **(1) is the case the checker's own message describes.** `MarlinW4A16Selects` is a SELECTION wearing a build guard, and every term it reads exists in every build: `MarlinW4A16Enabled()` is declared above the guarded region, and `vt::OpRegistered` is the op/provider table's own answer to whether the Marlin arm is realized for a device. The flag and the registration are one condition, not two — `CMakeLists.txt`'s single `if(VLLM_CPP_MARLIN)` block adds `src/vt/cuda/cuda_moe_marlin.cu`, whose file-scope `Registrar` holds the tree's only `RegisterOp(OpId::kMoeGroupedGemmNvfp4Marlin, …)`, and defines `VT_MARLIN_NVFP4=1` in that same block. A build without the macro registers nothing, so the query already resolves false on exactly the builds the `#ifdef` excluded. This is the call `nemotron_h_device.cpp`'s `moe_on_device` selection had already made, in a comment that says so. **(2) was measured rather than reasoned.** `LmHeadNvfp4View` names nothing the Marlin build adds — `Nvfp4Weight` comes from `qwen3_5_weights.h` and `OwnedBytes` from the loader, both unconditional, and no `vt::cuda::` symbol appears in it. The claim that its external linkage at `namespace vllm` scope is what makes an unused definition harmless where its only call site is compiled out is proven by the mutation that removes that property: adding `static` turns the same CPU compile RED at `rc 1`, `error: 'vllm::Nvfp4Weight vllm::LmHeadNvfp4View(...)' defined but not used [-Werror=unused-function]`. The file was restored to an identical sha256 (`9719ea70…`) afterwards. **(3) is TYPES-not-behaviour and takes the checker's documented escape hatch.** `DSR-ALLOW` is not a baseline change: the site is excluded from the count but COUNTED AND PRINTED on every run, so the exemption is visible in CI output rather than invisible in the diff. `DeviceLmHeadD`'s body names two symbols that do not EXIST without the guarded arena region — the `ResidentIn` template, defined inside it, and `dense_nvfp4::MarlinDenseResident`, which the header declares unconditionally and defines only under `VT_MARLIN_NVFP4`, so the reference cannot bind to an incomplete type. It is the same class and the same stated reason as the five sibling guards A2-Q2a and A2-P already carry in this file, and its `#else` refuses by name. The SELECTION for this arm stays a runtime op-table query. **Measured on this tree**, a CPU build with `VT_MARLIN_NVFP4` absent from `build/compile_commands.json` — positive control: 1020 `VLLM_CPP` hits in the same file, so the grep is not silently wrong — which is the configuration that exercises both removals, because it is the arm the deleted `#else` branches used to serve. | what | before (`7a3909187`) | after | `rc` | |---|---|---|---| | `check-device-leakage.py` `vt_ifdef` | 35 | **32** | 1 → **0** | | `DSR-ALLOW` exemptions in force | 20 | **21** | — | | `scripts/device-leakage-baseline.json` | 32 | **32, untouched** | — | | per-file table vs `origin/main` | +1 header, +2 model TU | **identical** | — | | `nemotron_h_device.cpp`, `nemotron_h.cpp`, `qwen3_5.cpp` at `-Wall -Wextra -Werror` | — | compile | **0** | Each repair is individually load-bearing, proven by reverting it alone in a scratch worktree and re-running the gate. Every mutation was verified applied by `git diff --stat` and restored to an identical sha256, with the unmutated control green immediately before and after. | mutation | applied | `vt_ifdef` | `rc` | verdict | |---|---|---|---|---| | — (control) | — | 32 | 0 | `ratchet holds` | | **M-B** — restore the guard on `MarlinW4A16Selects` | 4 ins | **33** | **1** | **RED**, `DSR REGRESSION` | | **M-C** — restore the guard around `LmHeadNvfp4View` | 2 ins | **33** | **1** | **RED**, `DSR REGRESSION` | | **M-D** — delete the `DSR-ALLOW(A2-Q2b)` line | 1 del | **33** | **1** | **RED**, `DSR REGRESSION` | | — (control, after restore) | — | 32 | 0 | `ratchet holds` | The two pre-existing allowlist entries the report also prints, `deepseek_v4_device.cpp [kcuda] x8` and `platform.cpp [dev_cast] x1`, are byte-identical at `9ecaf1bb3` and here. This change moves one bucket and nothing else. **`docs/USAGE.md` rides in the same commit, because it has to and because it is true.** Any commit touching `include/vllm/` is `user_usage` to `scripts/check-doc-checkpoint.py` and owes the surface in that same commit. `doc-checkpoint range` is already red on this branch for `1c62d9974` and `8fa900a62`, which an operator decision covers; a third error of the identical kind would be new damage rather than inherited, so the header hunk was recommitted with the USAGE edit beside it. The edit is not written to feed the gate: the NemotronH arms table enumerates what selects the host `lm_head` projection and omitted `VT_NVFP4_MARLIN=0`, a user-settable knob that selects an arm and appears nowhere else in USAGE, and this change is precisely what makes that knob reach the model's eligibility test in every build rather than only where the guard compiled it in. `check-doc-checkpoint.py --commit` is `OK` on that commit, with the still-red `1c62d9974` as the positive control that the checker is armed. **`origin/main` moved four times during this repair** — `9ecaf1bb3`, `f07f96e1c`, `aeba0de6f`, `01854663c` — and each time `git merge-base --is-ancestor` answered 1, which is the arm that SKIPS both trailer gates without failing. Each was merged rather than worked around, and every auto-merged keyed record was resolved by key on every merge: the branch delta across the merge byte-identical to the branch delta from the merge base, checked file by file with a positive control on the comparison itself. ### The three red CPU jobs were inherited, and the merge cleared them `build-test-cpu` and both `sanitize-cpu` arms failed on this pull request, all three on `test_nemotron_h_paged_forward` — [#1371](#1371), which [#1392](#1392) fixed on `main` after the review rounds. `sanitize-cpu` is NOT a no-baseline job, so that had to be triaged by WHICH test failed rather than assumed. The prediction is verified rather than asserted. With `origin/main` merged, the same binary is green on this tree with NO overlay: `13 | 13 passed | 0 failed`, `assertions: 3269 | 3269 passed | 0 failed`, `Status: SUCCESS!`, `run_rc 0` — the same numbers the spec's tables previously recorded only WITH #1392 applied to the working tree and reverted. **The `#1392` overlay is therefore retired**; the historical rows in the spec keep their `overlay` cells, because they describe the tree they were measured on and rewriting them would make them false. ### `main` brought four red suites with it, and they are attributed both ways The third sync carried `4712dac40` (`VT-ACT-ROUND-POLARITY`, [#1322](#1322) via [#1347](#1347)), and with it four red CPU suites this branch does not touch. Both `ctest` runs are on this box, no overlay of any kind. | tree | base | `ctest` | result | |---|---|---|---| | this row + `9ecaf1bb3` | before `4712dac40` | 567 | **`100% tests passed, 0 tests failed out of 567`**, `rc 0` | | this row + `01854663c` | after `4712dac40` | 569 | `rc 8`, **4 failed**: `test_minimax_music3_ar`, `test_ltx2_text_encoder`, `test_muse_glimmer_text`, `test_muse_glimmer_text_fallback` | Those four are [#1458](#1458), filed from another flow before this control was run. Inherited, not caused, and proven in both directions on the same tree rather than argued — each mutation verified applied by `git diff --stat` and restored to an identical sha256: | control | change | `compile_rc` | `run_rc` (music3 / ltx2 / glimmer) | verdict | |---|---|---|---|---| | **A** | revert THIS row's two source files to their pre-repair `7a3909187` content | 0 | **1 / 1 / 1** | still RED — **not this row's** | | **B** | revert `src/vt/cpu/cpu_ops.cpp` alone to `4712dac40^` | 0 | **0 / 0 / 0**, `37/37`, `27/27`, `24/24` | GREEN — **`4712dac40` is the cause** | Control A answers the attribution question on its own; control B is here because naming a cause is more useful to the next reader than clearing oneself. Neither repairs anything, deliberately: #1458 needs `VT-ACT-ROUND-POLARITY` to decide whether its kernel or four bf16 error floors that were never re-derived are the wrong side, which is that row's oracle work rather than a small and clear in-flow fix. **That window has since closed.** [#1458](#1458) was fixed on `main` and closed, and this branch has merged the repair, so the four are not expected on the current head. The controls stay recorded because a red that was attributed and then disappeared is still the reason two `ctest` runs in this description disagree, and deleting the evidence would leave that unexplained. ### `documentation-checkpoint` is the same two commits, and it is not new It went red for the same reason `device-leakage` did: it had never COMPLETED during review. Its whole failure set is `1c62d9974` and `8fa900a62`, the two an operator decision covers and which cannot be repaired in place — `check-doc-checkpoint.py` iterates `commits_in_range` and judges each commit on its own contents, so a later commit cannot close either, and discharging them means rewriting the reviewed base. The job's three invocations were reproduced locally at this head with CI's own arguments (`--base` the PR base `63d87805c`, `--head` HEAD): | invocation | result | |---|---| | `check-doc-checkpoint.py` | `rc 1` — **exactly `1c62d9974` and `8fa900a62`, nothing else** | | `check-now-current.py` | `rc 0`, `OK: .agents/NOW.md is a current, in-budget resume digest` | | `check-role-discipline.py` | `rc 0`, `OK: every change on main arrived on a task branch` | **This repair's own commits are clean, and that is not an accident.** The first draft of the header fix added a THIRD error of the identical kind, and it is the reason `docs/USAGE.md` rides in the same commit as the `include/vllm/` hunk rather than in a follow-up: the checker's obligation is per-commit, so a follow-up cannot discharge it. `check-doc-checkpoint.py --commit 65ab066` is `OK`, with the still-red `1c62d9974` as the positive control that the checker is armed against exactly that class. ### Nothing ran at all for a whole push, and the cause was a conflict rather than a queue The push before this one produced **zero check runs** on `2c63cc87a` while other pull requests were starting normally. `gh pr view` named it: `mergeable=CONFLICTING`, `mergeStateStatus=DIRTY`. GitHub could not build `refs/pull/1415/merge`, so no `pull_request` workflow had anything to run against. A healthy Actions queue plus a branch with zero runs is the shape a conflict makes, and reading it as a slow queue would have cost the whole verdict. The conflict was `docs/STATUS.md` and `docs/USAGE.md`, and neither was resolved by picking a side, because both had been **relocated** on `main`. `USAGE.md` is now a 231-line hub whose per-model content moved under `docs/models/`, and `STATUS.md` is a 27-row surface summary with no per-model rows at all. So both were taken WHOLE from `main` and this row's scoped edits re-applied BY KEY at their new home, `docs/models/nemotron-3-5-lightning.md` — 22 insertions and exactly ONE deletion there, every unrelated key byte-identical. One of this row's edits is **dropped rather than re-applied**, deliberately, and the other was never dropped at all — the claim that both were is corrected below. The `STATUS.md` Nemotron row is genuinely gone: merged `main`'s `STATUS.md` is a 98-line surface summary that enumerates no model at all. `Nemotron` returns 0 hits there, and so do the positive controls `Kimi` and `Laguna`, while the file plainly greps. Re-adding it would re-create a row a relocation removed, which is the duplicate a second relocation makes. The `USAGE.md` `NemotronHForCausalLM` refusal row is the opposite case, and the earlier claim that it "no longer exists anywhere under `docs/`" was **wrong**: `aee6c48d6` (#1491) did not delete that row, it **RELOCATED** it to `docs/reference/model-loading.md` — a file that did not exist at this branch's pre-merge base, which is precisely why the earlier pass read it as gone. It is still on `main`, and this row now updates it. `docs/FEATURES.md` auto-merged and kept this row's cell — checked, not assumed. `check-conflict-markers.py` (#1450, new on `main`) reports 0 findings over 3820 tracked text files. Run locally from the ORIGINAL merge base, `check-doc-checkpoint.py` also reports `d995c52f0`, `af25bd251` and `995ed1ccd` — each verified an ancestor of `origin/main`, none in this branch's own range, all arrived with the merge. CI does not see those, because it resolves `--base` to the CURRENT merge base: **its whole failure set is `1c62d9974` and `8fa900a62`**, the two an operator decision covers, and nothing else. This row's own commits stay clean either way: `check-doc-checkpoint.py --commit` is `OK` on `65ab06636` and on the merge itself. ### Two public rows still said the head runs on the host, and this change is what made them false Following the relocation above to its consequence: `docs/reference/model-loading.md:188` still read "`lm_head` and FP8 Mamba2 projections run on the host". That sentence was true before this row and is false after it, which is the test for whose obligation it is. Left alone, the tree would land with `docs/models/nemotron-3-5-lightning.md` calling the head a device arm and this row calling it a host arm, on the same merge commit. Only the clause this row falsified moves. `lm_head` reaches the device **on the paged forward** and is carried as implemented-and-unmeasured, matching what the model recipe and the spec's `## Now` already say. The FP8 Mamba2 projections are a **different unit** (A2-Q1, #1289, still held DRAFT) and still run on the host, so that clause is kept and only re-worded to stop sharing a verb with the head. The pending-token-gate and no-GGUF clauses are untouched, because both are still accurate. Sweeping by CLAIM rather than by file, at the MERGED tree, found a **second** site the scoped review did not reach, because it is not a refusal row. The checkpoint registry at `docs/USAGE.md:230` lists per-checkpoint arms, and its "Supported arms" cell read "host FP8 Mamba2 and NVFP4 head". That "NVFP4 head" is this same `lm_head`, so the same test applies and the same one clause moves. That cell is the surface the "say which weights, and from where" rule owns, so a supported-arms cell putting the head on the host is exactly the contradiction this finding is about. The sweep carried its own controls, because a null grep only proves the terms wrong. Every `docs/**.md` and `README.md` line asserting a host placement for a head or for logits is 11 hits, 2 of them the sites repaired here (positive control); the same pipeline with a nonsense term returns 0 (negative control). The other 9 were read rather than pattern-matched, and none is stale: `BENCHMARKS.md:16` is this row's own host-re-expansion ATTRIBUTION and is what motivates the device arm; `BENCHMARKS.md:276` is Laguna GB10 ATS weight residency; `ENVIRONMENT.md:144,187,212` are Qwen3.5/Qwen3.6 levers; `server.md:88` is the CPU-only `prompt_logprobs` full-logits route on the shared runner, not this model; `model-loading.md:144` is the FP8 scalar-scale guard; and two are historical `docs/superpowers/plans/` documents, which are not projections. **No gate can catch either repair.** `check-doc-checkpoint.py` asks whether SOME `docs/USAGE.md` edit accompanied a commit of the changed class. It never asks whether a sentence inside it is still true. ### Every check has COMPLETED on the current head, and the remaining reds are all inherited `ci` run `32537115264` on `99f9f672a`: **25 checks, 0 pending** — 14 pass, 6 fail, 5 skipped by design on a pull request (`attest`, `baseline-summary`, `manifest`, `promote`, `publish`). | check | on `b7d89b43e` | on `99f9f672a` | whose | |---|---|---|---| | **`device-leakage`** | SUCCESS | **SUCCESS** | **this row's.** The restructure survives the 38-commit merge at `vt_ifdef` 32, and `scripts/device-leakage-baseline.json` is byte-identical to `main`, so the row adds no new allowance | | `build-newest-gcc` | FAILURE | **SUCCESS** | was **`main`'s**, and it cleared exactly as predicted once `main` carrying #1581 was merged. CI builds the MERGE commit, so the red came from the base side; #1565 is closed | | `build-test-cpu`, `sanitize-cpu (address,undefined)`, `sanitize-cpu (thread)` | SUCCESS | **FAILURE** | **`main`'s**, [#1608](#1608) and [#1602](#1602) — see below | | `build-test-vulkan`, `build-test-cpu-arm64`, `cuda-fat-build`, `cuda-arch-features`, `vulkan-spirv-freshness`, `agent-record`, `pr-size`, `commit-protocol-tag`, `last-gated-commit`, `plan`, `verify` x2 | SUCCESS | SUCCESS | — | | `documentation-checkpoint` | FAILURE | FAILURE | `1c62d9974` + `8fa900a62` ONLY — the two an operator decision covers, verified against the job log rather than assumed | | `windows-msvc-cpu`, `windows-msvc-vulkan` | FAILURE | FAILURE | the two no-baseline PR-only jobs, [#584](#584) | The three CPU legs all fail on **`282 - test_runner`**, and the failure is inherited rather than this row's. The failing case, `runner: initialize_kv_cache refuses a non-multiple-of-16 block size`, has **0** occurrences at `b7d89b43e` — the head where all three legs were SUCCESS — and **1** at the merge base `5453e571d`. It was introduced by `e2a9e035d` (#1273), which **is** among the 38 commits the merge brought in, and this row's range touches **0** files in that test (control: it does touch the two `tests/vllm/models/test_nemotron_h_*.cpp` files, so the query discriminates). It is also **not a data race**, which matters because `sanitize-cpu (thread)` is the lane that would expose one. All three legs report the SAME deterministic assertion — `test_runner.cpp:1557`, `20 test cases / 19 passed / 1 failed`, `544 assertions / 543 passed / 1 failed` — and **0** sanitizer findings in any of them (`WARNING: ThreadSanitizer` = 0, `ERROR: AddressSanitizer` / `runtime error:` = 0). A race cannot make a non-sanitized `build-test-cpu` fail identically. The assertion expects `"Block size must be a multiple of 16"` but the attention registry refuses first, with `"No valid attention backend for device type 0 from {CPU_ATTN: [block_size not supported], FLASH_ATTN: [block_size not supported]}"` — which is precisely what #1602 describes. Nothing is suppressed here and nothing is pragma'd, because there is nothing of this row's to suppress. ### Gate table | leg | state | |---|---| | seam extension, device arm, production wiring | DONE; built on BOTH arms, `-Werror`, 0 warnings | | the CPU-reachable half, gated through a real `GPUModelRunner` | DONE and RUN, with M1/M2 red-first | | routing allowlist narrowed, [#1410](#1410) filed | DONE | | synthetic device `lm_head` numeric gate | COMPILES; CUDA-only; **NEVER RUN** | | `nvcc` build of the Marlin kernel, real-checkpoint numeric leg, token identity, reachability deletion mutation, the `VT_NVFP4_MARLIN=0` fallback red | PENDING a `dgx:gpu0` window, job queued | A reviewer should treat every CUDA leg as unrun. The spec's `## 6. Now` carries the same tables, so the pending state lives with the row rather than only in this description. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

mudler added 4 commits August 19, 2026 11:30

localai-bot merged commit 4712dac into main Aug 20, 2026
0 of 15 checks passed

localai-bot mentioned this pull request Aug 20, 2026

MiniMax-Music3: the five changes main took on 2026-08-19/20 have NO end-to-end number — the depth device arm, the vocoder conv tiling and the conv f32 accumulator were each gated and none was timed on a real request (#672) #1512

Open

localai-bot mentioned this pull request Aug 21, 2026

feat(A2-Q2b): lm_head reaches the device on NVFP4, and the seam stops handing out an address-keyed cache #1415

Merged

Navigation Menu

fix(VT-ACT-ROUND-POLARITY): narrow act(gate) to the input dtype, which upstream pins bit-exactly (#1322) - #1347

fix(VT-ACT-ROUND-POLARITY): narrow act(gate) to the input dtype, which upstream pins bit-exactly (#1322)#1347
localai-bot merged 4 commits into
mainfrom
row/VT-ACT-ROUND-POLARITY

localai-bot commented Aug 19, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

localai-bot commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The premise was re-verified before any code was written

Why this is safe

Evidence

Scope, and what this does NOT claim

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

localai-bot commented Aug 19, 2026 •

edited

Loading