record(quant): QUANT-CT-MXFP4-CLOSERS binding verdict — slivers improve EVERY tput axis vs #49, c1 crosses to parity+; goal still <1.0x on c2-c8

Clean-checkout binding grid at the committed sha d3b412f (not an overlay) via
mxfp4-online-serving-grid.sh, oracle vLLM 0.25.0 (FlashInfer mxfp4 disabled),
single-load/arm, drop_caches+mincore. #44 model gate re-passed inside the grid;
24/24 legs failed:0; reps tight (ours c8 CoV ~0.4%). Grid EXIT=1 is the
gate-FAIL signal (gate_pass:false) + single-model cross-summary wait, not a crash.

Binding (ours/vLLM normalized ratio) vs #49 tput [0.990/0.922/0.930/0.942]:
  total_token_throughput  c1 1.005  c2 0.925  c4 0.939  c8 0.953
  median_tpot_ms          c1 1.002  c2 0.922  c4 0.915  c8 0.939
  mean_ttft_ms            c1 1.034  c2 0.962  c4 1.004  c8 0.999
  median_itl_ms           c1 1.004  c2 0.919  c4 0.920  c8 0.929
  peak host-mem           ours 35.2 GiB vs vLLM 76.7 GiB = 2.18x LESS

VERDICT: c1 PASSES every axis; c2-c8 BELOW on tput/tpot/itl (best c8 0.953),
TTFT parity c4/c8. gate_pass FALSE => MXFP4 parity goal NOT DONE (below-floor on
c2-c8). The byte-exact slivers moved EVERY tput axis up vs #49 (c1 +1.5pp crosses
to parity+, c4 +0.9pp, c8 +1.1pp, c2 +0.3pp), no regression.

Residual map (c8 ~4.75% gap): (1) block=16 padding CLOSED (sliver a); (2)
grouped-Marlin decode +7-9% per-call (E=1 indirect sorted_token_ids gather + fp32
C_tmp vs vLLM dense marlin_gemm direct-A, #46/#50); (3) host slice. Host-slice
attribution (step 2, VT_LOOP_TRACE under decode): every window interval_ms ≈
step_ms (delta ≤0.02ms; admits=0 pure-decode windows interval-step ≈ 0), so the
born-on-runner engine-core loop has negligible host overhead — the #47 ~0.7ms/step
residual is NOT an engine-loop lever (it is the shared async frontend or the
cross-tool attribution boundary). Records: STATUS, BENCHMARKS, benchmark-record,
state, NOW.

FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]

Author: mudler

.agents/NOW.md

@@ -22,9 +22,9 @@ checkpoint on `upstream/main` at `59674cf1d`.
 | Kimi-Linear-48B (KDA+NoPE-MLA+MoE) | **Full-model GB10 e2e RUNS** (bf16-resident §13): CPU+CUDA 13/13·656, no OOM. **Token gate NEAR-TIE 106/128** (6/8 token-exact) | device GDN/MLA islands + bf16 stream; 1.59 tok/s; default OFF |
 | 35B fresh grid | **BOUND** @`1ea26427`: tput 0.93-1.03x, c16 0.93x. INTAKE + Option A both **RESOLVED NEGATIVE** (H2D-out-of-capture tput WASH) | Real lever left: prefill glue (task #61) |
 | Qwen3.5-4B revalidation | 0.9971x @`59674cf1` (#35); TTFT/PSS pass, TPOT/ITL open | `docs/bench-evidence/` |
-| MXFP4 parity (Qwen3-8B) | c1 0.990/mem 2.6x WIN; c2-c8 0.92-0.94 (#49). **Closers LANDED byte-exact default-ON (`QUANT-CT-MXFP4-CLOSERS`): dense M≤8 8-row Marlin tile + per-call ws re-zero dropped; block8≡block16 bitdiff=0, memcheck clean, #44 smoke 3/3** | Binding grid + host nsys pending |
+| MXFP4 parity (Qwen3-8B) | **Closers binding x3 (`QUANT-CT-MXFP4-CLOSERS`): c1 1.005 PASSES, c2-c8 0.925/0.939/0.953 BELOW (c8 +1.1pp vs #49), mem 2.18x WIN, gate NO. Byte-exact bitdiff=0, #44 3/3** | Residual: grouped→dense-direct-A; host slice NOT engine-loop |
 | ROW-SERVE-ASYNC-DENSE-MIRROR | **LANDED+dgx-VERIFIED** (`f9c969ae`): #31 async mirror on classic dense Qwen3; gate RED→GREEN, SACRED 184/184 | Residual: sibling scope one-liner |
-| MXFP4 parity goal | swap-ON binding <1.0x (best c8 0.942); closers byte-exact default-ON; residual = ~0.7ms host/sched (substantive) + grouped-Marlin +7-9% | NEXT: binding x3 verdict + host nsys |
+| MXFP4 parity goal | binding <1.0x on c2-c8 (best c8 0.953); c1 crosses to parity+. Residual = grouped-Marlin +7-9% (only real lever) + host slice (VT_LOOP_TRACE: NOT engine-loop) | NEXT: grouped→dense-direct-A E1 marlin |
 
 In-flight branches (gated default-OFF, not pushed): `laguna-fp4proj-prod`
 (fp4 opt-in), laguna bf16/legacy/pipeline-gemv, `ds4-hc-expand-fuse`.

.agents/benchmark-record.md

@@ -12576,3 +12576,42 @@ unconditional (no `VT_*` gate). Blast radius (header consumers Qwen3-8B-MXFP4, Q
 Laguna) closed by proving BOTH quant schemes byte-exact. Binding c1-c8 x3 (clean-checkout grid) + the
 substantive ~0.7ms/step host/sched slice (the #47 residual) remain the parity verdict's open terms.
 Box left clean (both locks free, GPU idle, worker down, disk 21G, tmux gone).
+
+## QUANT-CT-MXFP4-CLOSERS BINDING: clean-checkout grid at d3b412f5 — slivers improve EVERY throughput axis vs #49 (c1 crosses to parity+); MXFP4 goal still <1.0x on c2-c8 (2026-08-09, GB10, vLLM oracle 0.25.0, evidence `dgx:~/work/vllm.cpp-online-gate/evidence/d3b412f5c191aace1f2960fa7940d8eef925762a`)
+Full c1/c2/c4/c8 x3 binding on the CLEAN-checkout build at the committed sha d3b412f5 (not an
+overlay), oracle `VLLM_DISABLED_KERNELS=FlashInferMxFp4LinearKernel`, single-load/arm,
+drop_caches+mincore, RelWithDebInfo+oracle-cutlass. #44 model gate re-passed inside the grid
+(mxfp4_smoke_battery). 24/24 legs `failed:0`; reps tight (ours c8 [1481.7,1475.1,1486.7] tok/s,
+CoV ~0.4%; vllm c8 CoV ~0.06%). The grid EXIT=1 is the gate-FAIL signal (`gate_pass:false`) + the
+single-model "cross-model summary waits for the other model", NOT a crash — all q3mxfp4 artifacts
+are complete. (q3mxfp4's sweep IS c1-c8, matching #48/#49; c16/c32 are other-model points.)
+
+| axis (ours/vLLM normalized ratio) | c1 | c2 | c4 | c8 | vs #49 (tput) |
+|---|---|---|---|---|---|
+| total_token_throughput | **1.005** | 0.925 | 0.939 | **0.953** | +0.015/+0.003/+0.009/+0.011 |
+| median_tpot_ms         | 1.002 | 0.922 | 0.915 | 0.939 | — |
+| mean_ttft_ms           | 1.034 | 0.962 | 1.004 | 0.999 | — |
+| median_itl_ms          | 1.004 | 0.919 | 0.920 | 0.929 | — |
+
+Peak host-mem footprint (peak_mem_available_drop): ours **35.2 GiB** vs vLLM **76.7 GiB** = **2.18x
+LESS** (WIN). VERDICT: **c1 PASSES every axis (tput 1.005, tpot 1.002, ttft 1.034, itl 1.004)**;
+c2-c8 BELOW on tput/tpot/itl (best c8 0.953 tput), TTFT at parity c4/c8 (1.004/0.999). gate_pass
+FALSE => MXFP4 parity goal **NOT DONE** (below-floor on c2-c8). The slivers moved EVERY throughput
+axis up vs #49 (c1 +1.5pp crosses to parity+, c4 +0.9pp, c8 +1.1pp, c2 +0.3pp), exactly the
+byte-exact block=8 recovery (~0.8pp @c8, #50) plus the memset-drop; nothing regressed.
+
+RESIDUAL MAP (per-shape, c8 ~4.75% tput gap): (1) block=16 padding — **CLOSED this run** (sliver a).
+(2) grouped-Marlin decode **+7-9% per-call** (GPU): `MoeGroupedGemmNvfp4Marlin` E=1 indirect
+`sorted_token_ids` gather + fp32 `C_tmp` vs vLLM dense `marlin_gemm` direct-A (#46/#50; a delicate
+grouped->dense-direct-A port, per-shape parity at M<=8 so not a config lever). (3) the host slice.
+
+HOST-SLICE ATTRIBUTION (step 2, VT_LOOP_TRACE on our server under decode load, this session): in
+every 1 s window the engine-core loop shows **interval_ms ≈ step_ms** (mean 25.5 vs 25.5, delta
+<=0.02 ms; `admits=0` pure-decode windows show interval-step ≈ 0). So the born-on-runner
+engine-core decode loop (scheduler + drain + admit) carries **negligible** per-iteration host
+overhead — the #47 ~0.7ms/step "host/sched" residual is NOT in the engine loop; it lives in the
+shared-architecture async frontend (HTTP / output-processing / detokenize, which vLLM runs too) or
+within the cross-tool attribution boundary (~0.7ms is ~24% of the c2 gap, near measurement error).
+Not a born-on-runner lever. (Caveat: the curl load under-saturated vs steady c8, so step_ms 25.5
+is not the c8 TPOT 37.6; the interval≈step finding is batch-independent and robust.)
+Box left clean (both locks free, GPU idle, worker down, disk 21G, tmux gone).

.agents/state.md

@@ -37144,3 +37144,39 @@ UNADDRESSED. So the honest projection stays <1.0x on c2-c8 (best c8 ~0.95x); the
 binding x3 confirms + attributes. NEXT: clean-checkout binding grid
 (`mxfp4-online-serving-grid.sh`) + host-slice nsys.
 
+## QUANT-CT-MXFP4-CLOSERS BINDING + host-slice attribution: slivers improve EVERY throughput axis vs #49 (c1 crosses to parity+); MXFP4 goal still <1.0x on c2-c8 → NOT DONE; the ~0.7ms host slice is NOT in the engine loop
+<!-- state: 2026-08-09T12:00 -->
+
+Ran the definitive clean-checkout binding grid at the committed sha `d3b412f5` (NOT an overlay:
+`git checkout -f d3b412f5` in `~/mxfp4-bench/src`, tree clean, binaries rebuilt) via
+`mxfp4-online-serving-grid.sh`, oracle vLLM 0.25.0
+`VLLM_DISABLED_KERNELS=FlashInferMxFp4LinearKernel`, single-load/arm, drop_caches+mincore. #44 model
+gate re-passed inside the grid. 24/24 legs `failed:0`, reps tight (ours c8 CoV ~0.4%). The grid
+EXIT=1 is the gate-FAIL signal (`gate_pass:false`) + single-model "cross-model summary waits for the
+other model" — NOT a crash; all q3mxfp4 artifacts complete (q3mxfp4's sweep IS c1-c8, matching
+#48/#49). Evidence `dgx:~/work/vllm.cpp-online-gate/evidence/d3b412f5.../summary-q3mxfp4/ratios.json`.
+
+BINDING (ours/vLLM normalized ratio) vs #49 tput [0.990/0.922/0.930/0.942]:
+total_token_throughput c1 **1.005** / c2 0.925 / c4 0.939 / c8 **0.953** (Δ +0.015/+0.003/+0.009/
++0.011); median TPOT c1 1.002 / c2 0.922 / c4 0.915 / c8 0.939; mean TTFT c1 1.034 / c2 0.962 / c4
+1.004 / c8 0.999; median ITL c1 1.004 / c2 0.919 / c4 0.920 / c8 0.929. Peak host-mem 35.2 vs 76.7
+GiB = **2.18x LESS** (WIN).
+
+PARITY VERDICT (MXFP4 goal): **c1 PASSES every axis** (tput/tpot/ttft/itl all ≥1.0x); c2-c8 BELOW on
+tput/tpot/itl (best c8 0.953 tput), TTFT parity c4/c8. `gate_pass:false` → **NOT DONE** (below-floor
+on c2-c8). But the byte-exact slivers moved EVERY throughput axis up vs #49 (c1 +1.5pp crosses to
+parity+, c4 +0.9pp, c8 +1.1pp, c2 +0.3pp), no regression — exactly the #50 block=8 recovery
+(~0.8pp@c8) + the memset-drop.
+
+RESIDUAL MAP (c8 ~4.75% tput gap): (1) block=16 padding — CLOSED (sliver a). (2) grouped-Marlin
+decode +7-9% per-call (GPU: E=1 indirect `sorted_token_ids` gather + fp32 `C_tmp` vs vLLM dense
+`marlin_gemm` direct-A; #46/#50 — delicate grouped→dense-direct-A port, not a config knob). (3) the
+host slice. HOST-SLICE ATTRIBUTION (step 2, VT_LOOP_TRACE under decode this session): every window
+shows **interval_ms ≈ step_ms** (mean 25.5 vs 25.5, Δ≤0.02ms; `admits=0` pure-decode windows
+interval−step ≈ 0) → the born-on-runner engine-core loop (scheduler+drain+admit) has NEGLIGIBLE
+per-iteration host overhead. So the #47 ~0.7ms/step "host/sched" residual is NOT an engine-loop
+lever — it is in the shared-architecture async frontend (HTTP/output-proc/detokenize, which vLLM
+also runs) or within the cross-tool attribution boundary. NEXT lever (only real one left) =
+grouped→dense-direct-A marlin decode (#50 NO-GO'd the full port; a targeted E1 direct-A path is the
+open candidate). Box left clean (both locks free, GPU idle, worker down, disk 21G, tmux gone).
+

docs/BENCHMARKS.md

@@ -307,7 +307,7 @@ built on it rather than keeping the flattering one.
 | Qwen3-dense decode CUDA-graph | Token-exact pass, ~4.3% e2e directional | Steady-state per-step tok/s |
 | Kimi-Linear-48B-A3B (KDA+MLA+MoE) | Full-model GB10 e2e RUNS (bf16-resident §13), NEAR-TIE 106/128, pool math CLOSES; default OFF | Full model RUNS on GB10 (bf16-resident, RSS peak 1.7 GiB, min-avail 21 GiB, no OOM). Token NEAR-TIE 106/128 (6/8 prompts exact, numerics vs deterministic oracle). 1.59 tok/s. Detail: spec §13 |
 | vLLM 0.26 re-benchmark | Pending | Re-run the binding grids on the advanced pin |
-| MXFP4 Qwen3-8B (W4A16 Marlin) | Post-GQA-swap (#49): c1 0.990, c2-c8 0.92-0.94, TTFT parity, mem 2.6x WIN; dense-marlin NO-GO (#50). Closers byte-exact default-ON (`QUANT-CT-MXFP4-CLOSERS`): M≤8 8-row tile + ws re-zero dropped, bitdiff=0, #44 3/3 | Binding re-bench x3 + ~0.7ms host slice pending |
+| MXFP4 Qwen3-8B (W4A16 Marlin) | Closers binding x3 (d3b412f5): c1 **1.005 PASSES**, c2/c4/c8 0.925/0.939/0.953 BELOW (+0.3/0.9/1.1pp vs #49), TTFT parity, mem 2.18x WIN; byte-exact slivers (bitdiff=0), #44 3/3, gate NO | grouped→dense-direct-A marlin; host slice NOT engine-loop (VT_LOOP_TRACE interval≈step) |
 | SGLang floor arms | Never ran | Both arms of the SGLang comparison |
 | cuBLAS invocation-parity guard | CI guard landed (CPU); `kGemvHeuristicAlgos` refactor build-verify owed | `nvcc` rebuild + SACRED gate on dgx |