perf(fa2): GB10 num_splits cap (gated-OFF) + MXFP4 parity TERMINAL residual - #60
Closed
localai-bot wants to merge 1 commit into
Closed
perf(fa2): GB10 num_splits cap (gated-OFF) + MXFP4 parity TERMINAL residual#60localai-bot wants to merge 1 commit into
localai-bot wants to merge 1 commit into
Conversation
…rity TERMINAL residual
The two last named MXFP4 levers, closed and measured.
LEVER 1 (num_splits cap, default OFF): NumSplitsHeuristic is fed num_sms*2 and
over-splits the batch-1 decode to nnblk (7 at c1, ~41us/layer) because every
eligible split keeps waves<1 on GB10's latency-bound kernel. VT_FA2_NSPLITS_CAP
(off / auto=wave-optimal / explicit-N) clamps it at both decode launch sites.
Grounded in the sanctioned KERNEL-FA2-DECODE-PARAMS A/B (fa2ab_n{1,3}): cap=3 ->
~32us (~17% flash win) but self-corrects to ~3 at c8, so it only moves c1-c2 and
c1 already passes 1.020x. Op test covers both GQA ratios (engages split->no-split,
byte-correct, near-ties). Battery green: op 114/114 + memcheck 0 err; SACRED
0.6B/4B 184/184 and 32B-NVFP4A16 strict 144/144 with cap=auto CHAR-IDENTICAL
(flips no strict token); async 325/325; #44 smoke 3/3. Stays OFF (non-byte-exact,
no failing-axis win).
LEVER 2 (glue): already folds through vt::FusedChain (both add+RMSNorm sites,
default-ON); the residual +198-290us glue is vLLM's Inductor GEMM-pro/epilogue
fusion, a Marlin kernel rewrite the catalog cannot express (#46-declined).
VERDICT: TERMINAL residual statement. Every measured lever exhausted. c1 1.020x
PASS + mem 2.63x WIN, c2-c8 0.962-0.969 GPU-intrinsic. The definitive oracle
binding is projected UNCHANGED vs #57 (cap inert at c4-c8) and not re-run under
box-safety (OOM-risk #52 deferred; ENOSPC hit during build). One unexhausted path
remains: a from-scratch Marlin epilogue-fusion kernel.
FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
Collaborator
Author
|
Landed on main as the squash (mudler authorship). VT_FA2_NSPLITS_CAP ships gated-OFF per parity-enablers (non-byte-exact, no failing-axis win). Terminal MXFP4 verdict recorded. 🤖 Generated with Claude Code |
mudler
added a commit
that referenced
this pull request
Aug 6, 2026
… closed (#60) row/QUANT-CT-MXFP4-FINAL-STACK squash. Lever 1: VT_FA2_NSPLITS_CAP (off/auto/N) at both FA2 decode launch sites — wave-optimal cap on the real 48 SMs; battery green incl. 32B strict CHARACTER-IDENTICAL at cap=auto, memcheck 0, async 325/325, #44 smoke 3/3. Stays default-OFF per parity-enablers: non-byte-exact split-reduction with no failing-axis win (only touches c1-c2; c1 already 1.020). Lever 2: glue already default-ON through vt::FusedChain at both add+RMSNorm sites + fused gate_up; residual is Inductor GEMM-epilogue fusion the catalog cannot express (a Marlin kernel rewrite, the #46-declined redirect). TERMINAL MXFP4 VERDICT: c1 1.020 PASS + mem 2.63x WIN; c2-c8 0.962-0.969 GPU-intrinsic; every measured lever exhausted. The one unexhausted path = from-scratch Marlin prologue/epilogue fusion kernel. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
localai-bot
pushed a commit
that referenced
this pull request
Aug 14, 2026
… the probe was measuring its own transposition (#644) Answers the FAIL review of PR #746. Issue #644, spec `.agents/specs/ltx25-tiled-decode.md`. BLOCKING — the headline number was a measurement artifact. `scripts/probe_ltx2_tiled_equivalence.cpp` reassembled the streamed chunks with a flat `insert(streamed.end(), chunk.frames.data.begin(), ...)`. A chunk is [C, t, H, W] CHANNEL-MAJOR (ltx2_video_vae.h:211-219), so that append yields [c0 t0..][c1 t0..][c2 t0..][c0 t1..].. — not [C, T, H, W] once C > 1 AND there is more than one chunk, and the published run is [3,81,64,64] with chunks=2, so both hold. It was comparing channel 1 against channel 0's later frames. The probe even carried a comment asserting the opposite ("the C-major layout means the comparison below is elementwise regardless"); that comment was the error, and the row's own test (Collected::Concat in tests/vllm/models/test_ltx2_tiling.cpp) had the correct reassembly all along. Re-derived on the real ltx-2.5-video-vae-conv-bf16.safetensors, same request: committed probe : max|diff| = 0.71614238619804382 985849 / 995328 corrected probe : max|diff| = 0.050304323434829712 962983 / 995328 both : untiled |out|max = 0.75126725435256958 So 6.70% of the output's own range, not 95% — a factor of 14. The corrected probe prints the flat-append figure as a labelled diagnostic, so both columns come out of ONE binary on ONE run and the artifact is reproducible rather than argued. The QUALITATIVE conclusion survives: 96.75% of values still move, 81..120 frames is still a tiled regime the one-tile control does not cover, and the refusal stands. Corrected at all four published sites — ltx2_tiling.h, ltx2_video.cpp, docs/USAGE.md (user-facing) and the spec — plus docs/FEATURES.md. The spec's Outcome now records that this is the THIRD number in campaign #644 to move by a large factor the first time anyone re-derived it: 60 GiB measured 361.72 MiB (170x), "121 frames" measured 81, and 95% measures 6.70% (14x). That self-record is worth more than the number. F1-b — the walk's density was a comment, not an assertion. Reproduced first: the committed scripts/probe_ltx2_tiling_layout.py with its step changed 8 -> 80 and nothing else exits 0 and prints "first splits at 81 frames" after examining two frame counts, skipping latent_t 2..10; saw_untiled and `first_tiled is not None` cannot see that. The walk now asserts it starts at latent_t 1, that consecutive steps move latent_t by exactly 1, and that the transition is 10 -> 11 — the short-circuit's own bound (tiling.py:239-240). The same mutation on the repaired probe exits 1 ("the walk is NOT dense: latent_t jumps [(1, 11)]"); unmutated it exits 0 over 17 densely walked frame counts. MODERATE — the recorded `ctest -N` no longer described the head. Merged origin/main (7 commits) and re-measured: 456 here against 455 on main @ 7b8919d with the same command in this tree, the one added listing being `Test #60: test_ltx2_tiling`; `vllm_cpp_add_test(` goes 431 -> 432 and `git diff -- tests/CMakeLists.txt` is +7 lines with nothing removed. The "448 N/448 lines" line is restated as what it is — a `grep -c` compared against `ctest -N`'s total, both commands recorded — and no longer cited as a committed instrument. MINOR — the "upstream raises" golden pinned the type, not the reason. A bare `except TypeError` kept the constant at 1 for any future TypeError from any line. The generator now walks the traceback to the innermost frame and emits *UpstreamUntiledFramesRaiseFile / RaiseLine / RaiseMessage (measured: conv_video_decoder.py, 424, "unsupported operand type(s) for -: 'NoneType' and 'int'"), with generator-side assertions on the file and message. Goldens regenerate byte-identically apart from those 6 added lines. MINOR — ltx2_video_vae_tiled.cpp said every frames-tiled arm "reproduces `forward` exactly". That holds only where the frames tile exceeds the axis; where it splits, kLtx2TileDecCausalUpstreamTiledVsUntiled is 2.13274002 against an OutputSpan of 2.31735897. Bounded to the one-tile and untiled-spatial controls, which are the arms actually pinned at 0. Gate. Clean `rm -rf build` + `cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release` + `ninja -C build -j4`, BUILD_EXIT=0, 0 warnings, `No space left|BFD assertion` count 0 against a positive control of 462 "Building" lines. df / 89% -> 96%; build directory deleted after the gate. `ctest -N` 456; full `ctest -j4 --output-on-failure` 455 passed / 1 failed of 456, exit 8, with 456 lines matching `^ *[0-9]+/456 Test`. The one failure is test_op_parity (#737, a null in the Music3 golden manifest) which prints "assertions: 70 | 70 passed | 0 failed" while failing; this branch matches nothing under tests/parity/ or any music3 file. test_ltx2_tiling 10/10 915/915 (was 907; +8 = 4 new CHECKs x 2 arms) test_ltx2_vae 36/36 3039/3039 test_ltx2_video 32/32 684/684 (+1 case / +11 assertions arrive with fc903b8 (#674) on main, not this row) test_ltx2_pipeline 37/37 2382/2382 RED-first for the new assertions, goldens mutated in place and restored, verified by sha256 (bf2bdb4d…), rebuilt GREEN each time: RaiseLine 424 -> 425 : 8/10 cases, 2 failed assertions, exit 1 RaiseFile/RaiseMessage -> wrong values : 8/10 cases, 6 failed assertions, exit 1 docs/FEATURES.md and .agents/roadmap_v1.md are keyed records: verified to differ from main @ 7b8919d by exactly this row's one key each. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5[1m] [claude-code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the MXFP4 parity leg's two last named levers and states the terminal residual.
Lever 1 — num_splits cap
VT_FA2_NSPLITS_CAP(default OFF), LANDEDcuda_flash_attn_fa2.cu:Fa2NsplitsCapConfig/ApplyNsplitsCapat both decode launch sites. The FA2 heuristic is fednum_sms*2and over-splits the batch-1 decode tonnblk(7 at c1, ~41us/layer) because every eligible split keeps waves<1 on GB10's latency-bound kernel. Modes: off /auto(wave-optimal) / explicit-N (the fa2ab A/B knob). Grounded in the sanctioned KERNEL-FA2-DECODE-PARAMS A/B (cap=3 -> ~32us, ~17% flash win) but it self-corrects to ~3 splits at c8, so it only moves c1-c2 and c1 already passes 1.020x.Gate battery (GB10 sm_121a): op
test_ops_paged_attnnew case 114/114 both GQA ratios (cap engages split->no-split, byte-correct, near-ties) + memcheck 0 err; SACRED 0.6B/4B 184/184 and 32B-NVFP4A16 strict 144/144 withcap=autoCHARACTER-IDENTICAL (flips no strict token); async 325/325; #44 smoke 3/3. Stays default-OFF (non-byte-exact reduction order, no failing-axis win to bank).Lever 2 — glue: already landed as scoped
qwen3.cpp::RunLayeralready routes both add+RMSNorm sites throughvt::FusedChain(kFusedAddRmsNormStd)(default-ON). The residual glue is vLLM's Inductor GEMM-pro/epilogue fusion — a Marlin kernel rewrite the catalog cannot express (#46-declined).Verdict — TERMINAL residual
Every measured lever exhausted. c1 1.020x PASS + mem 2.63x WIN; c2-c8 0.962-0.969 GPU-intrinsic. The definitive oracle binding is projected UNCHANGED vs #57 (cap inert at c4-c8) and not re-run under box-safety (OOM-risk deferred by #52; ENOSPC hit during build). One unexhausted path: a from-scratch Marlin epilogue-fusion kernel.
🤖 Generated with Claude Code
https://claude.ai/code/session_014fZAcg1WcU8V629k6HWKys