diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 161856a04..8e14e0eda 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -490,3 +490,5 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1403](https://github.com/mudler/vllm.cpp/issues/1403) | `ENG-CUDAGRAPH-BREAK` | **`test_qwen3_5_decode_graph_seam` SIGSEGVs — a DUPLICATE of [#1394](https://github.com/mudler/vllm.cpp/issues/1394) that was already fixed when it was filed, so no code change was owed.** Filed against `96ed8346f`, which PREDATES the fix `7dec1d990` (PR #1393, merged 2026-08-20 00:49); `git merge-base --is-ancestor` puts the fix outside the reported tree. Re-measured at `b537a5344`, CPU-only Release x86_64: the test passes 5 runs of 5, `8 cases`/`8 passed`, `assertions: 138`, `Status: SUCCESS!`, exit 0. The cause was re-derived by single-variable mutation rather than inherited from #1394's report, and NAMED by a sanitizer rather than reasoned. Debug + `VLLM_CPP_SANITIZE=address,undefined` (NDEBUG OFF) with BOTH halves of `7dec1d990` reverted to their `96ed8346f` bytes: `AddressSanitizer: SEGV on unknown address 0x5045f5f84900`, `The signal is caused by a READ memory access`, at `src/vt/cpu/cpu_paged_attn.cpp:59` in `KvElem` reached from `:224`, on threadpool worker T1, inside case `W6: two spec shapes of EQUAL S and different q get two graphs` (`tests/vllm/models/test_qwen3_5_decode_graph_seam.cpp:800`). **It is NOT a use-after-free and there is no free site**, and `AddressSanitizer can not provide additional info` is the tell: the out-of-bounds block-table read is itself IN-BOUNDS for ASan because it lands inside the neighbouring pooled allocation, so this is not a `heap-buffer-overflow` either — the VALUE read becomes a wild block index that `KvElem` multiplies by the KV block stride and dereferences, which is why the fault address is unmapped rather than a redzone and why the crash is order-dependent instead of deterministic. Both halves are load-bearing: reverting ONLY the `SpecAttnMeta` fixture leaves the kernel bound to refuse by name at `src/vt/cpu/cpu_paged_attn.cpp:152` (exit 1, the case THROWS); reverting the bound as well restores the crash (Release exit 139, 3 runs of 3; ASan exit 1); restoring both gives exit 0, 8/8, zero sanitizer findings. The issue's `5f68e60df` pointer was right about the commit and wrong about the half — `git log -S'SpecAttnMeta'` and `git log -S'W6: two spec shapes of EQUAL S'` each return that commit ALONE, and both hits sit in the TEST fixture, not in the graph eligibility it widened. [#1405](https://github.com/mudler/vllm.cpp/issues/1405)'s truncation reproduced (`assertions: 135` crashed vs `138` complete) and is SHARPER under a threadpool fault: doctest printed the summary TWICE with different totals, `135` then `141`, because the main thread kept running after a worker died. That a red reached `main` unreported is [#1285](https://github.com/mudler/vllm.cpp/issues/1285)/[#1376](https://github.com/mudler/vllm.cpp/issues/1376), not this row | bug | | [#1458](https://github.com/mudler/vllm.cpp/issues/1458) | `VT-ACT-ROUND-POLARITY` | **`4712dac40` reds FOUR suites on `main` — `test_ltx2_text_encoder`, `test_muse_glimmer_text`, `test_muse_glimmer_text_fallback`, `test_minimax_music3_ar` — by exceeding bf16 error floors none of them had re-derived.** Found while gating [#1403](https://github.com/mudler/vllm.cpp/issues/1403) (PR #1457) and filed in flow, not that row's defect. Deterministic, not a load artifact: first seen under `-j4` and re-measured SERIALLY, 101 s across the four. `test_ltx2_text_encoder.cpp:2407` reads `CHECK( 0.1323 <= 0.109394 )` and `:2409` reads `CHECK( 0.0752773 <= 0.0573374 )`, both over by 20-30%, with the file at `26 passed / 1 failed`, `4118 assertions`, `Status: FAILURE!`. ATTRIBUTED BY MUTATION rather than inferred, both directions, in one build directory changing only `src/vt/cpu/cpu_ops.cpp`, compile rc 0 on every arm and `sha256sum` taken before and after: at `0adeb8b0e` four fail; reverted to `4712dac40^` four pass and `test_ltx2_text_encoder` is 27/27 `SUCCESS!`; restored to a re-matched `sha256` four fail again. They also passed at `b537a5344`, three commits earlier, in a full 567-test `ctest` run at 3-6 s each. `4712dac40` is +42/-3 in `src/vt/cpu/cpu_ops.cpp` plus a new `tests/vt/test_ops_activation.cpp`, and touches none of the four. NOT FIXED IN FLOW, deliberately: the question is a numerics decision, not a defect with an obvious repair — either the floors were calibrated against the rounding polarity that commit corrected and need re-deriving against the oracle, or the narrowing is wider than upstream's — and editing the four floors to pick the first branch is the scope-widening `AGENTS.md` prohibits | bug | | [#1469](https://github.com/mudler/vllm.cpp/issues/1469) | `KERNEL-DFLASH2-GROUPED-CONV` | **`test_audit_live_rows` is RED on `main`.** Measured on a detached worktree at `origin/main` `cae8ace0c` with nothing applied: `test_shipped_record_has_no_abandoned_active_row` fails with `stale ACTIVE rows remain: ['KERNEL-DFLASH2-GROUPED-CONV']`, 57 tests, 1 failure, and `scripts/agent-preflight.sh` reports both `audit-live-rows` and `test_audit_live_rows` as FAILED. `scripts/audit-live-rows.py` alone returns rc 0 and prints `245 live rows; 1 abandoned ACTIVE`, so the count is the gate and the script's exit status does not carry it. The row is `.agents/kernel-matrix.md:143`, state `ACTIVE`, claim `CLAIM-SPEC-DFLASH2-W2`, and its implementation landed on `main` in `028438e68` (SPEC-DFLASH2 W2, [#1314](https://github.com/mudler/vllm.cpp/issues/1314), PR #1465) - a row still claimed as in-progress whose work is already reachable on `main`. NOT FIXED IN FLOW: the kernel matrix, the row and the DFlash2 claim are outside the authority of the row that found it (`LTX25-DEVICE-RESIDENCY` W0, PR #1441), and a lifecycle move owes `docs/STATUS.md`, `docs/BENCHMARKS.md` and the moved row spec's `## Now`. Found while running the full preflight for that row's gate, where it is an inherited red | bug | +| [#1466](https://github.com/mudler/vllm.cpp/issues/1466) | `MODEL-MUSE-GLIMMER` | **`tests/vllm/models/test_muse_glimmer_text.cpp:532`'s `CHECK(diff <= 5e-4)` is a rounded-up W1 measurement, and one correct kernel change already spent 45 points of its 76-point margin.** `3a54c4b7d`'s own body quotes the number the constant was rounded up from ("max abs diff 1.21e-4 on logits of max 4.88e-2"); there is no derivation beside it. `4712dac40` narrowed `act(gate)` to the input dtype — upstream's polarity, and the only form that reproduces the committed `silu_and_mul_bf16_8x256` oracle golden bit-exactly — and grew the envelope 2.8x to 3.43e-04, which is 0.687 of the bound (measured at `aeba0de6f`, CPU-only Release, x86_64). It is the last member of that class in this file: [#1458](https://github.com/mudler/vllm.cpp/issues/1458) replaces the other one (`bdiff <= 1e-5`) in the same case and leaves this one to its own derivation, because a bigger constant is not a repair. Found in flow while gating #1458 (PR [#1461](https://github.com/mudler/vllm.cpp/pull/1461)); also listed under `## Owed` in [`muse-glimmer.md`](specs/muse-glimmer.md) | bug | +| [#1467](https://github.com/mudler/vllm.cpp/issues/1467) | `MODEL-DIFFUSION-ltx-2-5-ltx2-video-transformer-3d-model` | **The ltx2 prompt->conditioning case no longer detects position renumbering, and post-`4712dac40` the instrument is INVERTED — the mutant is closer to the oracle than the correct code.** MEASURED in one build directory, CPU-only Release, x86_64, compile rc 0 on every arm and every source restored `sha256sum`-verified, as ratios to the propagated floor: before `4712dac40`, correct 0.565/0.688 and renumbered 0.831/**1.099** (the 1.10x the case's own note claims, and it reded); at `aeba0de6f`, correct **1.209/1.313** and renumbered 0.683/0.931 — so at the old `1.0x` bound the case reds the port and passes the defect. `4712dac40` is right; what it did here was raise this comparison's noise floor above the defect's signal, and no constant recovers a detection whose ordering has reversed. [#1458](https://github.com/mudler/vllm.cpp/issues/1458) restores a functioning instrument at a derived `2.0x` (its scatter-offset mutation still reds at 14.08x/24.06x) and does not claim this coverage. A repair owes an instrument with no bf16 accumulation between the defect and the assertion — the integer `positions` contract, or the f32 rope table `scripts/gen-ltx2-gemma-tower-goldens.py:363-375` already names as the right one for this class. Found in flow while gating #1458 (PR [#1461](https://github.com/mudler/vllm.cpp/pull/1461)); also listed under `## Owed` in [`ltx-2-5.md`](specs/ltx-2-5.md) | bug | diff --git a/.agents/specs/ltx-2-5.md b/.agents/specs/ltx-2-5.md index 975a5d9c3..2827660ff 100644 --- a/.agents/specs/ltx-2-5.md +++ b/.agents/specs/ltx-2-5.md @@ -947,6 +947,22 @@ none of them. Per AGENTS.md that is the worse half of the silent/refused split: naming a missing part is documented debt, and silence is not. Each now has its own issue, each saying what is absent, what a future row starts from, and what blocks it. +- [#1467](https://github.com/mudler/vllm.cpp/issues/1467) — the + `"ltx2 prompt -> conditioning: the VALUES"` case no longer detects position + renumbering, and the note in it that claims 1.10x-of-floor detection is stale. + MEASURED while gating [#1458](https://github.com/mudler/vllm.cpp/issues/1458): + before `4712dac40` the mutant reached 1.099x the audio floor and reded; at + `aeba0de6f` it reaches 0.683x/0.931x while the CORRECT code reaches + 1.209x/1.313x, so the instrument is INVERTED — the mutant is closer to the + oracle than the port is. `4712dac40` is right (it is the only form that + reproduces the `silu_and_mul_bf16_8x256` oracle golden bit-exactly); it raised + this comparison's noise floor above the defect's signal. #1458 restores a + functioning instrument and does not recover the detection, and no constant on + this quantity can. A repair owes an instrument with no bf16 accumulation + between the defect and the assertion — the integer `positions` contract, or + the f32 rope table the generator already names as the right one for this class + (`scripts/gen-ltx2-gemma-tower-goldens.py:363-375`). + - [#1093](https://github.com/mudler/vllm.cpp/issues/1093) — `TI2VidTwoStagesPipeline` (`ti2vid_two_stages.py:61`). NOT our `distilled_two_stage`: stage 1 is CFG-guided on the FULL model (`:247-259`), stage 2 carries the distilled LoRA alone (`:151`), and stage-1 diff --git a/.agents/specs/muse-glimmer.md b/.agents/specs/muse-glimmer.md index 6f2224676..4a94aa27e 100644 --- a/.agents/specs/muse-glimmer.md +++ b/.agents/specs/muse-glimmer.md @@ -1177,3 +1177,16 @@ about magnitudes. **Two greedy prompts are not a token-exactness claim.** - **The drafter has still never run.** §10.5's acceptance A/B is unchanged. - **No speed axis.** §0 stands: the pinned oracle cannot load `muse_glimmer`, so there is no denominator and none is claimed here on any axis. + +## Owed + +- [#1466](https://github.com/mudler/vllm.cpp/issues/1466) — + `tests/vllm/models/test_muse_glimmer_text.cpp:532`, `CHECK(diff <= 5e-4)` is + the W1 measurement rounded up (`3a54c4b7d`'s body quotes 1.21e-04) with no + derivation beside it. `4712dac40` gave `vt`'s gated activations upstream's + rounding polarity, which is correct, and grew the envelope 2.8x to 3.43e-04 — + 0.687 of the bound. Found while gating + [#1458](https://github.com/mudler/vllm.cpp/issues/1458), which repairs the + other member of the same class in the same case (`bdiff <= 1e-5`) and + deliberately leaves this one to its own derivation. A repair owes a measured + floor or a precision argument, never a bigger constant. diff --git a/.agents/specs/vt-act-round-polarity.md b/.agents/specs/vt-act-round-polarity.md index dc09b245e..9ab642d31 100644 --- a/.agents/specs/vt-act-round-polarity.md +++ b/.agents/specs/vt-act-round-polarity.md @@ -3,7 +3,10 @@ Issues: [#1322](https://github.com/mudler/vllm.cpp/issues/1322) (the filed gap), [#1342](https://github.com/mudler/vllm.cpp/issues/1342) (Vulkan/Metal silu spelling), [#1343](https://github.com/mudler/vllm.cpp/issues/1343) -(`RmsNormPlusAdd` arm asymmetry). +(`RmsNormPlusAdd` arm asymmetry), +[#1458](https://github.com/mudler/vllm.cpp/issues/1458) (four model suites red +on `main` behind the landed narrowing; fixed, see `## The four suites #1458 +reds, and why the kernel is not the defect`). Row: `VT-ACT-ROUND-POLARITY`. ## Now @@ -19,7 +22,18 @@ existing f32 golden and every existing byte-exact composite contract unmoved, and it is asserted directly rather than inferred (see `tests/vt/test_ops_activation.cpp`, "leaves an f32 INPUT bit-identical"). -**Five providers are still owed and are listed under ` +**The narrowing survived a second, independent challenge.** +[#1458](https://github.com/mudler/vllm.cpp/issues/1458) reported four model +suites red behind `4712dac40`, attributed by mutation in both directions. The +kernel was re-verified from source at the pin and from the committed oracle +golden, and it is right; the four bounds were not. See `## The four suites #1458 +reds, and why the kernel is not the defect`. + +**Five providers are still owed and are listed under `## Owed`.** CUDA, ROCm, +Metal and Tenstorrent cannot be compiled from the dev box at all, and the Vulkan +arm additionally needs a GLSL toolchain to regenerate its committed SPIR-V. This +row does NOT claim provider parity it did not test. + ## What landed, and what it measured `RoundThrough(DType, float)` — the helper `kRmsNormGatedGroup` already uses for @@ -49,11 +63,6 @@ golden pass ran 46 cases on both sides: the table under `## Premise`, where the worst element margin against the harness's own `atol + rtol*|want|` moves 0.5364 -> 0.2989. -## Owed`.** CUDA, ROCm, -Metal and Tenstorrent cannot be compiled from the dev box at all, and the Vulkan -arm additionally needs a GLSL toolchain to regenerate its committed SPIR-V. This -row does NOT claim provider parity it did not test. - ## Gap verification Checked before writing anything, because several changes this session were found @@ -372,6 +381,122 @@ That is why this spec stops here. box for the byte-exact composite suites, a ROCm box, a Mac, and a glslang for the SPIR-V regenerate. +## The four suites #1458 reds, and why the kernel is not the defect + +[#1458](https://github.com/mudler/vllm.cpp/issues/1458) reported +`test_ltx2_text_encoder`, `test_muse_glimmer_text`, +`test_muse_glimmer_text_fallback` and `test_minimax_music3_ar` red on `main` +behind `4712dac40`, attributed by mutation in both directions. All four +reproduce at `aeba0de6f`, CPU-only Release (`-DCMAKE_BUILD_TYPE=Release`, so +NDEBUG), x86_64. + +**The premise was re-derived rather than taken.** Every anchor the landing +commit cites is exact at the pin `555967922`, read in the checkout whose +`git rev-parse HEAD` is that sha: +`csrc/libtorch_stable/activation_kernels.cu:158` (`silu_kernel` returns +`(T)(...)`), `:36` (`compute` returns `(scalar_t)(ACT_FN(gate, alpha) * ...)`, +so `ACT_FN` has already narrowed), `:205` (`gelu_tanh_kernel`, same shape), +`vllm/model_executor/layers/activation.py:143` and `:418` (the native arms, +`F.silu` / `F.gelu` on a bf16 tensor yielding bf16), and +`tests/kernels/core/test_activation.py:108` +(`assert_close(out, ref_out, atol=0.0, rtol=0.0)`, which pins the two together +bit-exactly). The VECTORIZED arm agrees: `packed_compute` at +`activation_kernels.cu:50-95` narrows through `cast_to_packed` as +well, so upstream has ONE polarity across both of its arms and the `atol=0.0` +test has no second behaviour to hide. + +**The committed oracle golden settles it without reading a kernel.** +Recomputing `tests/parity/goldens/silu_and_mul_bf16_8x256/out.npy` from its own +`x.npy` in numpy, with bf16 round-to-nearest-even applied by hand: + +| expression | max abs err vs golden | bit-exact | +|---|---|---| +| `silu_f32(g) * up`, no store round | 1.434994e-02 | no | +| `bf16(silu_f32(g) * up)` — the PRE-`4712dac40` kernel | 1.562500e-02 | no | +| `bf16(silu_f32(g)) * up`, no store round | 7.812500e-03 | no | +| `bf16(bf16(silu_f32(g)) * up)` — the POST kernel | **0** | **yes** | + +The golden's every value is exactly bf16-representable, so it came from a vLLM +that rounds twice. The landed kernel is the only one of the four that reproduces +it. Reverting it is not available. + +**What the four bounds were, and what each one actually measured.** + +- `test_ltx2_text_encoder`, the prompt->conditioning VALUES case. Its floor is + the oracle's own f32-vs-bf16 spread carried through the same projection, which + is the right SCALE. Its constant was not: the bf16 arm carried `1.0x` while the + f32 arm carried `2.0x`, and the `1.0x` was imported from the state-level parity + case, where the same two objects are compared elementwise. This case compares + them through `Ltx2TextEncoderConditioning`, which stacks all 13 states — the + floor is the projection of ONE error vector and the gated quantity is the + projection of a DIFFERENT one, and a linear map that combines 13 states does + not preserve the relation. Both arms now carry `2.0x`, which is the triangle + bound with each side's departure from the shared f32 trajectory taken at one + floor. Video 1.209x -> margin 0.60; audio 1.313x -> margin 0.66. + The state-level case is untouched and still green at 0.870x of its `1.0x`. +- `test_muse_glimmer_text` and its `VT_FUSED_CHAIN_ADOPT=0` registration. + `bdiff <= 1e-5` was the W1 measurement (5.28e-06 at `3a54c4b7d`) rounded up, + 1.89x of headroom and no derivation. The biting-soft-cap case exists to pin the + ORDER of the output multiplier and the cap, and the order is an ALGEBRAIC + property of the output range, not a max|diff| band: at `cap = 1e-3` the logits + saturate the tanh to within 1e-40, so upstream's order reaches `cap` and the + swap reaches `out_mult * cap`. Measured: our saturation and the reference's + agree EXACTLY (delta 0) and the swap moves it by 2.5e-04, 32x the bound, which + is two bf16 relative spacings on `cap`. The knee-driven max|diff| is kept at + its rigorous Lipschitz value (`bdiff <= diff`; the cap is a contraction). +- `test_minimax_music3_ar`, the composed depth stage. `device_codes == + host_codes` over an argmax on two implementations whose measured separation is + 308 bf16 ULP. Draw 6 is a tie: relative top-2 margins 1.95e-03 (device) and + 2.93e-03 (host), both BELOW one bf16 ULP (2^-8 = 3.91e-03), between codes 2 and + 17. Draws 0..5 have margins 1.74e-02 to 3.53e-01 and agree. The `==` was + reading a coin. It is now: codes agree, or the divergence is a SHARED near-tie + in which each arm ranks the other's pick within one bf16 ULP of its own. + +**Two candidate replacements were tried and rejected**, recorded so nobody +re-derives them. For muse, `bdiff <= diff` alone is rigorous but does NOT red the +order swap (uncapped envelope 3.4e-04, defect 2.5e-04). A measured twin at +`out_mult = 1`, where the two orders coincide by algebra, gives 6.10e-06 — but it +is a different model sitting at a different point on the tanh knee, and the gated +run reaches 2.4x it with nothing wrong. + +**What this costs, MEASURED, and it is `4712dac40` that spent it rather than the +new bound.** The ltx2 conditioning case documents that renumbering the positions +from zero reds it at 1.10x the audio floor. Ratios to the propagated floor, one +build directory, compile rc 0 on every arm, every source restored and +`sha256sum`-verified: + +| `src/vt/cpu/cpu_ops.cpp` | production code | video | audio | at the old `1.0x` | +|---|---|---:|---:|---| +| before `4712dac40` | correct | 0.565 | 0.688 | pass | +| before `4712dac40` | renumbered | 0.831 | **1.099** | **RED** — the documented 1.10x | +| at `aeba0de6f` | correct | **1.209** | **1.313** | **RED** | +| at `aeba0de6f` | renumbered | 0.683 | 0.931 | pass | + +Read the bottom two rows together: post-`4712dac40` the instrument is INVERTED — +it reds the correct code and passes the mutant, and the mutant is measurably +CLOSER to the oracle than the port is. The `2.0x` restores a functioning +instrument; it does not recover that detection, and no constant can, because the +ordering of the two has reversed. It is also a property of the defect — +upstream's own f32 answers for positions 12..19 and 0..7 agree to 3.6e-06 +relative — and `scripts/gen-ltx2-gemma-tower-goldens.py:363-375` already records +that the end-to-end states are the wrong instrument for that class and the f32 +rope table is the right one. Filed as +[#1467](https://github.com/mudler/vllm.cpp/issues/1467) and owed by +[`ltx-2-5.md`](ltx-2-5.md). + +**One more bound of the same class is left standing, and it is filed rather than +re-tuned.** `tests/vllm/models/test_muse_glimmer_text.cpp:532`'s +`CHECK(diff <= 5e-4)` is the other W1 measurement rounded up (1.21e-04 at +`3a54c4b7d`), and `4712dac40` moved it from 0.242 to 0.687 of its bound. It is +green, it is out of scope for this repair because it needs its own derivation, +and it is [#1466](https://github.com/mudler/vllm.cpp/issues/1466), owed by +[`muse-glimmer.md`](muse-glimmer.md). + +**The narrowing did not make the port worse.** At the state level, where the +comparison is elementwise and the amplification above does not apply, our +distance to the f32 oracle IMPROVED at 11 of the 13 states (worst ratio 0.644 -> +0.572) and the bf16 arm stayed inside its `1.0x` floor (0.714 -> 0.870). + ## Stop conditions - A golden that gets **worse** after the activation change stops the row and is diff --git a/tests/vllm/models/test_ltx2_text_encoder.cpp b/tests/vllm/models/test_ltx2_text_encoder.cpp index 9e136309e..0f5a0d41f 100644 --- a/tests/vllm/models/test_ltx2_text_encoder.cpp +++ b/tests/vllm/models/test_ltx2_text_encoder.cpp @@ -2382,7 +2382,8 @@ TEST_CASE("ltx2 prompt -> conditioning: the VALUES, against the left-padded orac // The floor, propagated rather than borrowed: it is the oracle's OWN // f32-vs-bf16 spread carried through the identical projection, so it is the // smallest difference this comparison can resolve. It cannot be widened to - // rescue a failure without regenerating the oracle. + // rescue a failure without regenerating the oracle; what the two CHECKs below + // carry is a derivation over it, stated where they stand. auto max_diff = [](const std::vector& a, const std::vector& b) { if (a.size() != b.size()) return std::numeric_limits::infinity(); double d = 0.0; @@ -2404,9 +2405,61 @@ TEST_CASE("ltx2 prompt -> conditioning: the VALUES, against the left-padded orac << "x the propagated floor " << video_floor << "), vs f32 oracle = " << video_f32 << "; audio " << audio_bf16 << " (" << (audio_bf16 / audio_floor) << "x " << audio_floor << ") / " << audio_f32); - CHECK(video_bf16 <= video_floor); + // BOTH arms carry the SAME bound. The reasoning is the triangle step the + // state-level parity case above spells out; the NUMBER comes from this case's + // own measurements, cited below. `out.conditioning` is OUR bf16 realization, + // `want_bf16` is the ORACLE's, and neither is a rounding of the other. Both depart from the shared f32 trajectory `want_f32`, so + // + // |ours - oracle_bf16| <= |ours - oracle_f32| + |oracle_f32 - oracle_bf16| + // + // and the second term IS the propagated floor. The first term is `video_f32`, + // which THIS case measures a line above: 0.79x the floor on video and 0.475x + // on audio, so taking it at one floor is the measurement, not an assumption. + // Hence 2x on the bf16 arm as well as on the f32 one. + // + // WHY NOT 3x, since the sibling assertion below gates that first term at 2x. + // Chaining the two ASSERTED bounds would give 3x, and 3x is a bound that can + // never fire: `|ours - oracle_bf16| <= |ours - oracle_f32| + floor` is the + // triangle inequality itself, so once the f32 arm passes at 2x the bf16 arm at + // 3x is arithmetic rather than a gate. 2x is the tighter value the measured + // premise supports, and it is the one that can still say something. + // + // WHY IT USED TO SAY 1x, AND WHY THAT WAS NEVER DERIVED. The state-level case + // gates `d_bf16 <= floor` and derives its f32 arm from that primitive. This + // case IMPORTED the 1x across `Ltx2TextEncoderConditioning`, which stacks all + // 13 states and combines them. A per-state relation that holds elementwise + // does not survive that: the floor is the projection of ONE error vector and + // this quantity is the projection of a DIFFERENT one, and the projection can + // amplify the second relative to the first. It held until the seam's bf16 + // rounding polarity was corrected to upstream's in `4712dac40`, which re-rolled + // both error directions and moved the video arm from 0.56x to 1.21x and the + // audio arm from 0.69x to 1.31x — with our distance to the f32 oracle + // IMPROVING at 11 of the 13 states. The old constant broke, not the port. + // #1458. + // + // WHAT THIS COSTS, MEASURED rather than estimated, and it is not this bound + // that spent it. The position-renumbering note at the top of this case claims + // detection at 1.10x the audio floor. Ratios to the propagated floor, one + // build directory, every source restored sha256-verified: + // + // cpu_ops.cpp code video audio + // before 4712dac40 correct 0.565 0.688 -> pass at 1.0x + // before 4712dac40 renumbered 0.831 1.099 -> RED at 1.0x + // at aeba0de6f correct 1.209 1.313 -> RED at 1.0x + // at aeba0de6f renumbered 0.683 0.931 -> pass at 1.0x + // + // Read the bottom two rows together: post-`4712dac40` the instrument is + // INVERTED, reding the correct code and passing the mutant, and the mutant is + // measurably CLOSER to the oracle than the port is. The 2x here restores a + // functioning instrument; it does not recover that detection, and no constant + // can, because the ordering of the two has reversed. It is also a property of + // the defect — upstream's own f32 answers for positions 12..19 and 0..7 agree + // to 3.6e-06 relative — and `gen-ltx2-gemma-tower-goldens.py:363-375` already + // records that the end-to-end states are the wrong instrument for this class + // and the f32 rope table is the right one. Owed as #1467. + CHECK(video_bf16 <= 2.0 * video_floor); CHECK(video_f32 <= 2.0 * video_floor); - CHECK(audio_bf16 <= audio_floor); + CHECK(audio_bf16 <= 2.0 * audio_floor); CHECK(audio_f32 <= 2.0 * audio_floor); // The reorder and the mask, compared EXACTLY — but read what that does and does diff --git a/tests/vllm/models/test_minimax_music3_ar.cpp b/tests/vllm/models/test_minimax_music3_ar.cpp index 492b41675..f7a216266 100644 --- a/tests/vllm/models/test_minimax_music3_ar.cpp +++ b/tests/vllm/models/test_minimax_music3_ar.cpp @@ -1698,14 +1698,37 @@ TEST_CASE("music3 ar: the COMPOSED depth stage TAKES the device arm and draws th int64_t host_draws = 0; int64_t device_draws = 0; - const auto make_sampler = [](int64_t* counter) { + // Each draw records the code it took AND the probability vector it took it + // from. The code comparison below needs the vector: an argmax whose top two + // candidates sit closer together than the two arms' own arithmetic resolution + // is not a property either arm can be held to, and the only way to say that + // without asserting a coincidence is to ask what the OTHER arm's pick was + // worth in this arm's own distribution. See #1458. + struct DrawRecord { + int64_t code = 0; + std::vector probs; + double best() const { return static_cast(probs[static_cast(code)]); } + double runner_up() const { + double r = -std::numeric_limits::infinity(); + for (size_t i = 0; i < probs.size(); ++i) + if (static_cast(i) != code) r = std::max(r, static_cast(probs[i])); + return r; + } + }; + std::vector host_rec; + std::vector device_rec; + const auto make_sampler = [](int64_t* counter, std::vector* rec) { return m3::Music3CodeSampler( - [counter](const std::vector& probs, const m3::Music3Draw&) -> int64_t { + [counter, rec](const std::vector& probs, const m3::Music3Draw&) -> int64_t { ++*counter; size_t best = 0; for (size_t i = 1; i < probs.size(); ++i) { if (probs[i] > probs[best]) best = i; } + DrawRecord r; + r.code = static_cast(best); + r.probs = probs; + rec->push_back(std::move(r)); return static_cast(best); }); }; @@ -1714,8 +1737,8 @@ TEST_CASE("music3 ar: the COMPOSED depth stage TAKES the device arm and draws th // default-constructed arm — which is what every existing caller passes. std::vector host_codes{semantic_code}; const std::vector host = m3::Music3DepthStage( - last_conditional, last_unconditional, frame_index, weights, make_sampler(&host_draws), - &host_codes); + last_conditional, last_unconditional, frame_index, weights, + make_sampler(&host_draws, &host_rec), &host_codes); vt::Queue queue = CpuQueue(); m3::DepthDecoderWeights stage_source = depth; @@ -1731,7 +1754,7 @@ TEST_CASE("music3 ar: the COMPOSED depth stage TAKES the device arm and draws th std::vector device_codes{semantic_code}; const std::vector device = m3::Music3DepthStage(last_conditional, last_unconditional, frame_index, weights, - make_sampler(&device_draws), &device_codes, arm); + make_sampler(&device_draws, &device_rec), &device_codes, arm); const uint64_t after = m3::Music3DepthDeviceForwardCount(); // THE ASSERTION #1131 SAYS IS MISSING. `num_codebooks` appends a frame: one @@ -1754,7 +1777,86 @@ TEST_CASE("music3 ar: the COMPOSED depth stage TAKES the device arm and draws th // A tolerance CANNOT see a dropped stage — the prefix append, the fed-back // projection row — because a schedule missing one still produces finite, // plausible numbers. The drawn codes can. - CHECK_MESSAGE(device_codes == host_codes, "the two arms drew different residual codes"); + // + // WHAT THE CODES CANNOT DECIDE, and why this is a per-draw comparison rather + // than one `==`. The two arms are two implementations of the same block, not + // two runs of one: the device arm goes through `vt` GEMMs and the host arm + // through the `ArCompute::kBFloat16` reference, and this suite measures their + // separation at 174.5 to 6924 bf16 ULP across its device/host comparisons — + // 308 in this case. Both store at bf16, so their probabilities agree to about + // one bf16 unit roundoff: 2^-8, the largest relative error a store at that + // width can carry. An argmax whose top two candidates are separated by LESS + // than that is decided by rounding, and holding the two arms to the same code + // there asserts a coincidence, not a guarantee. + // + // MEASURED on this fixture at `aeba0de6f`: draws 0..5 have relative top-2 + // margins of 1.74e-02 to 3.53e-01 — 4.5x to 90x the resolution — and both arms + // agree at every one. Draw 6 is a tie at 1.95e-03 (device) and 2.93e-03 + // (host), BELOW one unit roundoff, between codes 2 and 17, and each arm + // ranks the other's pick inside it. It flipped when `4712dac40` gave + // `vt`'s gated activations upstream's rounding polarity, which is the correct + // polarity and the only one that reproduces the committed + // `silu_and_mul_bf16_8x256` oracle golden bit-exactly. The blanket `==` was + // reading a coin. #1458. + // + // A dropped stage is NOT let through: it moves the distributions by O(1), so + // the arms stop agreeing about which candidates are tied at all. MEASURED — + // deleting the MLP residual add in `minimax_music3_depth_device.cpp` diverges + // at draw 1 with the DEVICE's pick 1.60e-01 below the host's own best, 41x the + // unit roundoff this admits, and `shared_tie` is RED. + // 2^-8 is bf16's UNIT ROUNDOFF -- the largest relative error a correctly + // rounded bf16 store can carry. It is NOT the format's relative spacing + // (machine epsilon), which is twice it at 2^-7, and it is not the `Bf16Ulp` + // helper this file uses for its value metric either. The unit roundoff is what + // bounds a stored probability's departure, which is what this comparison wants. + constexpr double kBf16UnitRoundoff = 1.0 / 256.0; + REQUIRE(host_rec.size() == device_rec.size()); + REQUIRE(host_codes.size() == device_codes.size()); + REQUIRE(host_rec.size() + 1 == host_codes.size()); + int64_t compared = 0; + bool diverged = false; + for (size_t i = 0; i < host_rec.size() && !diverged; ++i) { + ++compared; + const DrawRecord& h = host_rec[i]; + const DrawRecord& d = device_rec[i]; + if (h.code == d.code) continue; + // Each arm is about to be indexed with the OTHER arm's argmax, so the two + // heads must agree on the vocabulary width before that read happens. A + // defect that changed the device head's output width would otherwise be + // undefined behaviour here, and NDEBUG is the configuration this runs in. + REQUIRE(h.probs.size() == d.probs.size()); + REQUIRE(h.code >= 0); + REQUIRE(d.code >= 0); + REQUIRE(static_cast(h.code) < h.probs.size()); + REQUIRE(static_cast(d.code) < d.probs.size()); + // A divergence is admissible ONLY as a shared near-tie: each arm must rank + // the OTHER arm's pick within one bf16 unit roundoff of its own. That is not a + // tolerance anybody chose — it is the width the two arms store at, so a gap + // narrower than it is decided by rounding and not by the model. If either + // arm ranked the other's pick BELOW that, the two arms disagree about the + // distribution and not merely about a tie, and this fires. + const double h_gap = (h.best() - static_cast(h.probs[static_cast(d.code)])) / + std::max(h.best(), 1e-30); + const double d_gap = (d.best() - static_cast(d.probs[static_cast(h.code)])) / + std::max(d.best(), 1e-30); + const bool shared_tie = (h_gap <= kBf16UnitRoundoff) && (d_gap <= kBf16UnitRoundoff); + CHECK_MESSAGE(shared_tie, + "draw " << i << ": the two arms drew different codes (host " << h.code + << ", device " << d.code + << ") and it is NOT a shared tie — in the host's own distribution " + "the device's pick is " + << h_gap << " below its best, and in the device's the host's pick is " + << d_gap << " below its best, against one bf16 unit roundoff " << kBf16UnitRoundoff); + if (shared_tie) + MESSAGE("draw " << i << ": shared near-tie, host drew " << h.code << " (runner-up gap " + << ((h.best() - h.runner_up()) / h.best()) << "), device drew " << d.code + << " (runner-up gap " << ((d.best() - d.runner_up()) / d.best()) + << "), both inside one bf16 unit roundoff " << kBf16UnitRoundoff + << ". Every later draw is fed a different code, so the comparison stops " + "here."); + diverged = true; + } + CHECK_MESSAGE(compared > 0, "no draw was compared, so the code comparison gated nothing"); CHECK(device_draws == host_draws); CHECK(device_draws == config.residual_codebooks()); CHECK(static_cast(device_codes.size()) == config.num_codebooks); diff --git a/tests/vllm/models/test_muse_glimmer_text.cpp b/tests/vllm/models/test_muse_glimmer_text.cpp index 283312be0..33db867a2 100644 --- a/tests/vllm/models/test_muse_glimmer_text.cpp +++ b/tests/vllm/models/test_muse_glimmer_text.cpp @@ -527,6 +527,9 @@ TEST_CASE("muse_glimmer text: matches the fp32 reference transcribed from upstre double scale = 0.0; for (float x : want) scale = std::max(scale, std::abs(static_cast(x))); const double diff = MaxAbsDiff(got, want); + // This band is the same rounded-up W1 measurement class the biting case below + // no longer carries, and `4712dac40` moved it from 0.242 to 0.687 of its + // bound. Owed as #1466, with its own derivation; not re-tuned here. MESSAGE("muse_glimmer text vs fp32 reference: max|diff|=" << diff << " over logits of max|.|=" << scale); CHECK(diff <= 5e-4); @@ -534,12 +537,65 @@ TEST_CASE("muse_glimmer text: matches the fp32 reference transcribed from upstre // Same model with a BITING soft-cap, so the reference also pins the ORDER of the // output multiplier and the cap (:1618-1621). At the default cap of 20 the tanh is // linear over these logits and the order is unobservable; at 1e-3 it is not. + // + // WHERE THE ORDER IS VISIBLE, and it is not in a max|diff| band. At cap = 1e-3 + // the logits (max|.| = 4.88e-2) drive |x / cap| ~ 50, so the tanh is saturated + // to within 1e-40 and the two candidate orders reach DIFFERENT saturation + // magnitudes: `cap * tanh(mult * x / cap)` reaches `cap`, and the swapped + // `mult * cap * tanh(x / cap)` reaches `out_mult * cap`. On this fixture that + // is 1.0e-03 against 7.5e-04. So the order is an ALGEBRAIC property of the + // output's range, and gating it there costs no band to tune: MEASURED, our + // saturation and the reference's agree EXACTLY (delta 0), and the swap moves it + // by |1 - out_mult| * cap = 2.5e-04 — 32x the bound below. + // + // The bound: our logits pass a bf16 store, so the saturation we can report + // departs from `cap` by at most one bf16 unit roundoff (2^-8) on each side. + // + // WHAT THIS REPLACES, AND WHY. `bdiff <= 1e-5` was the W1 measurement (5.28e-06 + // at `3a54c4b7d`) rounded up, carrying 1.89x of headroom and no derivation. + // `4712dac40` narrowed `act(gate)` to the input dtype — upstream's polarity, and + // the only form that reproduces the committed `silu_and_mul_bf16_8x256` oracle + // golden bit-exactly — which adds one legitimate bf16 rounding per activation + // element and grew this envelope 2.8x, to 1.48e-05. A constant a CORRECT kernel + // change consumes was never a bound. #1458. + // + // Two replacements were tried and REJECTED, recorded so nobody re-derives them: + // `bdiff <= diff` is rigorous (the cap is 1-Lipschitz, so it cannot enlarge a + // difference) but does NOT red the swap, because the uncapped envelope is + // 3.4e-04 and the defect is 2.5e-04. A measured twin at `out_mult = 1`, where + // the two orders coincide by algebra, gives 6.10e-06 — but it is a DIFFERENT + // model, and its max|diff| sits at a different point on the tanh knee, so the + // gated run reaches 2.4x it with nothing wrong. The band is knee-driven and is + // kept only at its rigorous Lipschitz value. TinySpec biting; biting.softcap = 1e-3; const MuseGlimmerWeights wb = TinyWeights(MakeConfig(biting)); - const double bdiff = MaxAbsDiff(RunForward(wb, Positions()), RefForward(wb, Positions())); - MESSAGE("muse_glimmer text vs fp32 reference (biting soft-cap): max|diff|=" << bdiff); - CHECK(bdiff <= 1e-5); + const std::vector bgot = RunForward(wb, Positions()); + const std::vector bwant = RefForward(wb, Positions()); + const double bdiff = MaxAbsDiff(bgot, bwant); + double got_sat = 0.0, want_sat = 0.0; + for (float x : bgot) got_sat = std::max(got_sat, std::abs(static_cast(x))); + for (float x : bwant) want_sat = std::max(want_sat, std::abs(static_cast(x))); + // THE PRECONDITION, ASSERTED. Everything above turns on the tanh being + // saturated at this cap; if the fixture ever drifts so that it is not, the two + // orders stop having different ranges and this case degrades silently from an + // order gate into a weak magnitude comparison that the swap no longer reds. + // Saturated, the reference's own maximum IS the cap, and both sides are f32 + // values, so the check is an equality with nothing to tune. + REQUIRE(want_sat > 0.0); + REQUIRE(static_cast(want_sat) == static_cast(biting.softcap)); + // 2^-8 is bf16's UNIT ROUNDOFF -- the largest relative error a correctly + // rounded bf16 store can carry. It is NOT the format's relative spacing + // (machine epsilon), which is twice it at 2^-7. The unit roundoff is what a + // stored value's departure is bounded by, which is what both uses want. + constexpr double kBf16UnitRoundoff = 1.0 / 256.0; + MESSAGE("muse_glimmer text vs fp32 reference (biting soft-cap): max|diff|=" + << bdiff << " (uncapped envelope " << diff << "); saturation ours=" + << got_sat << " ref=" << want_sat << ", delta=" + << std::abs(got_sat - want_sat) << " against " + << (2.0 * kBf16UnitRoundoff * want_sat)); + CHECK(std::abs(got_sat - want_sat) <= 2.0 * kBf16UnitRoundoff * want_sat); + CHECK(bdiff <= diff); } TEST_CASE("muse_glimmer text: embed_norm is WEIGHTLESS RMSNorm, not a sqrt(H) scale") {