From d9614509c48f296759ddd81ab615507bba6e423f Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 12:42:34 +0000 Subject: [PATCH] fix(VT-ACT-ROUND-POLARITY): the four red suites are bounds, not the kernel (#1458) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four model suites are red on `main` behind `4712dac40`, and the issue names the decision plainly: either the narrowing is wider than upstream's, or the bounds were calibrated against the polarity it corrected. It is the second, and the bounds do not get a new constant. ## The kernel is right, re-derived rather than taken on trust Every anchor `4712dac40` cites is exact at the parity pin `555967922`, read in a checkout whose `git rev-parse HEAD` is that sha: - `csrc/libtorch_stable/activation_kernels.cu:158` — `silu_kernel` returns `(T)(((float)x) / (1.0f + expf((float)-x * alpha)))` - `csrc/libtorch_stable/activation_kernels.cu:36` — `compute` returns `(scalar_t)(ACT_FN(gate, alpha) * ((float)up + beta))`, so `ACT_FN` has already narrowed by the time the multiply happens - `csrc/libtorch_stable/activation_kernels.cu:205` — `gelu_tanh_kernel` returns `(T)(0.5f * f * (1.0f + ::tanhf(inner)))`, the same shape - `vllm/model_executor/layers/activation.py:143` and `:418` — the native arms are `F.silu(x[..., :d]) * x[..., d:]` and `F.gelu(x[..., :d], approximate=approximate) * x[..., d:]`, and both yield the input dtype on a bf16 tensor - `csrc/libtorch_stable/activation_kernels.cu:50-95` — the VECTORIZED `packed_compute` narrows too, through `cast_to_packed`, so upstream has ONE polarity across both of its arms - `tests/kernels/core/test_activation.py:108` — `assert_close(out, ref_out, atol=0.0, rtol=0.0)`, which pins the two together bit-exactly, so there is no second behaviour for the `atol=0.0` test to hide The committed oracle golden settles it without reading a kernel at all. Recomputing `tests/parity/goldens/silu_and_mul_bf16_8x256/out.npy` from its own `x.npy` in numpy, with bf16 round-to-nearest-even applied by hand — and every value of that golden is exactly bf16-representable, so it came from a vLLM that rounds: | expression | max abs err vs golden | bit-exact | |---|---|---| | `silu_f32(g) * up`, no store round | 1.434994e-02 | no | | `bf16(silu_f32(g) * up)` — the kernel BEFORE `4712dac40` | 1.562500e-02 | no | | `bf16(silu_f32(g)) * up`, no store round | 7.812500e-03 | no | | `bf16(bf16(silu_f32(g)) * up)` — the kernel AFTER | **0** | **yes** | Reverting is not available. `src/vt/` is untouched by this change. ## The four reproductions At `aeba0de6f`, CPU-only Release (`-DCMAKE_BUILD_TYPE=Release`, so NDEBUG), x86_64, one build directory: | suite | assertion | observed | bound | |---|---|---|---| | `test_ltx2_text_encoder` | `test_ltx2_text_encoder.cpp:2407` `video_bf16 <= video_floor` | 0.1323 | 0.109394 | | `test_ltx2_text_encoder` | `test_ltx2_text_encoder.cpp:2409` `audio_bf16 <= audio_floor` | 0.0752773 | 0.0573374 | | `test_muse_glimmer_text` | `test_muse_glimmer_text.cpp:542` `bdiff <= 1e-5` | 1.47797e-05 | 1e-05 | | `test_muse_glimmer_text_fallback` | the same line, the `VT_FUSED_CHAIN_ADOPT=0` registration of the same binary | 1.47797e-05 | 1e-05 | | `test_minimax_music3_ar` | `test_minimax_music3_ar.cpp:1757` `device_codes == host_codes` | `{11,9,24,3,13,4,9,17}` | `{11,9,24,3,13,4,9,2}` | `test_ops_activation` is 24/24 green throughout and is not one of the four. ## What each bound actually measured **`test_ltx2_text_encoder`** — its floor is the oracle's own f32-vs-bf16 spread carried through the same projection, which is the right SCALE. Its constant was not. The bf16 arm carried `1.0x` while the f32 arm carried `2.0x`, and that `1.0x` was imported from the state-level parity case, where the same two objects are compared elementwise. This case compares them through `Ltx2TextEncoderConditioning`, which stacks all 13 states: the floor is the projection of ONE error vector and the gated quantity is the projection of a DIFFERENT one, and a map that combines 13 states does not preserve a relation that holds pointwise. `out.conditioning` is OUR bf16 realization and `want_bf16` is the ORACLE's; neither is a rounding of the other, so the bound owes a triangle step through the shared f32 trajectory with one floor on each side. Both arms now carry `2.0x`. Video 1.209x, audio 1.313x — margins 0.60 and 0.66. The state-level case is untouched and still green at 0.870x of its `1.0x`. **`test_muse_glimmer_text`** — `bdiff <= 1e-5` was the W1 measurement (5.28e-06 at `3a54c4b7d`, quoted in that commit body) rounded up: 1.89x of headroom and no derivation. The biting-soft-cap case exists to pin the ORDER of the output multiplier and the cap, and that order is an ALGEBRAIC property of the output range rather than a `max|diff|` band. At `cap = 1e-3` the logits drive `|x / cap| ~ 50`, so the tanh saturates to within 1e-40 and upstream's order reaches `cap` while the swap reaches `out_mult * cap` — 1.0e-03 against 7.5e-04 on this fixture. Measured: our saturation and the reference's agree EXACTLY (delta 0), against a bound of two bf16 relative spacings on `cap` (7.81e-06). The knee-driven `max|diff|` is kept at its one rigorous value, `bdiff <= diff`, because the cap is a contraction and cannot enlarge a difference handed to it. **`test_minimax_music3_ar`** — `device_codes == host_codes` held two implementations to an identical argmax. This suite measures their separation at 174.5 to 6924 bf16 ULP across its device/host comparisons; 308 in this case. Draw 6 is a tie: relative top-2 margins 1.95e-03 (device) and 2.93e-03 (host), both BELOW one bf16 ULP (2^-8 = 3.91e-03), between codes 2 and 17. Draws 0..5 have margins 1.74e-02 to 3.53e-01 and both arms agree at every one. The `==` was reading a coin. It now admits a divergence only as a SHARED near-tie, in which each arm ranks the other's pick within one bf16 ULP of its own — the width both arms store at, so a gap narrower than it is decided by rounding and not by the model. ## Two replacements were tried and REJECTED Recorded so nobody re-derives them. For muse, `bdiff <= diff` alone is rigorous but does NOT red the order swap: the uncapped envelope is 3.4e-04 and the defect is 2.5e-04. A measured twin at `out_mult = 1`, where the two orders coincide by algebra, gives 6.10e-06 — but it is a different model sitting at a different point on the tanh knee, and the gated run reaches 2.4x it with nothing wrong. For music3, a top-2-margin threshold alone went QUIET under a dropped-stage mutation, because a grossly wrong arm can land its first divergence on a tie; asking what the other arm's pick was worth in this arm's own distribution does not have that hole. ## Both directions, and the resolution each new bound keeps Fix removed (the three test files at `HEAD`), one build directory, compile rc 0 on every arm, `sha256sum` re-matched after restore: | tree | compile | result | |---|---|---| | fix present | 0 | ltx2 27/27 4118 assertions, muse 24/24 530, muse fallback 24/24 530, music3 37/37 644, all `Status: SUCCESS!` | | fix removed | 0 | ltx2 26/27 (2 failed), muse 23/24 (1), muse fallback 23/24 (1), music3 36/37 (1), all `Status: FAILURE!` | | fix restored | 0 | `sha256sum -c` OK on all three files, green again | Each new bound was then mutated in the PRODUCTION source, never in the test: | mutation | site | verdict | |---|---|---| | scatter the tower states at offset 0 instead of `first_valid` | `ltx2_text_encoder.cpp:1167` | RED, 14.08x and 24.06x the floor — 7x and 12x the new bound | | swap the output multiplier and the soft cap | `muse_glimmer.cpp:447-450` | RED, saturation 7.5e-04 against 1.0e-03, delta 2.5e-04 = 32x the bound | | delete the MLP residual add on the device depth arm | `minimax_music3_depth_device.cpp` | RED, the host ranks the device's pick 1.60e-01 below its own best, 41x the ULP admitted | Every mutation was applied with `git diff --stat` printed after it, the compile exit code printed beside it, and the tree restored and `sha256sum`-verified. ## What this costs, MEASURED — and `4712dac40` spent it, not the new bound The ltx2 conditioning case documents that renumbering the positions from zero reds it at 1.10x the audio floor. Ratios to the propagated floor, one build directory, compile rc 0 on every arm, every source restored and `sha256sum`-verified: | `src/vt/cpu/cpu_ops.cpp` | production code | video | audio | verdict at the OLD `1.0x` | |---|---|---:|---:|---| | before `4712dac40` | correct | 0.565 | 0.688 | pass | | before `4712dac40` | renumbered | 0.831 | **1.099** | **RED** — the documented 1.10x | | at `aeba0de6f` | correct | **1.209** | **1.313** | **RED** | | at `aeba0de6f` | renumbered | 0.683 | 0.931 | pass | Read the bottom two rows together. Post-`4712dac40` the instrument is INVERTED: it reds the correct code and passes the mutant, and the mutant is measurably CLOSER to the oracle than the port is. So the renumbering coverage was already gone on `main` before this change, and what this change does is strictly restore a functioning instrument — the correct code passes and every O(1) defect the case exists for still reds. It does not recover that detection, and no constant can, because the ordering of the two has reversed. Filed as [#1467](https://github.com/mudler/vllm.cpp/issues/1467), owed by [`ltx-2-5.md`](.agents/specs/ltx-2-5.md), with the two candidate instruments that have no bf16 accumulation between the defect and the assertion. `test_muse_glimmer_text`'s other band, `diff <= 5e-4`, is the same rounded-up W1 measurement class; `4712dac40` moved it from 0.242 to 0.687 of its bound. Green, out of scope for this repair because it needs its own derivation, and filed as [#1466](https://github.com/mudler/vllm.cpp/issues/1466), owed by [`muse-glimmer.md`](.agents/specs/muse-glimmer.md). ## The narrowing did not make the port worse At the state level, where the comparison is elementwise and the amplification above does not apply, our distance to the f32 oracle IMPROVED at 11 of the 13 gemma4 tower states (worst arm ratio 0.644 -> 0.572) and the bf16 arm stayed inside its `1.0x` floor (0.714 -> 0.870). ## Records `#1458`'s index row landed with [#1457](https://github.com/mudler/vllm.cpp/pull/1457), the pull request that filed it, already naming `VT-ACT-ROUND-POLARITY` as the owner. Appending a second row is exactly the duplicate `merge=union` produces from two branches, and `scripts/check-agent-record.py` refuses it, so this change does not add one. The three places that must agree are that row, the spec section this change adds, and this body. Two rows ARE appended, for the two gaps found in flow: [#1466](https://github.com/mudler/vllm.cpp/issues/1466) under `MODEL-MUSE-GLIMMER` and [#1467](https://github.com/mudler/vllm.cpp/issues/1467) under `MODEL-DIFFUSION-ltx-2-5-ltx2-video-transformer-3d-model`, each also listed under its spec's `## Owed`. **And one pre-existing record defect is repaired, because this change writes immediately above it.** `.agents/specs/vt-act-round-polarity.md` carried a sentence whose inline code span was split across 30 lines of intervening document, leaving a bare `## Owed` fragment ~285 lines before the real section. `owed_issues()` splits on the FIRST `\n## Owed`, so it parsed the 4-line fragment. Measured on the file as it stands on `origin/main` versus repaired: | spec text | `owed_issues()` parse | |---|---| | `origin/main` | `[]` — a 231-character fragment | | repaired | `['1322', '1342', '1343']` — the real 2265-character section | It is harmless today only because all three issues also name a Row in the index. An issue owed by this spec alone would have been invisible to the ownership gate and would have silently consumed the `UNOWNED_HIGH_WATER` ratchet. It came from `4712dac40` — the same row — and it is one line, in flow. ## Gate Full `ctest` at the branch head, on `origin/main` `fdefb4529`, CPU-only Release (so NDEBUG), x86_64, `-j 4`: **573 of 573 passed, 0 failed**. The three suites individually, with non-zero case counts and the `Status:` line read rather than the assertion line: `test_ltx2_text_encoder` 27/27 (4118 assertions), `test_muse_glimmer_text` 24/24 (531), the same binary under `VT_FUSED_CHAIN_ADOPT=0` 24/24 (531), `test_minimax_music3_ar` 37/37 (649) — all `Status: SUCCESS!`. The assertion counts RISE against `main` (528 -> 531, 640 -> 649): this change adds gates, it does not remove them. `scripts/check-agent-record.py` is rc 0 (`ENGINE=168 MODEL=377 QUANT=84 KERNEL=53 BACKEND=85 ANCHOR-ROT=38`) and `scripts/audit-live-rows.py` output is BYTE-IDENTICAL to the same script run on a detached `origin/main` worktree, so neither record surface moves. `scripts/agent-preflight.sh --staged` is rc 0. `test_cpu_x86_llamacpp_floor` reds there when the box is loaded (`waiting for quiet: busy=117% load=76.31`), which is [#618](https://github.com/mudler/vllm.cpp/issues/618) and load-dependent. An earlier intermediate run of this branch reported `test_engine_core_proc` red at `tests/vllm/v1/test_engine_core_proc.cpp:481` on `CHECK(abort_seen)`. That is [#1052](https://github.com/mudler/vllm.cpp/issues/1052), which names that file and line, and it is not this change: nothing here touches `v1/`, it passes on its own (`ctest -R`, rc 0), and six concurrent runs of the binary are 6/6 `Status: SUCCESS!`. It needs no new issue. FOLLOWING_AGENTS_PROTOCOL Closes #1458 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 2 + .agents/specs/ltx-2-5.md | 16 +++ .agents/specs/muse-glimmer.md | 13 ++ .agents/specs/vt-act-round-polarity.md | 139 ++++++++++++++++++- tests/vllm/models/test_ltx2_text_encoder.cpp | 59 +++++++- tests/vllm/models/test_minimax_music3_ar.cpp | 114 ++++++++++++++- tests/vllm/models/test_muse_glimmer_text.cpp | 62 ++++++++- 7 files changed, 386 insertions(+), 19 deletions(-) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 161856a04..8e14e0eda 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -490,3 +490,5 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1403](https://github.com/mudler/vllm.cpp/issues/1403) | `ENG-CUDAGRAPH-BREAK` | **`test_qwen3_5_decode_graph_seam` SIGSEGVs — a DUPLICATE of [#1394](https://github.com/mudler/vllm.cpp/issues/1394) that was already fixed when it was filed, so no code change was owed.** Filed against `96ed8346f`, which PREDATES the fix `7dec1d990` (PR #1393, merged 2026-08-20 00:49); `git merge-base --is-ancestor` puts the fix outside the reported tree. Re-measured at `b537a5344`, CPU-only Release x86_64: the test passes 5 runs of 5, `8 cases`/`8 passed`, `assertions: 138`, `Status: SUCCESS!`, exit 0. The cause was re-derived by single-variable mutation rather than inherited from #1394's report, and NAMED by a sanitizer rather than reasoned. Debug + `VLLM_CPP_SANITIZE=address,undefined` (NDEBUG OFF) with BOTH halves of `7dec1d990` reverted to their `96ed8346f` bytes: `AddressSanitizer: SEGV on unknown address 0x5045f5f84900`, `The signal is caused by a READ memory access`, at `src/vt/cpu/cpu_paged_attn.cpp:59` in `KvElem` reached from `:224`, on threadpool worker T1, inside case `W6: two spec shapes of EQUAL S and different q get two graphs` (`tests/vllm/models/test_qwen3_5_decode_graph_seam.cpp:800`). **It is NOT a use-after-free and there is no free site**, and `AddressSanitizer can not provide additional info` is the tell: the out-of-bounds block-table read is itself IN-BOUNDS for ASan because it lands inside the neighbouring pooled allocation, so this is not a `heap-buffer-overflow` either — the VALUE read becomes a wild block index that `KvElem` multiplies by the KV block stride and dereferences, which is why the fault address is unmapped rather than a redzone and why the crash is order-dependent instead of deterministic. Both halves are load-bearing: reverting ONLY the `SpecAttnMeta` fixture leaves the kernel bound to refuse by name at `src/vt/cpu/cpu_paged_attn.cpp:152` (exit 1, the case THROWS); reverting the bound as well restores the crash (Release exit 139, 3 runs of 3; ASan exit 1); restoring both gives exit 0, 8/8, zero sanitizer findings. The issue's `5f68e60df` pointer was right about the commit and wrong about the half — `git log -S'SpecAttnMeta'` and `git log -S'W6: two spec shapes of EQUAL S'` each return that commit ALONE, and both hits sit in the TEST fixture, not in the graph eligibility it widened. [#1405](https://github.com/mudler/vllm.cpp/issues/1405)'s truncation reproduced (`assertions: 135` crashed vs `138` complete) and is SHARPER under a threadpool fault: doctest printed the summary TWICE with different totals, `135` then `141`, because the main thread kept running after a worker died. That a red reached `main` unreported is [#1285](https://github.com/mudler/vllm.cpp/issues/1285)/[#1376](https://github.com/mudler/vllm.cpp/issues/1376), not this row | bug | | [#1458](https://github.com/mudler/vllm.cpp/issues/1458) | `VT-ACT-ROUND-POLARITY` | **`4712dac40` reds FOUR suites on `main` — `test_ltx2_text_encoder`, `test_muse_glimmer_text`, `test_muse_glimmer_text_fallback`, `test_minimax_music3_ar` — by exceeding bf16 error floors none of them had re-derived.** Found while gating [#1403](https://github.com/mudler/vllm.cpp/issues/1403) (PR #1457) and filed in flow, not that row's defect. Deterministic, not a load artifact: first seen under `-j4` and re-measured SERIALLY, 101 s across the four. `test_ltx2_text_encoder.cpp:2407` reads `CHECK( 0.1323 <= 0.109394 )` and `:2409` reads `CHECK( 0.0752773 <= 0.0573374 )`, both over by 20-30%, with the file at `26 passed / 1 failed`, `4118 assertions`, `Status: FAILURE!`. ATTRIBUTED BY MUTATION rather than inferred, both directions, in one build directory changing only `src/vt/cpu/cpu_ops.cpp`, compile rc 0 on every arm and `sha256sum` taken before and after: at `0adeb8b0e` four fail; reverted to `4712dac40^` four pass and `test_ltx2_text_encoder` is 27/27 `SUCCESS!`; restored to a re-matched `sha256` four fail again. They also passed at `b537a5344`, three commits earlier, in a full 567-test `ctest` run at 3-6 s each. `4712dac40` is +42/-3 in `src/vt/cpu/cpu_ops.cpp` plus a new `tests/vt/test_ops_activation.cpp`, and touches none of the four. NOT FIXED IN FLOW, deliberately: the question is a numerics decision, not a defect with an obvious repair — either the floors were calibrated against the rounding polarity that commit corrected and need re-deriving against the oracle, or the narrowing is wider than upstream's — and editing the four floors to pick the first branch is the scope-widening `AGENTS.md` prohibits | bug | | [#1469](https://github.com/mudler/vllm.cpp/issues/1469) | `KERNEL-DFLASH2-GROUPED-CONV` | **`test_audit_live_rows` is RED on `main`.** Measured on a detached worktree at `origin/main` `cae8ace0c` with nothing applied: `test_shipped_record_has_no_abandoned_active_row` fails with `stale ACTIVE rows remain: ['KERNEL-DFLASH2-GROUPED-CONV']`, 57 tests, 1 failure, and `scripts/agent-preflight.sh` reports both `audit-live-rows` and `test_audit_live_rows` as FAILED. `scripts/audit-live-rows.py` alone returns rc 0 and prints `245 live rows; 1 abandoned ACTIVE`, so the count is the gate and the script's exit status does not carry it. The row is `.agents/kernel-matrix.md:143`, state `ACTIVE`, claim `CLAIM-SPEC-DFLASH2-W2`, and its implementation landed on `main` in `028438e68` (SPEC-DFLASH2 W2, [#1314](https://github.com/mudler/vllm.cpp/issues/1314), PR #1465) - a row still claimed as in-progress whose work is already reachable on `main`. NOT FIXED IN FLOW: the kernel matrix, the row and the DFlash2 claim are outside the authority of the row that found it (`LTX25-DEVICE-RESIDENCY` W0, PR #1441), and a lifecycle move owes `docs/STATUS.md`, `docs/BENCHMARKS.md` and the moved row spec's `## Now`. Found while running the full preflight for that row's gate, where it is an inherited red | bug | +| [#1466](https://github.com/mudler/vllm.cpp/issues/1466) | `MODEL-MUSE-GLIMMER` | **`tests/vllm/models/test_muse_glimmer_text.cpp:532`'s `CHECK(diff <= 5e-4)` is a rounded-up W1 measurement, and one correct kernel change already spent 45 points of its 76-point margin.** `3a54c4b7d`'s own body quotes the number the constant was rounded up from ("max abs diff 1.21e-4 on logits of max 4.88e-2"); there is no derivation beside it. `4712dac40` narrowed `act(gate)` to the input dtype — upstream's polarity, and the only form that reproduces the committed `silu_and_mul_bf16_8x256` oracle golden bit-exactly — and grew the envelope 2.8x to 3.43e-04, which is 0.687 of the bound (measured at `aeba0de6f`, CPU-only Release, x86_64). It is the last member of that class in this file: [#1458](https://github.com/mudler/vllm.cpp/issues/1458) replaces the other one (`bdiff <= 1e-5`) in the same case and leaves this one to its own derivation, because a bigger constant is not a repair. Found in flow while gating #1458 (PR [#1461](https://github.com/mudler/vllm.cpp/pull/1461)); also listed under `## Owed` in [`muse-glimmer.md`](specs/muse-glimmer.md) | bug | +| [#1467](https://github.com/mudler/vllm.cpp/issues/1467) | `MODEL-DIFFUSION-ltx-2-5-ltx2-video-transformer-3d-model` | **The ltx2 prompt->conditioning case no longer detects position renumbering, and post-`4712dac40` the instrument is INVERTED — the mutant is closer to the oracle than the correct code.** MEASURED in one build directory, CPU-only Release, x86_64, compile rc 0 on every arm and every source restored `sha256sum`-verified, as ratios to the propagated floor: before `4712dac40`, correct 0.565/0.688 and renumbered 0.831/**1.099** (the 1.10x the case's own note claims, and it reded); at `aeba0de6f`, correct **1.209/1.313** and renumbered 0.683/0.931 — so at the old `1.0x` bound the case reds the port and passes the defect. `4712dac40` is right; what it did here was raise this comparison's noise floor above the defect's signal, and no constant recovers a detection whose ordering has reversed. [#1458](https://github.com/mudler/vllm.cpp/issues/1458) restores a functioning instrument at a derived `2.0x` (its scatter-offset mutation still reds at 14.08x/24.06x) and does not claim this coverage. A repair owes an instrument with no bf16 accumulation between the defect and the assertion — the integer `positions` contract, or the f32 rope table `scripts/gen-ltx2-gemma-tower-goldens.py:363-375` already names as the right one for this class. Found in flow while gating #1458 (PR [#1461](https://github.com/mudler/vllm.cpp/pull/1461)); also listed under `## Owed` in [`ltx-2-5.md`](specs/ltx-2-5.md) | bug | diff --git a/.agents/specs/ltx-2-5.md b/.agents/specs/ltx-2-5.md index 975a5d9c3..2827660ff 100644 --- a/.agents/specs/ltx-2-5.md +++ b/.agents/specs/ltx-2-5.md @@ -947,6 +947,22 @@ none of them. Per AGENTS.md that is the worse half of the silent/refused split: naming a missing part is documented debt, and silence is not. Each now has its own issue, each saying what is absent, what a future row starts from, and what blocks it. +- [#1467](https://github.com/mudler/vllm.cpp/issues/1467) — the + `"ltx2 prompt -> conditioning: the VALUES"` case no longer detects position + renumbering, and the note in it that claims 1.10x-of-floor detection is stale. + MEASURED while gating [#1458](https://github.com/mudler/vllm.cpp/issues/1458): + before `4712dac40` the mutant reached 1.099x the audio floor and reded; at + `aeba0de6f` it reaches 0.683x/0.931x while the CORRECT code reaches + 1.209x/1.313x, so the instrument is INVERTED — the mutant is closer to the + oracle than the port is. `4712dac40` is right (it is the only form that + reproduces the `silu_and_mul_bf16_8x256` oracle golden bit-exactly); it raised + this comparison's noise floor above the defect's signal. #1458 restores a + functioning instrument and does not recover the detection, and no constant on + this quantity can. A repair owes an instrument with no bf16 accumulation + between the defect and the assertion — the integer `positions` contract, or + the f32 rope table the generator already names as the right one for this class + (`scripts/gen-ltx2-gemma-tower-goldens.py:363-375`). + - [#1093](https://github.com/mudler/vllm.cpp/issues/1093) — `TI2VidTwoStagesPipeline` (`ti2vid_two_stages.py:61`). NOT our `distilled_two_stage`: stage 1 is CFG-guided on the FULL model (`:247-259`), stage 2 carries the distilled LoRA alone (`:151`), and stage-1 diff --git a/.agents/specs/muse-glimmer.md b/.agents/specs/muse-glimmer.md index 6f2224676..4a94aa27e 100644 --- a/.agents/specs/muse-glimmer.md +++ b/.agents/specs/muse-glimmer.md @@ -1177,3 +1177,16 @@ about magnitudes. **Two greedy prompts are not a token-exactness claim.** - **The drafter has still never run.** §10.5's acceptance A/B is unchanged. - **No speed axis.** §0 stands: the pinned oracle cannot load `muse_glimmer`, so there is no denominator and none is claimed here on any axis. + +## Owed + +- [#1466](https://github.com/mudler/vllm.cpp/issues/1466) — + `tests/vllm/models/test_muse_glimmer_text.cpp:532`, `CHECK(diff <= 5e-4)` is + the W1 measurement rounded up (`3a54c4b7d`'s body quotes 1.21e-04) with no + derivation beside it. `4712dac40` gave `vt`'s gated activations upstream's + rounding polarity, which is correct, and grew the envelope 2.8x to 3.43e-04 — + 0.687 of the bound. Found while gating + [#1458](https://github.com/mudler/vllm.cpp/issues/1458), which repairs the + other member of the same class in the same case (`bdiff <= 1e-5`) and + deliberately leaves this one to its own derivation. A repair owes a measured + floor or a precision argument, never a bigger constant. diff --git a/.agents/specs/vt-act-round-polarity.md b/.agents/specs/vt-act-round-polarity.md index dc09b245e..9ab642d31 100644 --- a/.agents/specs/vt-act-round-polarity.md +++ b/.agents/specs/vt-act-round-polarity.md @@ -3,7 +3,10 @@ Issues: [#1322](https://github.com/mudler/vllm.cpp/issues/1322) (the filed gap), [#1342](https://github.com/mudler/vllm.cpp/issues/1342) (Vulkan/Metal silu spelling), [#1343](https://github.com/mudler/vllm.cpp/issues/1343) -(`RmsNormPlusAdd` arm asymmetry). +(`RmsNormPlusAdd` arm asymmetry), +[#1458](https://github.com/mudler/vllm.cpp/issues/1458) (four model suites red +on `main` behind the landed narrowing; fixed, see `## The four suites #1458 +reds, and why the kernel is not the defect`). Row: `VT-ACT-ROUND-POLARITY`. ## Now @@ -19,7 +22,18 @@ existing f32 golden and every existing byte-exact composite contract unmoved, and it is asserted directly rather than inferred (see `tests/vt/test_ops_activation.cpp`, "leaves an f32 INPUT bit-identical"). -**Five providers are still owed and are listed under ` +**The narrowing survived a second, independent challenge.** +[#1458](https://github.com/mudler/vllm.cpp/issues/1458) reported four model +suites red behind `4712dac40`, attributed by mutation in both directions. The +kernel was re-verified from source at the pin and from the committed oracle +golden, and it is right; the four bounds were not. See `## The four suites #1458 +reds, and why the kernel is not the defect`. + +**Five providers are still owed and are listed under `## Owed`.** CUDA, ROCm, +Metal and Tenstorrent cannot be compiled from the dev box at all, and the Vulkan +arm additionally needs a GLSL toolchain to regenerate its committed SPIR-V. This +row does NOT claim provider parity it did not test. + ## What landed, and what it measured `RoundThrough(DType, float)` — the helper `kRmsNormGatedGroup` already uses for @@ -49,11 +63,6 @@ golden pass ran 46 cases on both sides: the table under `## Premise`, where the worst element margin against the harness's own `atol + rtol*|want|` moves 0.5364 -> 0.2989. -## Owed`.** CUDA, ROCm, -Metal and Tenstorrent cannot be compiled from the dev box at all, and the Vulkan -arm additionally needs a GLSL toolchain to regenerate its committed SPIR-V. This -row does NOT claim provider parity it did not test. - ## Gap verification Checked before writing anything, because several changes this session were found @@ -372,6 +381,122 @@ That is why this spec stops here. box for the byte-exact composite suites, a ROCm box, a Mac, and a glslang for the SPIR-V regenerate. +## The four suites #1458 reds, and why the kernel is not the defect + +[#1458](https://github.com/mudler/vllm.cpp/issues/1458) reported +`test_ltx2_text_encoder`, `test_muse_glimmer_text`, +`test_muse_glimmer_text_fallback` and `test_minimax_music3_ar` red on `main` +behind `4712dac40`, attributed by mutation in both directions. All four +reproduce at `aeba0de6f`, CPU-only Release (`-DCMAKE_BUILD_TYPE=Release`, so +NDEBUG), x86_64. + +**The premise was re-derived rather than taken.** Every anchor the landing +commit cites is exact at the pin `555967922`, read in the checkout whose +`git rev-parse HEAD` is that sha: +`csrc/libtorch_stable/activation_kernels.cu:158` (`silu_kernel` returns +`(T)(...)`), `:36` (`compute` returns `(scalar_t)(ACT_FN(gate, alpha) * ...)`, +so `ACT_FN` has already narrowed), `:205` (`gelu_tanh_kernel`, same shape), +`vllm/model_executor/layers/activation.py:143` and `:418` (the native arms, +`F.silu` / `F.gelu` on a bf16 tensor yielding bf16), and +`tests/kernels/core/test_activation.py:108` +(`assert_close(out, ref_out, atol=0.0, rtol=0.0)`, which pins the two together +bit-exactly). The VECTORIZED arm agrees: `packed_compute` at +`activation_kernels.cu:50-95` narrows through `cast_to_packed` as +well, so upstream has ONE polarity across both of its arms and the `atol=0.0` +test has no second behaviour to hide. + +**The committed oracle golden settles it without reading a kernel.** +Recomputing `tests/parity/goldens/silu_and_mul_bf16_8x256/out.npy` from its own +`x.npy` in numpy, with bf16 round-to-nearest-even applied by hand: + +| expression | max abs err vs golden | bit-exact | +|---|---|---| +| `silu_f32(g) * up`, no store round | 1.434994e-02 | no | +| `bf16(silu_f32(g) * up)` — the PRE-`4712dac40` kernel | 1.562500e-02 | no | +| `bf16(silu_f32(g)) * up`, no store round | 7.812500e-03 | no | +| `bf16(bf16(silu_f32(g)) * up)` — the POST kernel | **0** | **yes** | + +The golden's every value is exactly bf16-representable, so it came from a vLLM +that rounds twice. The landed kernel is the only one of the four that reproduces +it. Reverting it is not available. + +**What the four bounds were, and what each one actually measured.** + +- `test_ltx2_text_encoder`, the prompt->conditioning VALUES case. Its floor is + the oracle's own f32-vs-bf16 spread carried through the same projection, which + is the right SCALE. Its constant was not: the bf16 arm carried `1.0x` while the + f32 arm carried `2.0x`, and the `1.0x` was imported from the state-level parity + case, where the same two objects are compared elementwise. This case compares + them through `Ltx2TextEncoderConditioning`, which stacks all 13 states — the + floor is the projection of ONE error vector and the gated quantity is the + projection of a DIFFERENT one, and a linear map that combines 13 states does + not preserve the relation. Both arms now carry `2.0x`, which is the triangle + bound with each side's departure from the shared f32 trajectory taken at one + floor. Video 1.209x -> margin 0.60; audio 1.313x -> margin 0.66. + The state-level case is untouched and still green at 0.870x of its `1.0x`. +- `test_muse_glimmer_text` and its `VT_FUSED_CHAIN_ADOPT=0` registration. + `bdiff <= 1e-5` was the W1 measurement (5.28e-06 at `3a54c4b7d`) rounded up, + 1.89x of headroom and no derivation. The biting-soft-cap case exists to pin the + ORDER of the output multiplier and the cap, and the order is an ALGEBRAIC + property of the output range, not a max|diff| band: at `cap = 1e-3` the logits + saturate the tanh to within 1e-40, so upstream's order reaches `cap` and the + swap reaches `out_mult * cap`. Measured: our saturation and the reference's + agree EXACTLY (delta 0) and the swap moves it by 2.5e-04, 32x the bound, which + is two bf16 relative spacings on `cap`. The knee-driven max|diff| is kept at + its rigorous Lipschitz value (`bdiff <= diff`; the cap is a contraction). +- `test_minimax_music3_ar`, the composed depth stage. `device_codes == + host_codes` over an argmax on two implementations whose measured separation is + 308 bf16 ULP. Draw 6 is a tie: relative top-2 margins 1.95e-03 (device) and + 2.93e-03 (host), both BELOW one bf16 ULP (2^-8 = 3.91e-03), between codes 2 and + 17. Draws 0..5 have margins 1.74e-02 to 3.53e-01 and agree. The `==` was + reading a coin. It is now: codes agree, or the divergence is a SHARED near-tie + in which each arm ranks the other's pick within one bf16 ULP of its own. + +**Two candidate replacements were tried and rejected**, recorded so nobody +re-derives them. For muse, `bdiff <= diff` alone is rigorous but does NOT red the +order swap (uncapped envelope 3.4e-04, defect 2.5e-04). A measured twin at +`out_mult = 1`, where the two orders coincide by algebra, gives 6.10e-06 — but it +is a different model sitting at a different point on the tanh knee, and the gated +run reaches 2.4x it with nothing wrong. + +**What this costs, MEASURED, and it is `4712dac40` that spent it rather than the +new bound.** The ltx2 conditioning case documents that renumbering the positions +from zero reds it at 1.10x the audio floor. Ratios to the propagated floor, one +build directory, compile rc 0 on every arm, every source restored and +`sha256sum`-verified: + +| `src/vt/cpu/cpu_ops.cpp` | production code | video | audio | at the old `1.0x` | +|---|---|---:|---:|---| +| before `4712dac40` | correct | 0.565 | 0.688 | pass | +| before `4712dac40` | renumbered | 0.831 | **1.099** | **RED** — the documented 1.10x | +| at `aeba0de6f` | correct | **1.209** | **1.313** | **RED** | +| at `aeba0de6f` | renumbered | 0.683 | 0.931 | pass | + +Read the bottom two rows together: post-`4712dac40` the instrument is INVERTED — +it reds the correct code and passes the mutant, and the mutant is measurably +CLOSER to the oracle than the port is. The `2.0x` restores a functioning +instrument; it does not recover that detection, and no constant can, because the +ordering of the two has reversed. It is also a property of the defect — +upstream's own f32 answers for positions 12..19 and 0..7 agree to 3.6e-06 +relative — and `scripts/gen-ltx2-gemma-tower-goldens.py:363-375` already records +that the end-to-end states are the wrong instrument for that class and the f32 +rope table is the right one. Filed as +[#1467](https://github.com/mudler/vllm.cpp/issues/1467) and owed by +[`ltx-2-5.md`](ltx-2-5.md). + +**One more bound of the same class is left standing, and it is filed rather than +re-tuned.** `tests/vllm/models/test_muse_glimmer_text.cpp:532`'s +`CHECK(diff <= 5e-4)` is the other W1 measurement rounded up (1.21e-04 at +`3a54c4b7d`), and `4712dac40` moved it from 0.242 to 0.687 of its bound. It is +green, it is out of scope for this repair because it needs its own derivation, +and it is [#1466](https://github.com/mudler/vllm.cpp/issues/1466), owed by +[`muse-glimmer.md`](muse-glimmer.md). + +**The narrowing did not make the port worse.** At the state level, where the +comparison is elementwise and the amplification above does not apply, our +distance to the f32 oracle IMPROVED at 11 of the 13 states (worst ratio 0.644 -> +0.572) and the bf16 arm stayed inside its `1.0x` floor (0.714 -> 0.870). + ## Stop conditions - A golden that gets **worse** after the activation change stops the row and is diff --git a/tests/vllm/models/test_ltx2_text_encoder.cpp b/tests/vllm/models/test_ltx2_text_encoder.cpp index 9e136309e..0f5a0d41f 100644 --- a/tests/vllm/models/test_ltx2_text_encoder.cpp +++ b/tests/vllm/models/test_ltx2_text_encoder.cpp @@ -2382,7 +2382,8 @@ TEST_CASE("ltx2 prompt -> conditioning: the VALUES, against the left-padded orac // The floor, propagated rather than borrowed: it is the oracle's OWN // f32-vs-bf16 spread carried through the identical projection, so it is the // smallest difference this comparison can resolve. It cannot be widened to - // rescue a failure without regenerating the oracle. + // rescue a failure without regenerating the oracle; what the two CHECKs below + // carry is a derivation over it, stated where they stand. auto max_diff = [](const std::vector& a, const std::vector& b) { if (a.size() != b.size()) return std::numeric_limits::infinity(); double d = 0.0; @@ -2404,9 +2405,61 @@ TEST_CASE("ltx2 prompt -> conditioning: the VALUES, against the left-padded orac << "x the propagated floor " << video_floor << "), vs f32 oracle = " << video_f32 << "; audio " << audio_bf16 << " (" << (audio_bf16 / audio_floor) << "x " << audio_floor << ") / " << audio_f32); - CHECK(video_bf16 <= video_floor); + // BOTH arms carry the SAME bound. The reasoning is the triangle step the + // state-level parity case above spells out; the NUMBER comes from this case's + // own measurements, cited below. `out.conditioning` is OUR bf16 realization, + // `want_bf16` is the ORACLE's, and neither is a rounding of the other. Both depart from the shared f32 trajectory `want_f32`, so + // + // |ours - oracle_bf16| <= |ours - oracle_f32| + |oracle_f32 - oracle_bf16| + // + // and the second term IS the propagated floor. The first term is `video_f32`, + // which THIS case measures a line above: 0.79x the floor on video and 0.475x + // on audio, so taking it at one floor is the measurement, not an assumption. + // Hence 2x on the bf16 arm as well as on the f32 one. + // + // WHY NOT 3x, since the sibling assertion below gates that first term at 2x. + // Chaining the two ASSERTED bounds would give 3x, and 3x is a bound that can + // never fire: `|ours - oracle_bf16| <= |ours - oracle_f32| + floor` is the + // triangle inequality itself, so once the f32 arm passes at 2x the bf16 arm at + // 3x is arithmetic rather than a gate. 2x is the tighter value the measured + // premise supports, and it is the one that can still say something. + // + // WHY IT USED TO SAY 1x, AND WHY THAT WAS NEVER DERIVED. The state-level case + // gates `d_bf16 <= floor` and derives its f32 arm from that primitive. This + // case IMPORTED the 1x across `Ltx2TextEncoderConditioning`, which stacks all + // 13 states and combines them. A per-state relation that holds elementwise + // does not survive that: the floor is the projection of ONE error vector and + // this quantity is the projection of a DIFFERENT one, and the projection can + // amplify the second relative to the first. It held until the seam's bf16 + // rounding polarity was corrected to upstream's in `4712dac40`, which re-rolled + // both error directions and moved the video arm from 0.56x to 1.21x and the + // audio arm from 0.69x to 1.31x — with our distance to the f32 oracle + // IMPROVING at 11 of the 13 states. The old constant broke, not the port. + // #1458. + // + // WHAT THIS COSTS, MEASURED rather than estimated, and it is not this bound + // that spent it. The position-renumbering note at the top of this case claims + // detection at 1.10x the audio floor. Ratios to the propagated floor, one + // build directory, every source restored sha256-verified: + // + // cpu_ops.cpp code video audio + // before 4712dac40 correct 0.565 0.688 -> pass at 1.0x + // before 4712dac40 renumbered 0.831 1.099 -> RED at 1.0x + // at aeba0de6f correct 1.209 1.313 -> RED at 1.0x + // at aeba0de6f renumbered 0.683 0.931 -> pass at 1.0x + // + // Read the bottom two rows together: post-`4712dac40` the instrument is + // INVERTED, reding the correct code and passing the mutant, and the mutant is + // measurably CLOSER to the oracle than the port is. The 2x here restores a + // functioning instrument; it does not recover that detection, and no constant + // can, because the ordering of the two has reversed. It is also a property of + // the defect — upstream's own f32 answers for positions 12..19 and 0..7 agree + // to 3.6e-06 relative — and `gen-ltx2-gemma-tower-goldens.py:363-375` already + // records that the end-to-end states are the wrong instrument for this class + // and the f32 rope table is the right one. Owed as #1467. + CHECK(video_bf16 <= 2.0 * video_floor); CHECK(video_f32 <= 2.0 * video_floor); - CHECK(audio_bf16 <= audio_floor); + CHECK(audio_bf16 <= 2.0 * audio_floor); CHECK(audio_f32 <= 2.0 * audio_floor); // The reorder and the mask, compared EXACTLY — but read what that does and does diff --git a/tests/vllm/models/test_minimax_music3_ar.cpp b/tests/vllm/models/test_minimax_music3_ar.cpp index 492b41675..f7a216266 100644 --- a/tests/vllm/models/test_minimax_music3_ar.cpp +++ b/tests/vllm/models/test_minimax_music3_ar.cpp @@ -1698,14 +1698,37 @@ TEST_CASE("music3 ar: the COMPOSED depth stage TAKES the device arm and draws th int64_t host_draws = 0; int64_t device_draws = 0; - const auto make_sampler = [](int64_t* counter) { + // Each draw records the code it took AND the probability vector it took it + // from. The code comparison below needs the vector: an argmax whose top two + // candidates sit closer together than the two arms' own arithmetic resolution + // is not a property either arm can be held to, and the only way to say that + // without asserting a coincidence is to ask what the OTHER arm's pick was + // worth in this arm's own distribution. See #1458. + struct DrawRecord { + int64_t code = 0; + std::vector probs; + double best() const { return static_cast(probs[static_cast(code)]); } + double runner_up() const { + double r = -std::numeric_limits::infinity(); + for (size_t i = 0; i < probs.size(); ++i) + if (static_cast(i) != code) r = std::max(r, static_cast(probs[i])); + return r; + } + }; + std::vector host_rec; + std::vector device_rec; + const auto make_sampler = [](int64_t* counter, std::vector* rec) { return m3::Music3CodeSampler( - [counter](const std::vector& probs, const m3::Music3Draw&) -> int64_t { + [counter, rec](const std::vector& probs, const m3::Music3Draw&) -> int64_t { ++*counter; size_t best = 0; for (size_t i = 1; i < probs.size(); ++i) { if (probs[i] > probs[best]) best = i; } + DrawRecord r; + r.code = static_cast(best); + r.probs = probs; + rec->push_back(std::move(r)); return static_cast(best); }); }; @@ -1714,8 +1737,8 @@ TEST_CASE("music3 ar: the COMPOSED depth stage TAKES the device arm and draws th // default-constructed arm — which is what every existing caller passes. std::vector host_codes{semantic_code}; const std::vector host = m3::Music3DepthStage( - last_conditional, last_unconditional, frame_index, weights, make_sampler(&host_draws), - &host_codes); + last_conditional, last_unconditional, frame_index, weights, + make_sampler(&host_draws, &host_rec), &host_codes); vt::Queue queue = CpuQueue(); m3::DepthDecoderWeights stage_source = depth; @@ -1731,7 +1754,7 @@ TEST_CASE("music3 ar: the COMPOSED depth stage TAKES the device arm and draws th std::vector device_codes{semantic_code}; const std::vector device = m3::Music3DepthStage(last_conditional, last_unconditional, frame_index, weights, - make_sampler(&device_draws), &device_codes, arm); + make_sampler(&device_draws, &device_rec), &device_codes, arm); const uint64_t after = m3::Music3DepthDeviceForwardCount(); // THE ASSERTION #1131 SAYS IS MISSING. `num_codebooks` appends a frame: one @@ -1754,7 +1777,86 @@ TEST_CASE("music3 ar: the COMPOSED depth stage TAKES the device arm and draws th // A tolerance CANNOT see a dropped stage — the prefix append, the fed-back // projection row — because a schedule missing one still produces finite, // plausible numbers. The drawn codes can. - CHECK_MESSAGE(device_codes == host_codes, "the two arms drew different residual codes"); + // + // WHAT THE CODES CANNOT DECIDE, and why this is a per-draw comparison rather + // than one `==`. The two arms are two implementations of the same block, not + // two runs of one: the device arm goes through `vt` GEMMs and the host arm + // through the `ArCompute::kBFloat16` reference, and this suite measures their + // separation at 174.5 to 6924 bf16 ULP across its device/host comparisons — + // 308 in this case. Both store at bf16, so their probabilities agree to about + // one bf16 unit roundoff: 2^-8, the largest relative error a store at that + // width can carry. An argmax whose top two candidates are separated by LESS + // than that is decided by rounding, and holding the two arms to the same code + // there asserts a coincidence, not a guarantee. + // + // MEASURED on this fixture at `aeba0de6f`: draws 0..5 have relative top-2 + // margins of 1.74e-02 to 3.53e-01 — 4.5x to 90x the resolution — and both arms + // agree at every one. Draw 6 is a tie at 1.95e-03 (device) and 2.93e-03 + // (host), BELOW one unit roundoff, between codes 2 and 17, and each arm + // ranks the other's pick inside it. It flipped when `4712dac40` gave + // `vt`'s gated activations upstream's rounding polarity, which is the correct + // polarity and the only one that reproduces the committed + // `silu_and_mul_bf16_8x256` oracle golden bit-exactly. The blanket `==` was + // reading a coin. #1458. + // + // A dropped stage is NOT let through: it moves the distributions by O(1), so + // the arms stop agreeing about which candidates are tied at all. MEASURED — + // deleting the MLP residual add in `minimax_music3_depth_device.cpp` diverges + // at draw 1 with the DEVICE's pick 1.60e-01 below the host's own best, 41x the + // unit roundoff this admits, and `shared_tie` is RED. + // 2^-8 is bf16's UNIT ROUNDOFF -- the largest relative error a correctly + // rounded bf16 store can carry. It is NOT the format's relative spacing + // (machine epsilon), which is twice it at 2^-7, and it is not the `Bf16Ulp` + // helper this file uses for its value metric either. The unit roundoff is what + // bounds a stored probability's departure, which is what this comparison wants. + constexpr double kBf16UnitRoundoff = 1.0 / 256.0; + REQUIRE(host_rec.size() == device_rec.size()); + REQUIRE(host_codes.size() == device_codes.size()); + REQUIRE(host_rec.size() + 1 == host_codes.size()); + int64_t compared = 0; + bool diverged = false; + for (size_t i = 0; i < host_rec.size() && !diverged; ++i) { + ++compared; + const DrawRecord& h = host_rec[i]; + const DrawRecord& d = device_rec[i]; + if (h.code == d.code) continue; + // Each arm is about to be indexed with the OTHER arm's argmax, so the two + // heads must agree on the vocabulary width before that read happens. A + // defect that changed the device head's output width would otherwise be + // undefined behaviour here, and NDEBUG is the configuration this runs in. + REQUIRE(h.probs.size() == d.probs.size()); + REQUIRE(h.code >= 0); + REQUIRE(d.code >= 0); + REQUIRE(static_cast(h.code) < h.probs.size()); + REQUIRE(static_cast(d.code) < d.probs.size()); + // A divergence is admissible ONLY as a shared near-tie: each arm must rank + // the OTHER arm's pick within one bf16 unit roundoff of its own. That is not a + // tolerance anybody chose — it is the width the two arms store at, so a gap + // narrower than it is decided by rounding and not by the model. If either + // arm ranked the other's pick BELOW that, the two arms disagree about the + // distribution and not merely about a tie, and this fires. + const double h_gap = (h.best() - static_cast(h.probs[static_cast(d.code)])) / + std::max(h.best(), 1e-30); + const double d_gap = (d.best() - static_cast(d.probs[static_cast(h.code)])) / + std::max(d.best(), 1e-30); + const bool shared_tie = (h_gap <= kBf16UnitRoundoff) && (d_gap <= kBf16UnitRoundoff); + CHECK_MESSAGE(shared_tie, + "draw " << i << ": the two arms drew different codes (host " << h.code + << ", device " << d.code + << ") and it is NOT a shared tie — in the host's own distribution " + "the device's pick is " + << h_gap << " below its best, and in the device's the host's pick is " + << d_gap << " below its best, against one bf16 unit roundoff " << kBf16UnitRoundoff); + if (shared_tie) + MESSAGE("draw " << i << ": shared near-tie, host drew " << h.code << " (runner-up gap " + << ((h.best() - h.runner_up()) / h.best()) << "), device drew " << d.code + << " (runner-up gap " << ((d.best() - d.runner_up()) / d.best()) + << "), both inside one bf16 unit roundoff " << kBf16UnitRoundoff + << ". Every later draw is fed a different code, so the comparison stops " + "here."); + diverged = true; + } + CHECK_MESSAGE(compared > 0, "no draw was compared, so the code comparison gated nothing"); CHECK(device_draws == host_draws); CHECK(device_draws == config.residual_codebooks()); CHECK(static_cast(device_codes.size()) == config.num_codebooks); diff --git a/tests/vllm/models/test_muse_glimmer_text.cpp b/tests/vllm/models/test_muse_glimmer_text.cpp index 283312be0..33db867a2 100644 --- a/tests/vllm/models/test_muse_glimmer_text.cpp +++ b/tests/vllm/models/test_muse_glimmer_text.cpp @@ -527,6 +527,9 @@ TEST_CASE("muse_glimmer text: matches the fp32 reference transcribed from upstre double scale = 0.0; for (float x : want) scale = std::max(scale, std::abs(static_cast(x))); const double diff = MaxAbsDiff(got, want); + // This band is the same rounded-up W1 measurement class the biting case below + // no longer carries, and `4712dac40` moved it from 0.242 to 0.687 of its + // bound. Owed as #1466, with its own derivation; not re-tuned here. MESSAGE("muse_glimmer text vs fp32 reference: max|diff|=" << diff << " over logits of max|.|=" << scale); CHECK(diff <= 5e-4); @@ -534,12 +537,65 @@ TEST_CASE("muse_glimmer text: matches the fp32 reference transcribed from upstre // Same model with a BITING soft-cap, so the reference also pins the ORDER of the // output multiplier and the cap (:1618-1621). At the default cap of 20 the tanh is // linear over these logits and the order is unobservable; at 1e-3 it is not. + // + // WHERE THE ORDER IS VISIBLE, and it is not in a max|diff| band. At cap = 1e-3 + // the logits (max|.| = 4.88e-2) drive |x / cap| ~ 50, so the tanh is saturated + // to within 1e-40 and the two candidate orders reach DIFFERENT saturation + // magnitudes: `cap * tanh(mult * x / cap)` reaches `cap`, and the swapped + // `mult * cap * tanh(x / cap)` reaches `out_mult * cap`. On this fixture that + // is 1.0e-03 against 7.5e-04. So the order is an ALGEBRAIC property of the + // output's range, and gating it there costs no band to tune: MEASURED, our + // saturation and the reference's agree EXACTLY (delta 0), and the swap moves it + // by |1 - out_mult| * cap = 2.5e-04 — 32x the bound below. + // + // The bound: our logits pass a bf16 store, so the saturation we can report + // departs from `cap` by at most one bf16 unit roundoff (2^-8) on each side. + // + // WHAT THIS REPLACES, AND WHY. `bdiff <= 1e-5` was the W1 measurement (5.28e-06 + // at `3a54c4b7d`) rounded up, carrying 1.89x of headroom and no derivation. + // `4712dac40` narrowed `act(gate)` to the input dtype — upstream's polarity, and + // the only form that reproduces the committed `silu_and_mul_bf16_8x256` oracle + // golden bit-exactly — which adds one legitimate bf16 rounding per activation + // element and grew this envelope 2.8x, to 1.48e-05. A constant a CORRECT kernel + // change consumes was never a bound. #1458. + // + // Two replacements were tried and REJECTED, recorded so nobody re-derives them: + // `bdiff <= diff` is rigorous (the cap is 1-Lipschitz, so it cannot enlarge a + // difference) but does NOT red the swap, because the uncapped envelope is + // 3.4e-04 and the defect is 2.5e-04. A measured twin at `out_mult = 1`, where + // the two orders coincide by algebra, gives 6.10e-06 — but it is a DIFFERENT + // model, and its max|diff| sits at a different point on the tanh knee, so the + // gated run reaches 2.4x it with nothing wrong. The band is knee-driven and is + // kept only at its rigorous Lipschitz value. TinySpec biting; biting.softcap = 1e-3; const MuseGlimmerWeights wb = TinyWeights(MakeConfig(biting)); - const double bdiff = MaxAbsDiff(RunForward(wb, Positions()), RefForward(wb, Positions())); - MESSAGE("muse_glimmer text vs fp32 reference (biting soft-cap): max|diff|=" << bdiff); - CHECK(bdiff <= 1e-5); + const std::vector bgot = RunForward(wb, Positions()); + const std::vector bwant = RefForward(wb, Positions()); + const double bdiff = MaxAbsDiff(bgot, bwant); + double got_sat = 0.0, want_sat = 0.0; + for (float x : bgot) got_sat = std::max(got_sat, std::abs(static_cast(x))); + for (float x : bwant) want_sat = std::max(want_sat, std::abs(static_cast(x))); + // THE PRECONDITION, ASSERTED. Everything above turns on the tanh being + // saturated at this cap; if the fixture ever drifts so that it is not, the two + // orders stop having different ranges and this case degrades silently from an + // order gate into a weak magnitude comparison that the swap no longer reds. + // Saturated, the reference's own maximum IS the cap, and both sides are f32 + // values, so the check is an equality with nothing to tune. + REQUIRE(want_sat > 0.0); + REQUIRE(static_cast(want_sat) == static_cast(biting.softcap)); + // 2^-8 is bf16's UNIT ROUNDOFF -- the largest relative error a correctly + // rounded bf16 store can carry. It is NOT the format's relative spacing + // (machine epsilon), which is twice it at 2^-7. The unit roundoff is what a + // stored value's departure is bounded by, which is what both uses want. + constexpr double kBf16UnitRoundoff = 1.0 / 256.0; + MESSAGE("muse_glimmer text vs fp32 reference (biting soft-cap): max|diff|=" + << bdiff << " (uncapped envelope " << diff << "); saturation ours=" + << got_sat << " ref=" << want_sat << ", delta=" + << std::abs(got_sat - want_sat) << " against " + << (2.0 * kBf16UnitRoundoff * want_sat)); + CHECK(std::abs(got_sat - want_sat) <= 2.0 * kBf16UnitRoundoff * want_sat); + CHECK(bdiff <= diff); } TEST_CASE("muse_glimmer text: embed_norm is WEIGHTLESS RMSNorm, not a sqrt(H) scale") {