Qwen3.5/3.8 text-only arms: register Qwen3_5[Moe]ForCausalLM, resolve the backbone prefix, refuse unsupported MoE layouts (#490) - #601
Merged
Conversation
…and a prefix (#490) FOLLOWING_AGENTS_PROTOCOL Qwen/Qwen3.8-2.4T-A95B declares Qwen3_5MoeForCausalLM / qwen3_5_moe_text -- the same GDN-hybrid MoE backbone we run token-exact 315/315 on Qwen3.6-35B-A3B. head_dim 256, linear k/v head dim 128, full_attention_interval 4, partial_rotary_factor 0.25, rope_theta 1e7, mtp 1, and vocab 248320 all match; the differences are scale only (hidden 2048->8192, layers 40->92, experts 256->512, top-k 8->10), and every one of those is read from config. Two things block loading it. ModelRegistry::Resolve is exact-match with no aliasing (model_registry.cpp:217-231) and the text-only string is unregistered. And the loaders concatenate the literal VL prefix model.language_model. (qwen3_5_weights.cpp:560,632,633,659 plus three dense sites), while the 3.8 checkpoint publishes model.* -- verified against both published safetensors indices, which are otherwise identical name for name, including the 3D-stacked experts and the top-level lm_head. So the loader body is already correct; only where it looks has to change, resolved once per checkpoint rather than per lookup, mirroring upstream's single WeightsMapper (qwen3_5.py:296-300). Ahead of the pin, and said so plainly: the text-only arms arrived upstream in PR #50210 / ad5d29db7, which is post-555967922. This is a deliberate forward port of one upstream PR; it does not advance the pin. The run gate cannot be met here and the spec records it as OWED, not waived. 2.4T bf16 is ~4.8 TB, the only other variant is FP8 at ~2.4 TB, GB10 has 128 GB, and no smaller Qwen3.8 sibling exists. The row may claim that the architecture dispatches and the weight namespace resolves -- nothing about generated tokens. Spec only, no product code touched. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…tring and a prefix (#490) `Qwen/Qwen3.8-2.4T-A95B` declares `Qwen3_5MoeForCausalLM` / `qwen3_5_moe_text`. That is the SAME GDN-hybrid MoE backbone we already run token-exact 315/315 on Qwen3.6-35B-A3B, at larger scale — hidden 2048->8192, layers 40->92, experts 256->512, top-k 8->10 — and every one of those knobs is read from config. Two things stopped it loading, and neither was the model: 1. `ModelRegistry::Resolve` is exact string match with no aliasing, and we registered only the `ForConditionalGeneration` strings. 2. Both loaders concatenated the literal VL prefix `model.language_model.`, while a text-only checkpoint publishes the backbone flat under `model.`. Read from both published safetensors indices, the names are otherwise IDENTICAL: 3D-stacked `mlp.experts.gate_up_proj`, `mlp.shared_expert_gate.weight`, top-level `lm_head.weight`. Registration is additive: two `REGISTER_VLLM_MODEL` lines against the EXISTING dense and MoE factories. No factory, forward, KV-cache or quantization change. The text arms get their own `_ModelInfo` (hybrid YES, multimodal NO) because upstream's `Qwen3_5ForCausalLMBase` inherits IsHybrid but not SupportsMultiModal — the `ForConditionalGeneration` wrappers remain the multimodal registrations. The namespace is resolved ONCE per checkpoint from the shard index and then used everywhere, mirroring upstream's single `WeightsMapper(orig_to_new_prefix={"model.language_model.": "model."})`. A per-lookup fallback was deliberately rejected: it would let a checkpoint bind half its tensors from each namespace and still appear to load. A mixed index is REFUSED. Only the three structural backbone spellings vote, so `model.visual.*` on a vision-inclusive 27B checkpoint cannot make it look like a flat text one, and the per-layer public seams keep the VL prefix as their default, which is what makes 27B/35B/Coder inert by construction rather than by re-measurement. AHEAD OF THE PIN, DELIBERATELY. Our parity pin is `555967922`, whose registry carries only the `ForConditionalGeneration` entries. The text-only arms landed upstream afterwards in PR vllm#50210 at `ad5d29db7` (`registry.py:202-203`, `qwen3_5.py:296-300,439-449`). This is a forward port of ONE upstream PR. It does not advance the pin and reconciles nothing else in that range. That is visible debt argued here, not a silent divergence. THE RUN GATE IS OWED, AND NOTHING HERE CLAIMS A TOKEN. The 2.4T checkpoint cannot be executed on this hardware: bf16 is ~4.8 TB, the released FP8 variant ~2.4 TB, GB10 has 128 GB unified, and no smaller Qwen3.8 sibling exists. So there is no token-exact oracle run and no speed number for it, both rows land at `PARTIAL` rather than `DONE`, and the owed gate is recorded in the spec, the matrix rows and docs/STATUS.md. It closes only when a text-only `Qwen3_5[Moe]ForCausalLM` checkpoint that fits GB10 appears. MTP, quantized and GGUF arms for 3.8 are NOT implemented on speculation and are recorded as owed. RED first: both architecture strings raised "are not supported for now", and the dense loader threw `tensor not found: model.language_model.embed_tokens.weight` on a flat checkpoint. Green after: focused 5/5 / 124 assertions, full CPU gate 396/396 (1 skipped, `test_voxtral_e2e` fixture absent), `tests/parity/goldens` byte-identical. Eight mutations were applied and each killed its own claim: deleting either registration, widening the namespace probe to a blanket prefix match, removing the mixed-index or empty-index refusal, reverting the dense loader to the VL literal, letting the text arms claim multimodal support, and dropping `qwen3_5_moe_text` from the 0.25 partial-rotary family. The strongest test is not a name-mapping assertion: two synthetic one-layer checkpoints with byte-identical payloads and only the namespace differing load to byte-identical weights through the production `LoadQwen3_5Dense`. `scripts/check-agent-record.py`'s MODEL row count moves 362 -> 364 because two rows EXIST, never to make a transition pass. The pin-derived static invariants are untouched: like the Muse Glimmer and Kimi-K3 beyond-pin rows, these carry no pinned module/class target, so a post-pin arm cannot inflate a count that describes `555967922`. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…the debt was recorded nowhere (#490) Review FAIL on `0e1a8a90`. The code was correct; what it lacked was coverage and a record. Five findings, repaired in place. THE HIGH ONE: three MoE prefix sites survived reversion to the hardcoded `model.language_model.` literal with the FULL suite still green. Every DENSE equivalent was killed, so the asymmetry was real, not a harness artifact — and `Qwen3_5MoeForCausalLM` is the arm this row exists for. The new case mirrors the dense byte-equality proof through the production `LoadQwen3_5Moe`: two synthetic one-layer MoE checkpoints with byte-identical payloads and only the namespace differing, carrying the checkpoint's REAL per-class quantization (bf16 embeds/norms/router, per-tensor FP8 attention, NVFP4 routed experts + shared expert + lm_head), loaded on BOTH expert-residency paths. The second path is the point: with a `shards_owner` the routed experts are DEFERRED behind `load_layer_experts`, whose body captures the resolved prefix by value and runs after the resolving frame is gone, so the closure is explicitly DRIVEN rather than merely installed. Each of the three sites was then reverted in turn and each RED is the flat checkpoint failing to bind: LoadLayerImpl:560-561 tensor not found: model.language_model.layers.0.input_layernorm.weight LoadQwen3_5Moe:677-678 tensor not found: model.language_model.embed_tokens.weight deferred closure:707-708 tensor not found: model.language_model.layers.0.mlp.experts.0.gate_proj.weight The third fires ONLY in the deferred subcase (the eager one passes, 469 assertions vs 228), which is exactly the coverage that did not exist before. `src/` is byte-identical after every mutation. THE FIXTURE IS NOW THE PUBLISHED DOCUMENT, not a paraphrase of it. The hand-typed "REAL flat 3.8 shape" disagreed with `Qwen/Qwen3.8-2.4T-A95B/config.json` in four ways, and each hid a branch: it put `rope_theta` at the top level and omitted `rope_parameters`, so the parse took the no-block early return instead of the nested-block path the checkpoint actually takes; it omitted `layer_types`, which BOTH loaders hard-require to equal `num_hidden_layers`; it wrote `torch_dtype` where transformers 4.57.3 writes `dtype`; and it invented an `intermediate_size` the real config lacks. The document is now committed verbatim (md5 303dc59227f1d03afc941646e8df3132) and the case asserts the 92-entry `layer_types` list AND its `[linear,linear,linear,full] x 23` pattern. `dtype` IS DELIBERATELY NOT CONSUMED. `hf_config.cpp:520-522` reads only `torch_dtype`, so `HfConfig::torch_dtype` is EMPTY on this checkpoint. That is inert — nothing in the tree reads that field — and teaching hf_config the new spelling is a behavior change on EVERY model that owes its own row, RED-first test and inertness proof. Recorded as deviation 17(d) and pinned by an assertion so it cannot drift silently, rather than smuggled in on this row. THE DEBT IS NOW WRITTEN DOWN. This is an ahead-of-pin forward port — anchors at `ad5d29db7` / vllm#50210, post-pin against `555967922` — and the spec's own Risks section PROMISED that recording. `porting-inventory.md` §9 gains deviation 17 (same shape as 16, the Muse Glimmer off-pin anchor) covering the anchor, the owed run gate, the mixed-index REFUSAL where upstream's `WeightsMapper` normalizes, and the unconsumed `dtype`; §5's Qwen3.5 row and its intro now name both arms. `docs/BENCHMARKS.md` gains the Open-gaps line for the owed run gate. The refusal behavior is UNCHANGED: it is deliberate and strictly safer than upstream's rewrite, so it is tracked, not reconciled. The BENCHMARKS entry is 148 chars because the file is at its 45000-char budget; the detail lives in the spec and §9, which is where forensic detail belongs. That budget is a shared-file lock of exactly the shape `AGENTS.md` §Records warns about ("cap the entry, never the file"), and it is reported as such rather than worked around by weakening the checker. Gates: clean full default build (libvllm.so + examples + server) zero warnings under -Werror; SERIAL `ctest` 396/396 (1 skipped, `test_voxtral_e2e`, fixture absent); focused `test_qwen3_8_text_only` 6/6, 698 assertions, Status SUCCESS (was 5/5, 124); `tests/parity/goldens` md5 UNCHANGED at 5a7c2ac515b207473101e7f8b594d36a, so 27B/35B/Coder stay inert. NO token gate ran: this host has no GPU, and the 2.4T checkpoint could not be executed even with one. Nothing here claims a token. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…etween us and the checkpoint -- that was false (#490) Review FAIL on `0097a014`, on records honesty. The code is sound and the review said so: all three MoE prefix mutations go RED, the namespace probe is correct, goldens are untouched, no checker was weakened. What failed is what the row CLAIMED about that code. THE FALSE CLAIM, stated plainly. `e71d74ad`, `0e1a8a90` and `0097a014` asserted -- in the spec, in SHIPPED PRODUCT SOURCE (`qwen3_5_weights.h`), in `docs/STATUS.md`, in `.agents/model-matrix.md` and in all three commit bodies -- that the published Qwen3.8 and Qwen3.6-35B weight names are "identical modulo the prefix", that "the loader body is already correct and the only structural change is where it looks", and therefore that the only thing between this code and `Qwen/Qwen3.8-2.4T-A95B` is 128 GB of unified memory. It is not. `LoadQwen3_5Moe` routes EVERY routed expert through `LoadMoeExpertsInto` (`qwen3_5_weights.cpp:519-530`) into `LoadNvfp4Raw` (`:433-462`), which hard-requires per-expert `experts.<e>.<proj>.weight` = `U8`, `.weight_scale` = `F8_E4M3` and `.weight_scale_2`. There is no stacked branch and no bf16 branch, unlike `gemma4_weights.cpp:326`, which dispatches between layouts. Re-read from the live safetensors indices on 2026-08-12: Qwen/Qwen3.8-2.4T-A95B 1609 tensors; 93x model.layers.N.mlp.experts .gate_up_proj + 93x .down_proj (3-D STACKED); ZERO weight_scale, ZERO input_scale; lm_head.weight alone; 0 names under model.language_model. Qwen/Qwen3.6-35B-A3B 1045 tensors; the same stacked spelling under the VL prefix; ZERO weight_scale So the 2.4T load dies at `w.lm_head_fp4 = LoadNvfp4Raw(get, "lm_head")` (`:679`) with "expected U8 for lm_head.weight" BEFORE an expert is reached, and would die again at the FP8 attention, the routed experts and the shared expert. Our gated 35B row reads the REQUANTIZED `nvidia/Qwen3.6-35B-A3B-NVFP4`. This loader has never read a published Qwen bf16 MoE repo, in either namespace. WHAT IS ACTUALLY OWED, and the debt was recorded INVERTED. Every surface said "MTP, quantized and GGUF arms for 3.8 are NOT implemented and are recorded as owed". The QUANTIZED arm is the only one that IS implemented; what is missing is the bf16 / 3-D-STACKED MoE routed-expert arm (with the bf16 shared expert, the FP8-less attention tower and the bf16 lm_head on that path), and it was named nowhere. `porting-inventory.md` §9 gains deviation 17(e) for it; the spec gains a "What this row does NOT make loadable" section; STATUS, FEATURES, USAGE, BENCHMARKS, the two matrix rows and the roadmap issue row are corrected. THE CLOSING CONDITION WAS ALSO FALSE. "It closes only when a text-only Qwen3_5[Moe]ForCausalLM checkpoint that fits GB10 appears" holds for the DENSE arm and NOT for the MoE one: a fitting PUBLISHED (bf16/stacked) MoE checkpoint would still be refused at load. The MoE gate needs a fitting checkpoint whose routed experts are per-expert NVFP4, or the owed arm implemented first. Every surface that stated the old condition now states this one. THE DENSE/MoE ASYMMETRY IS THE RECORD, not a footnote. `LoadQwen3_5Dense` DOES route BF16 vs FP8 vs NVFP4 per projection by tensor presence (`qwen3_5_dense_weights.cpp:354-360,472-503`) and routes the head by dtype (`LoadDenseLmHead` / `LoadLmHeadAnyDtype`, `:215-233,515-547`), so the DENSE text-only arm may genuinely load a flat bf16 checkpoint. Only the MoE arm cannot. Any statement about "the text-only arms" that flattens that is wrong. ONE PRODUCT CHANGE, A REFUSAL AND ONLY A REFUSAL. AGENTS.md requires an unimplemented arm be "refused with a message naming the missing piece ... never left to be discovered later", and this row's own stop conditions said the same; instead a published checkpoint got a dtype complaint about a tensor indistinguishable from a corrupt file. `CheckMoeExpertLayoutSupported` runs once per checkpoint, right after the namespace decision and before any tensor is touched, and names the offending tensor plus the layout that would be required. NO stacked/bf16 loading capability is added: that needs its own spec, RED-first test and NVFP4 inertness proof. RED FIRST, literal, before the check existed: test_qwen3_8_text_only.cpp:949: ERROR: CHECK( Mentions(message, "model.layers.0.mlp.experts.gate_up_proj") ) is NOT correct! logged: message := vt: qwen3_5 weights: expected U8 for lm_head.weight at src/vllm/model_executor/models/qwen3_5_weights.cpp:435 [doctest] test cases: 7 | 6 passed | 1 failed | 0 skipped [doctest] assertions: 739 | 715 passed | 24 failed | [doctest] Status: FAILURE! GREEN after: 7/7, 739 assertions, Status SUCCESS. Six mutations, each restored byte-for-byte and each killing its own claim (Status: FAILURE! in all six): M1 call site neutered 24 failed (the RED above, reproduced) M2 stacked branch deleted 2 failed (both stacked subcases) M3 unquantized-expert branch deleted 1 failed M4 unquantized-lm_head branch deleted 6 failed M5 refuse UNCONDITIONALLY 6 failed -- the SUPPORTED-layout and byte-equality subcases go red, which is the NVFP4 inertness proof M6 BOTH seam defaults VL->flat 4 failed -- the reviewer's exact F7 mutation, which the previous inertness case survived entirely F7 is why M6 now bites: the case asserted two named constants and nothing else, so flipping the header DEFAULT ARGUMENTS left it green. It now DRIVES `LoadQwen3_5MoeLayer` and `LoadQwen3_5DenseLayer` with the prefix argument OMITTED and requires the flat checkpoint to fail by the VL name. Also repaired: the test misdescribed its own fixture (F6) -- it called the per-expert NVFP4 layout "the real 35B/3.8 scheme" and presented `...experts.gate_up_proj` as "35B-shaped", a name the loader can never resolve; `docs/USAGE.md` scoped its caveat to whether a checkpoint had been RUN rather than whether it LOADS (F4); and `docs/STATUS.md` carried a stale "(5/5, 124 assertions)" (F5) -- REMOVED rather than re-pinned, per AGENTS.md "never store a measurement of one file inside another". History is NOT rewritten. The three commits below this one keep their wrong claims, and this commit is the correction; `git log --grep 490` shows both. The BENCHMARKS entry is terse because that file is at its 45000-char budget and paying for detail there means evicting someone else's row -- the shared-file lock AGENTS.md §Records warns about, reported rather than worked around. The detail lives in the spec and §9 deviation 17(e). Gates: clean out-of-tree default build (Release, CUDA=OFF, tests + examples + server) with ZERO warnings under -Werror; SERIAL ctest; focused `test_qwen3_8_text_only` 7/7, 739 assertions, Status SUCCESS; `scripts/agent-preflight.sh --staged` green; `tests/parity/goldens` md5 UNCHANGED at 5a7c2ac515b207473101e7f8b594d36a, so 27B/35B/Coder stay inert. NO token gate ran: this host has no GPU, and the 2.4T checkpoint could not be executed even with one. Nothing here claims a token. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…lling this refusal rejects (#490) Review finding 1 on `66a155d0`, closed by pinning behavior that nothing in-tree pinned. `CheckMoeExpertLayoutSupported` refuses 3-D stacked routed experts. The real gated index -- `nvidia/Qwen3.6-35B-A3B-NVFP4/model.safetensors.index.json`, fetched and read directly, 124,468 tensors -- DOES contain that exact spelling, as `mtp.layers.0.mlp.experts.gate_up_proj` and `.down_proj`, and only there. It loads anyway because the scan is anchored at `<backbone>layers.` (qwen3_5_weights.cpp:633,638) and `mtp.` is under neither backbone spelling. Everything else in that index is on the supported arm: ZERO stacked names under `model.language_model.layers.`, ZERO expert `.weight` without a `_scale` sibling, and `lm_head.weight_scale` + `weight_scale_2` both present. So the `mtp.` exclusion is LOAD-BEARING, and it was pinned by NOTHING. A later edit broadening the scan to every `.mlp.experts.` name would refuse the one checkpoint this arm is gated on -- on a CUDA-only load path, with the entire CPU suite green. That is the failure mode this commit makes CPU-visible. `MoeOneLayerSpecs` -- the SUPPORTED fixture -- now carries those two `mtp.` names, so the inertness assertions run against the real index's shape rather than a paraphrase of it. Case 4c gains a subcase that re-asserts the fixture still carries them (count == 1 each, and count == 0 for the same spelling under either backbone prefix, so a fixture that quietly lost them cannot leave the loads below proving nothing) and that both namespaces still load clean. RED FIRST, by dropping the `<backbone>layers.` filter so the scan sees every `.mlp.experts.` name: test_qwen3_8_text_only.cpp:1049: ERROR: CHECK( load(MoeOneLayerSpecs("model."), "mtp_stacked_flat").empty() ) is NOT correct! values: CHECK( false ) ... vt: qwen3_5 weights: 3-D stacked routed experts are not implemented for the safetensors MoE arm -- found "mtp.layers.0.mlp.experts.gate_up_proj". [doctest] test cases: 7 | 5 passed | 2 failed | 0 skipped [doctest] assertions: 277 | 272 passed | 5 failed | [doctest] Status: FAILURE! Note the assertion count FALLS 747 -> 277 there: the thrown cases abort, so the count alone reads like a smaller suite, not a failing one. `src/` is byte-identical after restoration (md5 16e44ce4e68e3699d52ed2c370a9d074) and the test binary returns to the pre-mutation md5, 7/7 cases, 747/747 assertions, Status SUCCESS (was 7/7, 739). NO BEHAVIOR CHANGED. The refusal is byte-identical; this commit is coverage and record only. The spec's `## Outcome` gains the index evidence, the load-bearing `mtp.` exclusion, and -- as a clarification, since no surface claims otherwise -- the covered set of the "refused by name" guarantee: routed experts and `lm_head` only. A bf16 shared expert, a bf16 attention tower, the compressed-tensors `weight_packed` spelling and a tied-head MoE checkpoint still surface raw loader errors, and widening the refusal belongs with the owed stacked/bf16 arm, which has to read those layouts anyway. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
… on its own (#490) Merged `origin/main` at `8b00f79f2` (87 commits ahead; merge-base `e1087a88`). Main did not touch any file this row owns -- `qwen3_5_weights.{h,cpp}`, `qwen3_5_dense{,_weights}.cpp`, `qwen3_5_moe.cpp`, `qwen3_5_common.h`, `test_qwen3_8_text_only.cpp`, `test_model_registry.cpp` are all untouched by it; its qwen3_5 work is in `qwen3_5.cpp` / `qwen3_5_internal.h` (the GDN packed bridge). No semantic conflict. SIX KEYED RECORDS WERE RE-RESOLVED BY HAND, not accepted from the three-way merge, per AGENTS.md: `.agents/roadmap_v1.md`, `.agents/model-matrix.md`, `docs/{BENCHMARKS,FEATURES,STATUS,USAGE}.md` were each reset to the `origin/main` version WHOLESALE and this row's scoped edit re-applied on top -- five by strict-context `git apply` of the merge-base->row diff (no fuzz, no 3-way fallback), and `.agents/roadmap_v1.md` by a scripted insert that asserts its anchor is unique (`| [#412](...)` count == 1) and that its own key is absent (`/issues/490)` count == 0) before writing. The result is byte-identical to the auto-merge, which is the evidence that the auto-merge was in fact correct here -- it is not a reason to have skipped the step. Each resolved file diffs against `origin/main` at exactly its own scoped size (1/0, 11/5, 1/0, 1/0, 46/0, 20/0), so every unrelated key is byte-for-byte main's. `docs/BENCHMARKS.md` needed no room: this row's Open-gaps line was already there and nobody was evicted. The two files both sides touched that are NOT keyed records -- `scripts/check-agent-record.py` (ours: MODEL count 362->364; main's: the code-span link stripper) and `tests/CMakeLists.txt` -- merged in disjoint regions. The 364 still holds: main's only model-matrix edit rewrote the nemotron-h row IN PLACE and added none. `origin/main` IS RED AND THE BREAKAGE IS NOT OURS. `af8170154` added `parity::Nemotron35LightningSnapshot()` at `tests/parity/hf_snapshot.h:51`, which calls `HfSnapshot` declared 11 lines BELOW it: hf_snapshot.h:52:10: error: 'HfSnapshot' was not declared in this scope Reproduced on the pristine `origin/main` blob in isolation -- `git show origin/main:tests/parity/hf_snapshot.h` plus a two-line TU, `g++ -std=c++20 -fsyntax-only`, no vllm.cpp include path -- so it is the header, not the merge. It takes out 14 TUs, every one of them a checkpoint-gated parity/GPU suite, with the identical single error. Per the task contract main's defects are reported, not repaired here; this row owns none of those files. GATE ON THE MERGED TREE, clean out-of-tree Release build on local disk (`ninja -k 0`, `-Werror`, 1207 targets): ZERO warnings, 1193 objects built, the 14 above blocked by main. SERIAL `ctest` (no `-j`): **97% tests passed, 14 tests failed out of 401**, total 528.73 s -- and all 14 are `***Not Run`, i.e. the missing executables from those same TUs. ZERO tests actually failed, under a box load average of ~150 from concurrent sessions; `test_voxtral_e2e` skipped (fixture absent), as before. `scripts/agent-preflight.sh --staged` green including `test_cpu_x86_llamacpp_floor`, which needed a re-run at lower load -- its first run reported `NO_QUIET_WINDOW after 15s (busy=125%)`, which is the foreign load, not a defect. THE ROW SURVIVED INTACT. Both arch strings still registered (`qwen3_5_dense.cpp:256`, `qwen3_5_moe.cpp:231`); `ResolveQwen3_5BackbonePrefix` still resolves both namespaces and still refuses a mixed and an empty index (`qwen3_5_weights.cpp:677-695`); `CheckMoeExpertLayoutSupported` still carries all three refusal branches (`:648`, `:656`, `:668`); `test_qwen3_8_text_only` 7/7 cases, 747/747 assertions, `Status: SUCCESS!` -- the same counts as before the merge. `tests/parity/goldens` is byte-identical to `origin/main` (this branch changes none of it; main added the new nemotron oracle). FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…irs the tree (#490) Merged `origin/main` at `7572b0f4e2fb03548f8672c1cc30e8a60d8befab` (13 commits; merge-base `8b00f79f2`, the transiently-RED main this row merged last time). THE BREAKAGE IS GONE, AND MERGING IS WHAT FIXED IT. `446edde5` merged main during the window where `tests/parity/hf_snapshot.h` called `HfSnapshot` 11 lines before its declaration, so every TU including that header failed with `'HfSnapshot' was not declared in this scope` and 14 parity/GPU suites did not link. Main repaired it in `fafa16f0f` and then reworked the header for content-pinning in `751325460`/`7572b0f4e`. This row owns nothing in that file: `tests/parity/hf_snapshot.h` and `tests/scripts/test_check_snapshot_pins.py` were taken from main WHOLESALE, and `git diff origin/main` on each is EMPTY. ONE CONFLICT, in `tests/vllm/models/test_model_registry.cpp`, and it is purely additive on both sides: main registered `NemotronHForCausalLM` (37 -> 38), this row registered `Qwen3_5ForCausalLM` + `Qwen3_5MoeForCausalLM` (37 -> 39). All four hunks resolved to the UNION at 40: the three pinned counts (`registrations.size()`, `supported.size()`, `kExampleConfigArchitectures`) and the `registry_model_property` branch, where main's Nemotron arm and this row's Qwen3.5 text arm are now two separate `else if` branches, each keeping its own upstream-anchored comment and its own `is_hybrid` / `!supports_multimodal` assertions. The sorted ledgers auto-merged in disjoint regions and were verified afterwards: `kSortedArchs` and `kExampleConfigArchitectures` each hold 40 entries and are still byte-sorted, and both `raise_for_unsupported` strings carry `'NemotronHForCausalLM'` and both `'Qwen3_5*ForCausalLM'` spellings. NO KEYED RECORD NEEDED HAND RE-RESOLUTION, and that was CHECKED rather than assumed. Main touched none of `.agents/model-matrix.md`, `.agents/porting-inventory.md` or `scripts/check-agent-record.py` in `8b00f79f2..origin/main` (`git diff --numstat` empty on each), so this row's version IS the merge result and no unrelated key could have moved; the MODEL count stays 364 because main's Nemotron-H work rewrote an existing matrix row rather than adding one. `.agents/roadmap_v1.md` and `docs/{BENCHMARKS,FEATURES, STATUS,USAGE}.md` were each verified the other way, by diffing the merged tree against `origin/main`: every one of them differs by exactly this row's own scoped addition and NOTHING else -- 1 line in roadmap (issue #490), 1 in BENCHMARKS, 1 in FEATURES, 46 in STATUS, 20 in USAGE, zero deletions anywhere, so no other row's entry was evicted and `docs/BENCHMARKS.md` needed no room. ONE RECORD ANCHOR WAS CORRECTED BECAUSE THE MERGE INVALIDATED IT. The `MODEL-TEXT-qwen3-5-qwen3-5-for-causal-lm` matrix row said the registry contract is re-pinned at `test_model_registry.cpp:150` (39 archs); the line is still 150 and the count is now 40. Updated by a script that asserts its anchor occurs exactly once. That is the only edit in this commit that is not a merge resolution, and it changes no behavior. GATE ON THE MERGED TREE. Clean out-of-tree Release build on local disk (`cmake -G Ninja -DVLLM_CPP_CUDA=OFF`, `-Werror`): exit 0, 1215/1215 targets, ZERO warnings and ZERO errors -- the tree compiles again. SERIAL `ctest` (no `-j`): **100% tests passed, 0 tests failed out of 404**, total 156.98 s. Two did not run and NEITHER is counted as coverage: `test_modelopt_mixed_precision_ checkpoint` (Skipped, main's new checkpoint-gated suite) and `test_voxtral_e2e` (Skipped, fixture absent). `scripts/agent-preflight.sh --staged` reports `All gates green.`, including `test_cpu_x86_llamacpp_floor`, which had failed a pre-merge baseline run purely on box load (loadavg 37 at the time, its own message is a contention discard). THE ROW SURVIVED INTACT. Both arch strings still registered (`qwen3_5_dense.cpp:256`, `qwen3_5_moe.cpp:231`); `ResolveQwen3_5BackbonePrefix` still resolves both namespaces and still refuses a mixed and an empty index (`qwen3_5_weights.cpp:677-695`); `CheckMoeExpertLayoutSupported` still carries all three refusal branches (stacked experts, unquantized experts, unquantized lm_head); the `mtp.`-exclusion is still pinned by the backbone-prefix guard and its two dedicated subcases. `test_qwen3_8_text_only` is 7/7 cases, 747/747 assertions, `Status: SUCCESS!` -- byte-identical source and dependencies to the reviewed head `1cc951cd`, and the same counts that head recorded. `test_model_registry` 24/24, 924 assertions, `Status: SUCCESS!`. `tests/parity/goldens` is byte-identical to `origin/main`, including the `nemotron_35_lightning_greedy/oracle.json` this branch inherited from main -- it is TRACKED here and on main, not a stray extraction. `test_qwen35_plain_weights` reports 4 assertions (`Qwen/Qwen3.5-4B` not cached) and is recorded as a SKIP, never as coverage. NO token gate ran: no GPU and no checkpoints on this box, so the SACRED 27B/35B/Coder gates are OWED to the DGX and are not claimed here. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
`pr-size` refused this row: a change to `scripts/check-agent-record.py` owes
executable mutation evidence in `tests/scripts/test_agent_record.py`, and the
row changed the checker without touching its suite. Two reviewers read the
362 -> 364 bump as "not semantics" and therefore exempt. The gate is the
authority on what it enforces, so the answer is evidence, not an exemption --
and the evidence turns out to be worth having on its own.
`test_model_row_ratchet_is_load_bearing` proves the pin BINDS by moving it one
either way. That holds for ANY value of the pin, so it cannot say whether THIS
value is the right one -- exactly the failure mode its own docstring names
("bumping the number to silence a failure is indistinguishable from bumping it
because a row really landed"). The new class closes that: the two new rows must
each appear exactly once in `.agents/model-matrix.md`, and the pin must equal
the MODEL rows that file carries, counted the way `check_matrices` counts them.
Shaped after `TenstorrentResidualGoldenRowIsCounted`, which ties the BACKEND
bump to its row the same way; no new convention.
RED before, both directions:
- pin reverted to its BASE value 362 against this tree ->
`AssertionError: 364 != 362 : the MODEL pin must equal the MODEL rows
model-matrix.md carries` (this is precisely what check-pr-size.py executes
when it swaps the BASE checker into a HEAD worktree)
- one of the two new matrix rows deleted -> `0 != 1 :
MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm must appear exactly once`
AND `363 != 364`
Both mutations restored byte-for-byte (`cmp` clean); 51/51 green after.
No checker was weakened: `check-pr-size.py` and `check-agent-record.py` are
untouched by this commit.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
pushed a commit
that referenced
this pull request
Aug 13, 2026
…CausalLM FOLLOWING_AGENTS_PROTOCOL The sweep reported 8 unrowed architectures. One of them, Qwen3_5MoeForCausalLM, was already owned by #490 and its open PR #601 -- which does not merely row it but REGISTERS it alongside Qwen3_5ForCausalLM, resolves the backbone prefix, refuses unsupported MoE layouts, and carries CPU 404/404 plus SACRED GPU 315/315, 235/235 and 138/138, reviewed twice over five reproduced mutations. #601 also adds a row this sweep never saw, MODEL-TEXT-qwen3-5-qwen3-5-for-causal-lm. So 7 are owed: BailingMoeV3ForCausalLM plus the six vllm-omni architectures. AGENTS.md says to re-verify a gap against issues AND PRs before claiming. I grepped issue titles with terms that did not match #490's wording and did not check open PRs at all. The verification that should have run first was cheap: diff every open PR against main for each architecture string. Run afterwards, it shows Qwen3_5MoeForCausalLM in two branches and the other seven in one -- which is exactly the answer needed before dispatching anyone. This matters beyond one wasted row. Two branches ADDING the same keyed row merge without a conflict and define it twice; nothing announces it, because a clean merge is the symptom. Same shape as the #609/#610 duplication removed two commits ago, from the same root cause: assuming a keyed record has one writer. Both counting lessons are recorded in the sweep section rather than just fixed, since the next sweep will face both. python3 scripts/check-agent-record.py: 0 errors. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This was referenced Aug 13, 2026
localai-bot
pushed a commit
that referenced
this pull request
Aug 13, 2026
FOLLOWING_AGENTS_PROTOCOL `test_model_row_ratchet_is_load_bearing`'s docstring still read "the eight recipe architectures ... took it 362 -> 370". Both numbers are from the pre-correction revision of this branch: seven rows landed, the constant is 369, and the new test's own docstring one screen below already said 362 -> 369. So the file stated two different bumps for the same change. That is worse than a typo because of where it sits. The docstring IS the prose explanation of why the ratchet exists, and AGENTS.md is explicit that keeping two descriptions in sync is the failure mode this protocol was built to remove -- a ratchet whose explanation contradicts its own constant is the exact drift it is there to catch. The next person to re-pin the count reads the docstring, not the constant. Verified by grep rather than by eye. Across every line this branch ADDS, the only remaining occurrence of 370, 322, 337, 292 or "eight" is the deliberate "SEVEN, not eight: the audit's eighth architecture" sentence in the checker comment, which is load-bearing -- it records why the audit's eighth architecture is left to #490 / PR #601. The corrected numbers appear consistently: 369 x5, 336 x2, 321, 291, seven x6. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
) The `sanitize-cpu (address,undefined)` leg went red on this row's new `test_qwen3_8_text_only`: src/vllm/model_executor/models/qwen3_5_weights.cpp:298:36: runtime error: load of misaligned address 0x7f00df8027db for type 'const uint16_t', which requires 2 byte alignment A safetensors tensor's offset is the running byte total of everything ahead of it, so nothing makes it even. `TransposeBf16` took a `const uint16_t*` and one of its callers built that pointer with `reinterpret_cast<const uint16_t*>( t.data)` straight off the mmap. Forming and loading through it is undefined regardless of x86 tolerating it, and it faults on stricter targets. PRE-EXISTING, not this row's: `git log -L 296,300` attributes the function to `8ee2c0766`. What this row supplied is the first synthetic checkpoint whose tensor offsets are not all even -- a legitimate shape a real file can have -- and the first time a safetensors loader ran under UBSan at all. The observed site was one instance of a class. Sweeping `reinterpret_cast<T*>(<sttensor>.data)` across `src/vllm/model_executor/models/` found FIFTEEN in nine `*_weights.cpp` loaders, two of them stricter than 2-byte: `olmo2_weights.cpp` forms a `const float*` and `qwen3_dspark_weights.cpp` a `const int64_t*`. UBSan only fires where a test happens to reach, so fixing only the one it caught would have left the class open. All fifteen now go through `vt::LoadUnaligned` -- the seam `ea4deb203` introduced, which `dense_loaders::TransposeBf16` and `minimax_h3_vae_loader.cpp:87` already use. No new helper: the local `qwen3_5_weights.cpp` copy of `TransposeBf16` had simply never been migrated to the shared one. Two sites could not take a byte pointer and were handled in kind -- `internlm2_weights.cpp` only bulk-`memcpy`s from its source so it keeps a `const uint8_t*` and scales its offsets, and `gemma4_weights.cpp` feeds a typed scale pointer to `DequantFp8ChannelToBf16` (whose header is outside this row's authority) so it copies the N-element scale row into an aligned buffer. INERTNESS, proven three ways rather than asserted. `vt::LoadUnaligned<T>` is `memcpy`, bit-identical to `*(const T*)p` on every input the old code was ALLOWED to read: a scratch harness ran the original and rewritten form of all six loops over one payload at an aligned base and got byte-identical output, then reproduced those same bytes from the rewritten form at misaligned bases 1..8. `tests/parity/goldens` is untouched and every golden-comparing suite passes. Full CPU gate 404/404 and full ASan+UBSan 404/404 (2 skipped both times: `test_voxtral_e2e`, `test_modelopt_mixed_precision_checkpoint`), `test_qwen3_8_text_only` 7/7 and 747/747 with zero runtime errors. NO LOAD-TIME REGRESSION, checked not assumed. `TransposeBf16` is a hot load-time loop and a per-element `memcpy` is exactly the change that can turn one load into a call. At -O2 it does not: the old and new inner loops are instruction-for-instruction identical -- same six instructions, same `movzx REG, WORD PTR [rax]`, no call emitted. The whole delta is five prologue instructions paid once per call. NEGATIVE MUTATION: reverting `TransposeBf16` to the typed-pointer form and rebuilding reproduces the identical UBSan report at the same call chain (`TransposeBf16` <- `LoadBf16Transposed` <- `LoadMoe` <- `LoadLayerImpl`), exit 1. Restored byte-for-byte (md5 `13e1ea8db85dbae9ea2e17aa61589448` before and after) and back to 7/7, 747/747. STILL OWED, outside this row's authority (`*_weights.cpp`): the same cast survives at `voxtral.cpp:51,347`, `qwen3_vl.cpp:78` and `qwen3_5_mtp.cpp:71`. The first three are genuine misaligned loads; the mtp one only forms the pointer before `memcpy`ing, so it is UB that will not trip the alignment check. None is reached by a sanitized suite today. DOCS: `check-doc-checkpoint.py` classifies any edit under `src/vllm/model_executor/models/` as a feature surface and requires `docs/FEATURES.md`, so the "Safetensors direct load" row now states the guarantee this commit actually establishes -- payloads are read at any byte offset without forming a typed pointer -- and names the three files where that is still owed, so the doc does not overclaim. No checker was weakened, no assertion deleted, no sanitizer suppressed; `check-doc-checkpoint.py` is untouched. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
added a commit
that referenced
this pull request
Aug 13, 2026
, #610) (#625) FOLLOWING_AGENTS_PROTOCOL Seven architectures behind official vLLM recipes had no row in model-matrix.md, which presents itself as the exhaustive architecture inventory. One is pin-lag -- BailingMoeV3ForCausalLM is registered on vLLM main and absent only at 555967922. Six live in the separate vllm-omni repository, mirroring the precedent the MiniMaxH3DiTModel row already set: MossTTSDelayModel, MossTTSRealtime, Qwen3TTSForConditionalGeneration and HiggsMultimodalQwen3ForConditionalGeneration are registered there, while VoxtralRealtimeForConditionalGeneration and BailingMMNativeForConditionalGeneration are target-pending, recording what was searched rather than an invented anchor. SEVEN, not the eight or nine the issues first stated. The unit of a row is the ARCHITECTURE, not the recipe: 11 unrowed recipes resolve to 8 architectures because MossTTSDelayModel alone is reached by four of them. And the eighth, Qwen3_5MoeForCausalLM, is owned by #490 / PR #601, which REGISTERS it rather than only rowing it -- the class already exists at our pin and is the text tower the multimodal class builds, so main added only a registry entry. Rowing it here would have collided on an identical stable ID, which merges without conflict and defines the row twice. Counts re-derived with the checker's own parser: 369 rows / 321 INVENTORIED. The at-the-pin model inventory is unchanged, because these rows carry no pinned-registry target -- the same convention the MuseGlimmer, KimiK3 and MiniMaxH3DiT rows already follow. No checker expectation was widened. The audit could not read 20 of the 157 recipe configs (gated or 404), so this is a floor rather than a total; recorded as #626. Records only, no product code touched. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ords, resolved by key (#490) Merges `cefacd2d0` (`origin/main`, re-pinned mid-session: `a20cdac30` was the tip when this started and `cefacd2d0` landed while the first resolution was in progress, so that merge was aborted and redone against the newer SHA rather than committed against a base that was already stale). Four conflicts, all on surfaces every row writes. KEYED RECORDS, resolved per AGENTS.md by taking the target-branch file WHOLESALE and re-applying only this row's scoped edit on top -- never by accepting the three-way result, even where it looked right: - `.agents/model-matrix.md`: `origin/main`'s file, plus this row's two rows (`MODEL-TEXT-qwen3-5-qwen3-5-for-causal-lm`, `MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm`) and their two checklist entries, both byte-identical to the row's own version. Verified afterwards: every one of `origin/main`'s MODEL keys is byte-identical, none removed, only those two added. - `.agents/roadmap_v1.md`: `origin/main`'s file plus the single #490 issue row. #489, #609 and #610 all survive. - `scripts/check-agent-record.py`: `origin/main`'s MODEL pin and both of its justifications kept intact; this row's bump re-written 370 -> 372 rather than the 362 -> 364 it was authored against. RECOUNTED, not assumed: the #609/#610 backfill (+7) and LTX-2.5 (+1) landed while this row was in review, and the checker now reports MODEL=372 against the tree. A justification framed against a number this file no longer carries would be false about the file it sits in, so the comment says so explicitly. Neither checker was weakened; both of `origin/main`'s new cases and this row's `Qwen35TextOnlyRowsAreCounted` run. - `tests/CMakeLists.txt`: pure union. `test_qwen3_8_text_only` + its verbatim config fixture AND `origin/main`'s `test_device_pool`; the `_glue_fuse_off` / `_fused_chain_off` lever entries from #489 are present and untouched. The six shared surfaces that auto-merged -- `docs/{BENCHMARKS,FEATURES,STATUS, USAGE}.md`, `.agents/porting-inventory.md`, `tests/scripts/test_agent_record.py` -- were each PROVED rather than trusted: the merged-vs-`origin/main` change lines are byte-identical to this row's own change lines against the merge base. Nobody was evicted, and `docs/BENCHMARKS.md` took its one line without displacing one. Two counters in `.agents/model-matrix.md` are corrected past this row's own +2 because `cefacd2d0` left them behind: the not-at-the-pin enumeration gains `LTX2VideoTransformer3DModel` (10 -> 13 with this row's two, 336 -> 339), and "Engaged architectures (the N non-`INVENTORIED` rows)" goes 48 -> 51 where 48+2 would have been knowingly wrong -- the rollup says 372 total against 321 `INVENTORIED`. This is arithmetic on lines the merge already had to touch, not an advance of anyone's row. VERIFIED on the merged tree, CPU only. Clean out-of-tree Ninja configure and `-Werror` build, ZERO warnings, 1261/1261. `ctest` SERIAL: 100% tests passed, 0 failed out of 424 (2 skipped: `test_modelopt_mixed_precision_checkpoint`, `test_voxtral_e2e`), 139.04 s. `scripts/agent-preflight.sh --staged` exit 0 with every record gate and mutation suite ok; `check-agent-record.py` exit 0 reporting `MODEL=372`; `check-model-checklist.py` exit 0; `test_agent_record.py` 54/54. The row survived intact. `test_qwen3_8_text_only` 7 cases / 747 assertions / Status SUCCESS -- the exact count its spec records, not a smaller one. `test_model_registry` 24/24, 924 assertions, still pinned at 40 architectures. Both `Qwen3_5ForCausalLM` and `Qwen3_5MoeForCausalLM` registered, `ResolveQwen3_5BackbonePrefix` resolving both namespaces and refusing mixed and empty, `CheckMoeExpertLayoutSupported` carrying all three refusal branches, the `mtp.` exclusion pinned, and no golden byte changed. ASan+UBSan re-run for #627: the alignment fix holds. `qwen3_5_weights.cpp` still reads through the SHARED `vt::LoadUnaligned` (`include/vt/unaligned.h`), not a private copy, and the merge reintroduced no typed-pointer cast into any loader it touched -- `origin/main` never edited those files. Under `address,undefined` with `print_stacktrace=1`, `detect_leaks=1` and `VT_POOL_BYPASS=1`: `test_qwen3_8_text_only` 747/747, `test_model_registry` 924/924, `test_parakeet_encoder` 543/543, `test_laguna_nvfp4_loader` 63/63, `test_qwen3_5_gdn_spec_routing` 52/52, `test_qwen36_weights` 45/45, `test_qwen3_dspark_weights` 33/33, `test_qwen3_5_lm_head_dtypes` 32/32, `test_qwen35_paged_forward` 8/8, `test_gemma4_honesty` 6/6, `test_qwen35_plain_weights` 4/4, `test_model_loader_gguf` 3/3 -- zero sanitizer diagnostics. Stated because it matters: `test_qwen3_load`, `test_qwen3_5_gguf_mtp`, `test_{olmo2,internlm2,phi}_paged_engine` reported 0 assertions here, which is a SKIP for want of checkpoints, not coverage. NOT re-run: every SACRED GPU gate. There is no GPU on this host. They were last run at `1cc951cd` and are OWED against this merged tree. Issue: #490 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Collaborator
Author
|
Housekeeping note from another session, no action needed on the diff. The shared box reached 100% disk / 3.6G free with six agents compiling, at which point builds start failing with errors that read like source defects. I reclaimed this branch's stale build directory to clear it. What was deleted: only the build directory (regenerable artifacts). Nothing tracked, nothing uncommitted, no branch, no worktree. The checks I ran immediately before deleting, all four of which had to pass: no process anywhere under the worktree ( Sorry for the recompile. If your session was mid-anything, that is on me and I would rather hear it than not. |
…s RE-DERIVED, not carried (#490) `main` advanced 16 commits past `cefacd2d0` while this row sat in review, and four of those commits move the one number this row also moves. Merged `a3aa02e19`, resolved by key, and re-derived every count off the tree that exists after the merge. The two content conflicts, and the four keyed records that had to be redone by hand whether git flagged them or not: - `.agents/model-matrix.md` (CONFLICT). `origin/main`'s file taken WHOLESALE, then this row's four scoped edits re-applied against unique anchors: the two keyed `MODEL-TEXT-qwen3-5-*-causal-lm` rows at the end of the MODEL-TEXT section, their two engaged-checklist entries after the `Qwen3_5MoeForConditionalGeneration` sibling, the beyond-pin clause naming both arms, and the counts those two rows move -- not-at-pin 14 -> 16, architecture rows 369 -> 371, rollup PARTIAL 20 -> 22 and Total 373 -> 375, engaged 50 -> 52. Every other line of `origin/main` is byte-identical, asserted line-by-line rather than eyeballed. - `scripts/check-agent-record.py` (CONFLICT). `main`'s comment history kept whole; this row's justification appended and re-dated. The pin is 375, and 375 is what the checker's OWN parser counts off the merged matrix -- it was not carried forward. This row was authored against 362 -> 364, re-derived once to 370 -> 372, and is re-derived here to 373 -> 375: the #609/#610 backfill, LTX-2.5 (#435), IndexTTS-2.5 (#634, two architectures) and MiniMax-Music3 (#672) all landed in the interval and all moved this pin. A number carried across a merge is a number about a file that no longer exists. - `.agents/roadmap_v1.md` issue table. Auto-merged CLEANLY, and redone anyway: keyed records never take a three-way merge, because the clean result and the wrong result look identical here. `origin/main`'s table taken wholesale, the single #490 entry re-inserted after its unique anchor, 0 evictions. - `tests/scripts/test_agent_record.py`. Union: `main`'s three new collision assertions (`test_music3_and_indextts_rows_both_survive_their_collision`, `test_indextts_rows_are_inside_the_model_ratchet`, `test_omni_pin_row_is_inside_the_engine_ratchet`) and this row's `Qwen35TextOnlyRowsAreCounted` all survive. Its docstring and the ratchet history now say 373 -> 375, because a docstring naming a bump this file no longer makes is evidence for nothing. `docs/{STATUS,FEATURES,USAGE,BENCHMARKS}.md`, `.agents/porting-inventory.md` and `tests/CMakeLists.txt` are pure unions -- every entry `main` landed in the interval survives, and this row evicts nobody. No row behavior changed: the diff against `origin/main` is the same 29 files the branch carried before the merge, goldens included (untouched). The UBSan fix of `2608b8138` still routes `qwen3_5_weights.cpp`'s bf16 transpose through `vt::LoadUnaligned`, with no typed-pointer cast reintroduced. Gate, on the merge result, CPU-only host: - clean out-of-tree `-Werror` build, exit 0, 0 warnings - `ctest` SERIAL, see below - `check-agent-record.py`: `MODEL=375`; `check-model-checklist.py` OK - `test_agent_record.py` 57/57 - `check-pr-size.py --base a3aa02e --head HEAD` exit 0 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…rged rather than gated against a stale base (#490) `main` landed two more commits while the previous merge was being gated: `1e2a0af6a` (IndexTTS-2.5 W3 FCM front end, #634) and `7138d1618` (the chunked- prefill token-budget scheduler test, #669). Neither touches this row, but gating a tree and pushing a different one is not evidence, so the newer base is merged in and the whole gate runs ONCE on the tree that is actually pushed. Both new commits are disjoint from this row: campplus / scheduler / a new perf spec. They touch NO keyed record, NO matrix and NOT the MODEL pin -- checked by name, not assumed. `.agents/roadmap_v1.md` auto-merged cleanly and was redone by key anyway, for the same reason as the previous merge: a keyed record never takes a three-way merge, because here the clean result and the wrong result are indistinguishable. `origin/main`'s table taken wholesale, the single #490 entry re-inserted after its unique anchor, 0 evictions. The MODEL pin was RE-DERIVED a third time against this final tree with the checker's own parser: 375, unchanged, because neither new commit adds a matrix row. It is re-derived rather than assumed unchanged -- that assumption is what made it stale twice before. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…hird re-merge, and the last one this row chases (#490) `main` landed `87dbe377b` and `5e010c948` (IndexTTS-2.5 CAMPPlus completion and the w2v-BERT-2.0 Conformer, #634) while the previous tree was being gated. Merged and re-gated rather than pushed against a base that had moved. The delta is disjoint from this row. It touches NO keyed record -- not the model matrix, not the roadmap issue table, not the porting inventory, not the MODEL pin -- checked by name rather than assumed. The single overlapping file is `tests/CMakeLists.txt`, which is a pure union: main's four new registrations and this row's `test_qwen3_8_text_only` all survive, asserted by count. The MODEL pin was re-derived a fourth time against this tree with the checker's own parser: 375, unchanged, because the delta adds no matrix row. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ds-only, the MODEL pin RE-DERIVED (#490) The fourth re-merge of this row. `main` lands every 20-30 minutes, so the branch went DIRTY again before it could be merged; the row's own code is unchanged and carries its full gate from `f308c995` (clean -Werror build, serial ctest 431/431, and green CI on both verify lanes, build-test-cpu, build-test-cpu-arm64, agent-record, documentation-checkpoint, pr-size, commit-protocol-tag, device-leakage and sanitize-cpu (thread)). `.agents/model-matrix.md` was the only conflict, as on the previous three. It is a KEYED record, so it was resolved by taking origin/main's file WHOLESALE and re-applying this row's four entries on top -- the two matrix rows `MODEL-TEXT-qwen3-5-qwen3-5-for-causal-lm` and `MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm` plus their two architecture-checklist lines -- each inserted against an anchor asserted to occur exactly once. Every unrelated key is byte-identical to origin/main: the file's whole diff against main is this row's four additions and the recounts below, and nothing was evicted. `docs/USAGE.md` and `tests/CMakeLists.txt` auto-merged; main's new test registrations and the row's `test_qwen3_8_text_only` wiring both survive. The derived counts were RE-DERIVED off the merged tree with the checker's own parser, never carried forward -- the reason this row has been wrong about them before. main reads 373 rows / 369 architectures / 50 engaged / PARTIAL 20; this row adds two PARTIAL rows, so the merged matrix is 375 / 371 / 52 / PARTIAL 22, and `scripts/check-agent-record.py`'s MODEL pin stays 375 because main happened to arrive at the same 373 base it did last time. That equality is a coincidence of this merge, not a carry-over: the parser was run against the file as it now stands and returned 375 rows, 323 INVENTORIED and 52 engaged. No checker was weakened. The merge touches NO `src/` and NO `include/` file. Of this row's 29 files it changes exactly seven -- model-matrix, roadmap, the four `docs/` pages and `tests/CMakeLists.txt` -- so every compiled byte is identical to the already-gated `f308c995`, and the C++ suite was deliberately NOT re-run here. CI recompiles the affected lanes on push. Verified after merging: both `Qwen3_5[Moe]ForCausalLM` still registered, `ResolveQwen3_5BackbonePrefix` still refusing a mixed index, `CheckMoeExpertLayoutSupported` intact, the `mtp.` exclusion pinned and `vt::LoadUnaligned` routing unchanged. `audit-live-rows` and `test_audit_live_rows` fail here on an abandoned ACTIVE row, `MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation`. That red is main's, not this row's: reproduced identically on a pristine detached checkout of origin/main (same 41 tests, same single failure), and this branch's `.agents/engine-matrix.md` is byte-identical to main's. The audit's verdicts differ from main's only by our two PARTIAL rows and the line numbers they shift; `1 abandoned ACTIVE` is unchanged on both sides. Repairing another row's record is outside this row's authority. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue: #490 · Spec:
.agents/specs/qwen38-text-only.mdWhat this is, stated precisely
Qwen/Qwen3.8-2.4T-A95BdeclaresQwen3_5MoeForCausalLM/model_type: qwen3_5_moe_text— the same GDN-hybrid MoE backbone we already run token-exact on Qwen3.6-35B-A3B. Every structural knob matches (head_dim256, linear k/v head dim 128,full_attention_interval4,partial_rotary_factor0.25,rope_theta1e7, mtp 1, vocab 248320); the differences are pure scale and all config-driven.What this delivers: both architecture strings dispatch; the backbone weight namespace resolves for either spelling; mixed and empty indices are refused; an unsupported MoE expert layout is refused with a message naming the missing piece; the dense arm routes BF16/FP8/NVFP4 by tensor presence; the gated rows stay byte-identical.
What it does NOT deliver — and this correction matters. An earlier revision of this row asserted that the tensor names being "identical modulo the prefix" meant the loader body was already correct. The names are identical; the conclusion did not follow.
LoadQwen3_5Moe→LoadMoeExpertsInto→LoadNvfp4Rawis a quantized-checkpoint loader: it requires per-expert.weightU8 +.weight_scaleF8_E4M3 +.weight_scale_2. The published bf16 checkpoint has 3-D stacked experts and zero scale tensors, so it dies atlm_headbefore reaching them. This was invisible because our gated 35B row loads the requantizednvidia/Qwen3.6-35B-A3B-NVFP4, not a published Qwen bf16 repo.That false claim had reached the spec, a shipped header,
docs/STATUS.md,model-matrix.mdand three commit messages. It is struck everywhere, the debt is recorded the right way round (the quantized arm is the one implemented; bf16/3-D-stacked is owed,porting-inventory.md§9 deviation 17(e)), and the previously-false closing condition is corrected on all seven surfaces. The dense-vs-MoE asymmetry is now stated wherever the limitation appears.Ahead of the pin, deliberately
Parity pin is
555967922, whose registry has only theForConditionalGenerationentries. The text-only arms arrived upstream in PR vllm#50210 /ad5d29db7, post-pin. This is a forward port of one upstream PR; it does not advance the pin. Recorded inporting-inventory.md§9 deviation 17, along with one tracked divergence: a mixed index is refused here where upstream'sWeightsMappernormalizes.Evidence
CPU (this host): clean
-Werrorbuild 1215/1215, zero warnings; serial ctest 404/404, 2 skips (test_modelopt_mixed_precision_checkpoint,test_voxtral_e2e) counted as skips, not coverage.test_qwen3_8_text_only7/7 cases, 747/747 assertions.GPU (dgx, GB10 sm_121a), at the reviewed head
1cc951cd: SACREDtest_qwen36_paged_engine315/315,test_qwen27_paged_engine235/235,test_qwen3coder_paged_engine138/138 — real runs, zero skips, goldens byte-identical.test_qwen35_plain_weightswas a SKIP (4 assertions,Qwen/Qwen3.5-4Bnot cached) and is not counted.Reviewed twice, independently. Final review reproduced five mutations — including deleting each side of the merged registry union, proving neither became dead code — and verified globally that no assertion was lost: the row's entire delta pre- and post-merge differs by exactly 48 lines, all accounted for.
The 747-vs-787 assertion difference between CPU and CUDA builds is explained exactly:
CheckSameAttn's runtimeDenseNativeEnabled()branch keeps projections fp8-resident on CUDA, adding 5 checks × 4 projections × 2 call sites = +40.Owed, recorded not waived
1cc951cd; the merges since changed none of the row's source, so the evidence transfers by construction, not by re-run — stated as an argument, not a run.🤖 Generated with Claude Code