feat(SPEC-DFLASH2): the grouped dynamic depthwise convolution, reached by a DFlash2 draft that now refuses at the selector (#1314) - #1465
Merged
Conversation
…Flash2 draft run it (#1314, #1327) Wave W2 of `SPEC-DFLASH2` ([#1314](#1314)). W1 shipped a refusal; this wave ships the first of the two mechanisms it named, and moves the refusal so that mechanism is REACHED. **`vt::DFlashGroupedConv` is the project's first dynamic, grouped, block-masked convolution.** `out[i,c] = sum_t (base[side,t,c] + delta[i,side,t,g(c)]) * x[i-t,c]`, with tap `t` live only where `(i mod (1+k)) >= t`, `g(c) = c / conv_group_size`, and `base_kernel` dim 0 the prepare/finish SIDE rather than a tap. Three things separate it from the shipped `KERNEL-DEPTHWISE-CONV1D`: the weights are projected per position from the sublayer input rather than static, one delta serves a GROUP of channels while the base is per channel, and the tap mask is over the query BLOCK rather than causal over the sequence -- which is what lets a proposal position see the ones before it without another backbone pass. The CPU kernel is the authoritative reference and rounds to the tensor dtype after each step, as upstream's bf16 chain materializes it, so the op is elementwise with no reduction-order freedom and the CUDA mirror is specified BIT-IDENTICAL rather than within an envelope. Both of upstream's position-mask arms are ported (`pos & (block-1)` and `pos % block`) and gated at block 5, 8 and 16. The op lands as kernel-matrix row `KERNEL-DFLASH2-GROUPED-CONV` (`ACTIVE`, `CLAIM-SPEC-DFLASH2-W2`), which is why the `KERNEL` count in `scripts/check-agent-record.py` moves 52 -> 53. It stays `ACTIVE` rather than `DONE` because its CUDA arm has never compiled here. **The refusal moved so the conv could be reached.** A safetensors `DFlash2DraftModel` draft is now ADMITTED at `CheckDflash2DraftArm`, loads its `attention_conv`/`mlp_conv` tensors, runs the conv in all three of the draft's layer bodies -- including the paged body the production decode path reaches through `ForwardBlockLogitsWithDeviceKV` -- and is refused BY NAME at `RefuseDflash2CandidateSelector`, after the block forward and before anything samples. Keeping the startup refusal would have left every line of this wave unreachable from any production entry point, which AGENTS.md `## Nothing lands dead` forbids. A startup NOTICE names the boundary so the later refusal is not a surprise. The GGUF arm keeps its startup refusal, because its drafter arm has no conv weight path at all and admitting the file would load a DFlash1 draft out of a DFlash2 checkpoint. Recorded as spec `## Risks/decisions` D10. **Two `## Owed` blockers are discharged, and they were blockers.** `MakeQwen3DFlashDraftConfig` could not parse EITHER published DFlash2 `config.json`: it did `c.at("rope_theta")` and `c.at("block_size")` while both drafts nest them under `rope_parameters` and `dflash_config` (O3). It also required `layer_types`, which `XiaomiMiMo/MiMo-V2.5-Pro-FP4-DFlash` does not declare, while upstream reads `getattr(config, "layer_types", None)` (O4). Both are FALLBACKS, tested after the flat spelling, so every published DFlash1 draft resolves byte-for-byte as before. O4 lands with a NAMED REFUSAL for `dflash_config.attention_sink_bias`: upstream passes a per-head sink into its `Attention` and this lane has none, so the parse repair alone would have turned a loud `key 'layer_types' not found` into a draft that loads with the sinks silently absent -- acceptance-only and invisible to a token gate. **#1327 is corrected here.** `## Upstream chain` claimed no published checkpoint exercises `input_embedding_scale`, `output_multiplier` or `final_logit_softcapping`. `z-lab/Muse-Glimmer-30B-DFlash2` sets `output_multiplier 0.19611613513818404` and `final_logit_softcapping 20.0`, and ships `block_size` 16 against the 27B's 8. Both are applied to candidate VALUES before the selector scores them, so a wrong one reorders the top-K and moves acceptance without raising. `## Scope`'s exclusion of a second DFlash2 target family is dropped (upstream registers ONE class; both checkpoints declare `model_type` `qwen3`), G1 now requires both block shapes, and D9 binds W3 to gate the scalars against the checkpoint that sets them. **Red before green, and two gate weaknesses found by the mutation pass.** The conv suite's red-before is a build failure (the op did not exist), exit 2. The draft suite's red-before is 5 cases / 4 failed, exit 1, with `[json.exception.out_of_range.403] key 'rope_theta' not found` -- O3 exactly as the spec predicted it. Eleven mutations then turn the focused suites red, each with its compile status printed and each restored byte-for-byte and verified by sha256. TWO of them came back GREEN first and are recorded rather than hidden: activating both convs at once could not tell one missing call site from none, and the first side probe could not see `args.side` forced to 0. Both gates were repaired before landing, and the second repair needed a fixture fix too -- an "inactive" taps=2 conv with an all-ones base is `x[i] + x[i-1]`, not the identity. **What is NOT reached, named as AGENTS.md `## Nothing lands dead` requires.** Three mutations stayed green and are `## Owed` in `.agents/specs/dflash2-spec-decode.md`, owned by row `SPEC-DFLASH2` under [#1314](#1314). O5: `LoadDflashDraft`'s own `conv_block_size = k + 1` is ungated, because that function is `static` in the loader's anonymous namespace and an entry-point gate on it must load a draft off a live target; owner W4. O6: the CUDA arm has NEVER COMPILED -- the authoring host has no `nvcc`, so the CUDA==CPU bit-identity case reports `no CUDA backend; skipping` and all 9410 assertions in that file are CPU; owed to a GPU lease. O7: `RefuseDflash2CandidateSelector`'s second call site, `GPUModelRunner::propose_drafts_block`, is not gated, and it is the one a user arrives through; owner W4. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…he selector refusal actually reaches (#1314) W2's fresh review returned FAIL on four findings. None was a defect in the shipped behaviour -- the kernel was verified line by line against upstream `_grouped_conv` at the PR head and the convolution is genuinely production-reached -- and all four were about what this repository CLAIMED to have proven. They are repaired here. **The wave's central numerics claim had no executing assertion.** W2 chose per-step bf16 rounding, and that choice is the whole basis for specifying the CUDA arm BIT-IDENTICAL rather than within a tolerance envelope. Nothing tested it. The reviewer replaced the bf16 branch of the `round` lambda in `src/vt/cpu/cpu_ops.cpp` with `return v;`, it compiled clean, and both focused suites stayed fully green: `test_ops_dflash2_grouped_conv` 6/6 cases, 9410/9410 assertions, `Status: SUCCESS!`; `test_qwen3_dflash2_draft` 16/16, 108/108, `SUCCESS!`. Three causes, each verified: the op suite drove the op only in f32, where `round()` is the IDENTITY by construction; `RunCudaParity` returns early on a host with no `nvcc`, so its bf16 shapes never execute; and the draft suite does execute the bf16 branch but asserts only RELATIONALLY between two runs of the same kernel, so a rounding change moves both arms together and cancels. Two CPU-only bf16 cases now pin it. One is hand-computed against literals, at taps 2 over two blocks with `delta = 2^-9` so all three rounding steps run: `bf16(3*89) = bf16(267) = 268` and then `bf16(268 + 3) = bf16(271) = 272`, where rounding once at the end answers 270. Six of its eight outputs move under the other policy. The other asserts bit-exactness at three shapes -- both published blocks, both sides, and taps 3 so more than one accumulate rounding chains -- against a reference that rounds where UPSTREAM materializes rather than where our kernel does. Red first, with the same mutation: 8 cases / 2 failed, 9930 assertions / 225 failed, `Status: FAILURE!`, compile rc 0, `git diff --stat` showing the hunk. Restored byte-for-byte (sha256 verified), rebuilt, green: 8/8, 9930/9930, `SUCCESS!`. `include/vt/ops.h` said "and the gate asserts that" about the bit-identity; it now separates the half that is pinned from the half that is not. **The staged-slice disclosure understated the gap, and that disclosure is what AGENTS.md's permission rests on.** `## Owed` O7 and two comments said `RefuseDflash2CandidateSelector` "has TWO production call sites", one of them gated. It has ONE. `DflashProposeBlock` has no caller outside `tests/` -- an exhaustive grep finds only its definition, its declaration, two prose comments in `runner.cpp`, and tests -- so the site a test can delete-and-redden is test-only, while deleting the real site at `runner.cpp` leaves all four focused suites GREEN. Production coverage of the refusal is ZERO, not one of two. Gating the real site was preferred and is not reachable here: `propose_drafts_dflash` returns early unless `dflash_weights_` is set, that member is only set on the `LoadedModel` construction path, and the synthetic-weights `GPUModelRunner` constructors take no `SpeculativeConfig` at all, so a gate needs an on-disk target plus draft driven through the loader -- the harness O5 already waits on and W4 builds. O7 now says zero, names why, and names W4. **A recorded sha256 did not hash what was embedded.** The three `config.json` documents in `tests/vllm/models/test_qwen3_dflash2_draft.cpp` were labelled VERBATIM with a sha256 beside each. Two were the published file minus its trailing newline, and the third, `XiaomiMiMo/MiMo-V2.5-Pro-FP4-DFlash`, had been re-indented to two spaces and had lost its `auto_map` key, so its recorded hash described a file that is not in this repository. All three literals are now the published bytes, re-fetched and re-hashed, and each recorded sha256 hashes the literal beside it. **The admitted checkpoints were not pinned.** `docs/USAGE.md` gains `## DFlash2 drafts: the exact checkpoints`: repo, revision, file, byte count and sha256 for the admitted bf16 draft (`z-lab/Qwen3.8-27B-DFlash2` @ `50307d4c`, 3 848 817 896 bytes) and for all three refused GGUF arms, plus the target it heads and the second published draft's revision. Every hash was computed over a local copy rather than read from a hub API, which can return an `lfs.oid` that hashes nothing, and the shard was checked semantically too: 81 tensors, all BF16, last data offset exactly on the file size. **One reviewer concern is recorded and NOT fixed.** New `## Owed` O8: `PrecomputeContextKVDevice` projects every layer's context K/V from one shared `hidden_norm(context_states)` and applies no convolution, while upstream has no analogue -- its context K/V is what earlier block forwards wrote, which under DFlash2 came from a conv'd stream. Whether the shortcut stays equivalent now that the conv exists is unaddressed, and if it is wrong the symptom is acceptance-only and token-invisible, which is this row's own named defect class. Owner W3/W4 under [#1314](#1314). O6 also gains what it did not say: the bf16 arm's CPU == CUDA bit-identity is still unpinned on BOTH sides, because that case has never compiled here. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
`main` moved 20 commits while this repair was in flight. One keyed record conflicted: the `ENG-RECORD-ANCHOR-RATCHET` row of `.agents/engine-matrix.md`. `main` had re-pointed its five `scripts/check-agent-record.py` line citations after that file moved; this branch's base had converted the same five to SYMBOL anchors, for the reason the row itself now states -- SPEC-DFLASH2 W2 shifted all five ranges at once by editing the very file the row cites. Resolved as AGENTS.md `## Records` requires: take the complete target-branch row, then reapply the scoped edit. The scoped edit is exactly "no line numbers", so the reapplied row is the symbol-anchored one, and every other field of the row was asserted byte-equal to `origin/main` before it was accepted. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
… 9410 assertions, and the rounding was in none of them (#1314) `KERNEL-DFLASH2-GROUPED-CONV` in `.agents/kernel-matrix.md` recorded the op gate as 6 cases / 9410 assertions and listed six mutations that redden it. The per-step rounding was not among them, and the wave's second fresh review proved why: every case in that file ran in f32, where the rounding is the identity by construction, so replacing the bf16 branch of the `round` lambda with `return v;` compiled clean and left both suites green. The row now records 8 cases / 9930 assertions with the two bf16 cases named, carries the rounding mutation in its set with the red it produces (2 cases and 225 assertions failed, `Status: FAILURE!`), counts THREE gate repairs from this row's mutation passes rather than two, and states beside the CUDA entry that the policy is pinned on CPU while CPU == CUDA bit-identity is pinned on neither side. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
`main` moved two more commits while the full gate ran on the previous merge: the 866-tensor GGUF accounting gate and the new conflict-marker gate. No conflicts; nothing in either commit touches this branch's paths. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
`origin/main` reached c8d926e while W2 was reviewed and repaired. The merge is clean, and the keyed records were verified rather than assumed: both matrices carry every row main has, invent none, and differ from main only where this branch owns the difference -- the `SPEC-DFLASH2` row, the symbol-anchored `ENG-RECORD-ANCHOR-RATCHET` row this branch's own edit forced, and the new `KERNEL-DFLASH2-GROUPED-CONV` row. `check-agent-record.py` reads ENGINE=168 KERNEL=53 with anchor rot unchanged at 38. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave W2 of
SPEC-DFLASH2(#1314):the grouped dynamic depthwise convolution, and a DFlash2 draft that actually runs
it.
vt::DFlashGroupedConvis the project's first dynamic, grouped,block-masked convolution:
out[i,c] = sum_t (base[t,c] + delta[i,t,g(c)]) * x[i-t,c], taps zeroed acrossthe block boundary, wrapping each attention and each MLP sublayer through a
prepare/finishpair driven by one projection of the sublayer input. CPUreference authoritative, CUDA mirroring it, as
KERNEL-ATTN-DFLASH-BLOCKlanded.The refusal moved so the convolution is REACHED. W1 refused a DFlash2 draft
at startup; a convolution behind that refusal would have been dead code. A
safetensors
DFlash2DraftModeldraft is now admitted with a startup notice,loads its
attention_conv/mlp_convtensors, runs the conv in all three draftlayer bodies including the paged one the decode path reaches, and is refused BY
NAME at the candidate selector, after the forward and before anything samples.
The GGUF arm keeps its startup refusal, which W5 owns. A fresh reviewer traced the
production path end to end and mutated all six call sites; every one reddens.
Two blockers discharged.
MakeQwen3DFlashDraftConfigcould parse NEITHERpublished DFlash2 config --
rope_thetais nested underrope_parametersandblock_sizeunderdflash_config-- and it required alayer_typesMiMo's draftdoes not declare. Both fallbacks are tried after the flat spelling, so DFlash1
drafts are unchanged, asserted against the real published artifacts rather than
synthetic ones.
dflash_config.attention_sink_biasis refused by name, becauselanding the parse fix alone would have turned a loud error into a quiet wrong
answer on a checkpoint whose per-head sink this lane does not implement.
The wave was reviewed twice and failed once. The first review found the
central numerics claim had NO executing assertion: per-step bf16 rounding is the
basis for specifying the CUDA arm as bit-identical rather than within an
envelope, and replacing the rounding with
return v;left both suites fullygreen -- the op suite ran only f32 where that rounding is the identity, the CUDA
parity helper returned early, and the draft suite compared two runs of the same
kernel so the change cancelled on both sides. It also found the staged-slice
disclosure understated itself, claiming two production call sites for the selector
refusal where there is one, and where the gated one is test-only.
Both are repaired. Two CPU-only bf16 cases now pin the policy: one hand-computed
against literals where the two policies genuinely disagree --
bf16(3*89) = 268,then
bf16(268+3) = 272, where rounding once at the end gives 270 -- and oneasserting bit-exactness at three shapes against a reference that rounds where
upstream materializes. Under the mutation the suite reds
2 cases / 225 assertions,Status: FAILURE!.## OwedO7 now states that ZERO production callsites are gated and names W4, with the measured reason:
propose_drafts_dflashreturns early unless
dflash_weights_is set, and that member is set only on theloader path.
The second review confirmed the repair independently, and added a mutation of its
own that the repair had not run -- truncating
F32ToBF16's rounding -- whichreddens the hand-computed case while leaving the reference-based one green,
making the shared-helper blindness executable rather than argued.
Gates, rerun by the operator on this head:
agent-ready.py86 ok, 0 FAIL,0 SKIP.
check-agent-record.pyENGINE=168 KERNEL=53, anchor rot unchanged at38. Full ctest 566/572 at
8f8f5957f; the six failures are#1458 (four suites, attributed
to
4712dac40by A/B mutation, not by argument),#294, and one load-dependent
test_serve_low_toolsflake that passes 3/3 serially. None is in this diff.Still owed, and named rather than implied. The CUDA arm has never compiled --
no
nvccon the box that built this -- so CPU/CUDA bit-identity is pinned onNEITHER side (O6), owed to a GPU lease. Zero production call sites of the selector
refusal are gated (O7, W4).
conv_block_size = k + 1inLoadDflashDraftismutation-proven ungated (O5, W4). And O8 records an open equivalence question:
PrecomputeContextKVDeviceprojects every layer's context K/V from one sharedunconvolved tensor, where upstream's context K/V came from a conv'd stream -- if
that shortcut is wrong it is acceptance-only and token-invisible, this row's own
named defect class.
No throughput number is claimed and none is admissible yet: the selector still
refuses, so nothing is timeable.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]