feat(MODEL-MM-indextts2): assemble the S2Mel DiT block (#634) - #722
Merged
Conversation
FOLLOWING_AGENTS_PROTOCOL Composes the primitives already on main into gpt-fast's TransformerBlock as the DiT uses it, and gates the WHOLE block against upstream executed directly. 7 cases / 34 assertions. h = x + attention(attention_norm(x, c)) out = h + feed_forward(ffn_norm(h, c)) BOTH RESIDUALS ARE FULL, unlike the macaron HALVES in the w2v-bert Conformer already on main — two block types with different residual conventions in the same lane, which is the kind of thing ported by muscle memory. The mutation that halves the attention residual goes RED. SwiGLU gates on W1, not W3: `w2(silu(w1 x) * w3 x)`. Swapping them yields an identically shaped network that still trains, and the checkpoint stores them as separate tensors, so only the values tell. Rotary applies to q and k, NEVER to v. A GENERATOR BUG THE GATE CAUGHT, and it is the instructive part. Two cases began FAILING WITH ZERO FAILED ASSERTIONS — they THREW instead of asserting, because `kQ` had been emitted at the wrong size. Cause: the new block section looped `for n, q in ff.named_parameters()`, SHADOWING the rotary input `q`, so the emitted tensor was an FFN weight. Reading only the assertion line would have shown "12 passed | 0 failed" and looked fine; the CASE count is what exposed it, and after the fix assertions went 12 -> 34 because ~22 had been skipped entirely. This is the trap this repo already records about `grep assertions:` hiding thrown cases, met from the generator side. Mutation results in this commit are therefore reported by CASE COUNT, not by the assertion line. MUTATION EVIDENCE with compile status: swapping W1/W3, skipping rotary on K, and halving the attention residual each RED at compile_err=0. S2Mel's block is complete. Remaining: the talker head, then composition. No e2e claim. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Composes the primitives on main into gpt-fast's
TransformerBlockas the DiT uses it, gated whole against upstream. 7 cases / 34 assertions.Both residuals are full, unlike the macaron halves in the w2v-bert Conformer already on main — two block types with different residual conventions in the same lane, which is what gets ported by muscle memory. The halving mutation goes RED.
SwiGLU gates on W1, not W3. Swapping yields an identically shaped network that still trains; the checkpoint stores them separately, so only values tell. Rotary applies to q and k, never v.
A generator bug the gate caught — and how
Two cases started failing with zero failed assertions: they threw rather than asserted, because
kQwas emitted at the wrong size. Cause: the new block section loopedfor n, q in ff.named_parameters(), shadowing the rotary inputq, so the emitted tensor was an FFN weight.Reading only the assertion line would have shown
12 passed | 0 failedand looked fine. The case count exposed it — and after the fix assertions went 12 → 34, because ~22 had been skipped entirely.That's this repo's recorded
grep assertions:trap, met from the generator side. Mutation results here are therefore reported by case count, not the assertion line.Mutation evidence
Swapping W1/W3, skipping rotary on K, halving the attention residual — each RED at
compile_err=0.Status
S2Mel's block is complete. Remaining: the talker head, then composition. No e2e render.