perf(gdn): optimize post-conv and causal-conv kernels - #155
Open
richiejp wants to merge 10 commits into
Open
Conversation
Build and upload exact causal-convolution work descriptors once per step, reuse them across GDN layers, and make the exact register-kernel mapping the default. Keep the production AsyncLLM benchmark path in the same checkpoint so the measured scheduler and kernel improvement is reproducible on current upstream/main. Row: row/KERNEL-SSM-MAMBA-EXACT-CHUNKS FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Canonicalize semantic consumer bodies without Python-version-only AST fields, and gate the representation under Python 3.12 and 3.13. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Make the row-PR provenance visible to GitHub's synthetic merge check without changing the measured tree. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Record the merged Gemma-4 MoE path as known merged-GEMM drift and close the stale embeddings claim after PR mudler#137 landed. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Select the measured sm_120 post-conv residual, reject the existing split grid, and bind the upstream 16-token CUDA experiment and gates. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Port the upstream 16-token per-head schedule behind an opt-in flag, preserve the existing reduction order byte-exactly, and record the sm_120 A/B evidence. Row: row/KERNEL-SSM-MAMBA-POSTCONV-TILE FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
Route runtime-width and both K=4 CUDA launch arms through the same portable, mutation-tested callback dispatcher. Pin strict arm-2 parsing and resolved non-K4 fallback semantics. row/KERNEL-SSM-MAMBA-POSTCONV-TILE FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5.6-sol [Codex]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Row
KERNEL-SSM-MAMBA— Qwen GDN post-conv and causal-conv kernel sublevers; one row per PR.Before starting
scripts/ready-for-helper.pyresult when applicable:.agents/kernel-matrix.md:157(KERNEL-SSM-MAMBA,INVENTORIED), feedingROAD-V1-C2-LOCAL-BF16. Spikes:.agents/specs/sm120-qwen35-postconv-token-tile-2026-08-08.mdand.agents/specs/sm120-qwen35-conv-channel-tile-2026-08-08.md.src/vt/cuda/cuda_gdn.cu,src/vt/cuda/gdn_prefill_conv.h,tests/vt/test_gdn_prefill_conv.cpp,tests/vt/test_ops_gdn.cpp, and pinned vLLM/FLA causal-conv plus fused post-conv kernels recorded withfile:lineanchors in the two spikes.What changed
Add two independently selectable, default-off CUDA specializations.
VT_GDN_POSTCONV_TOKEN_TILE=1ports the upstream-shaped 16-token/four-warp post-conv schedule while preserving the existing 128-lane reduction tree byte-for-byte.VT_CONV_CHANNEL_TILE=1specializes the causal-conv path for width four at the existing one-channel-per-thread grid; arm=2is retained as the measured 256-channel/two-channels-per-thread falsification arm. Production dispatch and the portable mutation gate share one callback dispatcher, so the tests prove that each selected arm is actually invoked.Evidence
scripts/agent-preflight.shpasses after rebasing onto canonicalupstream/mainc05cee1d2; all record gates, mutation suites, live-row audit, and committed-range document checks are green.vllm-benchrelink; portable dispatch 9/9 cases, 88/88 assertions; CUDA GDN 67/67 cases, 4631/4631 assertions; Qwen3.5 paged-forward 4/4 cases, 8/8 assertions. The accepted production arms produce identical token hashes.docs/STATUS.mdanddocs/BENCHMARKS.mdupdated; the model/feature surface did not expand. Full profiler evidence and the void stale-link series are retained in the two specs and.agents/benchmark-record.md.Speed claims
${GPU_LOCK}and they are recorded indocs/BENCHMARKS.mdwith the repro recipe.On the local RTX 5070 Ti / Qwen3.5-4B c32 workload, the post-conv tile reduces graph-node time 227.887 to 122.587 ms (1.859x) and improves every observed enclosing axis, including total/output throughput by 0.532%. The causal-conv K4 arm reduces graph-node time 234.605 to 219.506 ms (6.44%) and improves whole-run total/output throughput by about 0.12%. The 256-channel arm is 4.05% slower than the K4 arm and is recorded as falsified.
Honest gaps
vllm-benchhad not relinked the rebuilt CUDA library. The accepted series explicitly relinked the benchmark and trace-proved the specialized kernels.mainc05cee1d2independently failscheck-device-leakage.py:src/vllm/v1/worker/gpu/runner.cppcontains one new hard-codedDeviceType::kCUDA(DSRkcuda1 > baseline 0). perf(gdn): dispatch exact causal-conv chunks #127 and this PR inherit that merged-tree failure; neither kernel diff introduced it.