Commit b4f14ee
committed
feat(cuda): make CUDA-arch expansion ADDITIVE — feature table, capability threading, tactic registry, queried smem ceiling
Closes the four seam-gaps the breadth-sweep audit named in
.agents/specs/breadth-sweep-plan.md §A.2, so adding a CUDA architecture vLLM
supports becomes a table-row edit plus a tactic registration instead of a
scattered multi-file edit. Structural/mechanical only: behavior-preserving on
GB10 (sm_121a) by construction, no new kernel, no numerics change.
GAP #1 — per-arch build FEATURE TABLE. The four hardcoded
`MATCHES "12[01]a"` guards (VT_FP4_MMA_SM120A, the CUTLASS NVFP4/FP8 TUs, the
vendored Marlin MoE) were whole-list substring tests: they could not say WHICH
arch supports what, enabled every feature globally for archs that cannot run
them, and silently dropped fp4/fp8/Marlin for legitimate spellings such as
"121" or vLLM's own "12.1a". Replaced by VT_CUDA_FEATURE_TABLE in the new
cmake/CudaArchFeatures.cmake, resolved per arch and REPORTED at configure time.
`cuda_archs_loose_intersection()` is a 1:1 port of vllm/cmake/utils.cmake:376-485
@ e24d1b24; the per-feature `<F>_ARCHS` + `if(<F>_ARCHS)` idiom mirrors vLLM
CMakeLists.txt:949-953,963 (FP4_SM120), :775-787 (sm120 SCALED_MM), :556-558
(MARLIN). Recorded deviation: the intersection is post-filtered to archs
literally requested, because the `a` suffix is load-bearing for the fp4 MMA.
GAP #4 — capability threaded to the kernel layer. New cuda_device_caps.h caches
ONE probe (compute capability, opt-in smem ceiling, SM count, residency
attributes); cuda_backend.cu's registrar now uses it and the backend CARRIES the
capability via new vt::Backend::DeviceCapabilityMajor/Minor(). Mirrors
vllm/platforms/cuda.py::CudaPlatform.get_device_capability.
GAP #2 — runtime SM-dispatch seam. New cuda_arch_tactics.{h,cu}: a type-erased
per-family tactic registry with capability predicates and selection counters.
LaunchFp4Fp4 now takes the capability and dispatches through it. Exactly ONE
tactic is registered — the existing sm_12x native fp4 path — so on GB10 this
reduces to the old `#if VT_FP4_MMA_SM120A && NativeFp4MmaEnabled()` test: same
kernel, same grid, same stream. A future arch ADDS a tactic from its own TU;
the launcher is never edited. Shape mirrors FlashInfer's per-arch tactic
registry (fp4_gemm_cutlass_template_sm120.h:187-220).
GAP #3 — queried smem ceiling. cuda_paged_attn.cu's 100 KiB opt-in ceiling was
a comment assumption with nothing checking it; seven open-coded opt-in blocks
became SetDynamicSmemOptIn(), which enforces the cached, QUERIED
cudaDevAttrMaxSharedMemoryPerBlockOptin (101376 on GB10 — every existing tile
already fits, so this is behavior-preserving) and names the device on a
shortfall instead of failing opaquely in the driver. The d==256 shape gate is
deliberately untouched.
Positive signal (a passing gate does not prove a new path ran): the registry
counts selections/fallbacks and VT_ARCH_TACTIC_STATS=1 announces the chosen
tactic; new tests assert the counters MOVE and that the backend reports the
driver's capability.
This does NOT make any untested architecture supported. One tactic is
registered, no row moves off INVENTORIED, and a heterogeneous fat build still
needs per-source gencode narrowing plus per-arch kernel bodies — a
hardware-blocked kernel campaign, not an additive drop-in.
FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]1 parent c56ab28 commit b4f14ee
20 files changed
Lines changed: 1414 additions & 88 deletions
File tree
- .agents
- specs
- cmake
- docs
- include/vt
- scripts
- src/vt/cuda
- tests/vt
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
14 | 28 | | |
15 | 29 | | |
16 | 30 | | |
| |||
50 | 64 | | |
51 | 65 | | |
52 | 66 | | |
53 | | - | |
| 67 | + | |
54 | 68 | | |
55 | 69 | | |
56 | 70 | | |
| |||
59 | 73 | | |
60 | 74 | | |
61 | 75 | | |
| 76 | + | |
62 | 77 | | |
63 | 78 | | |
64 | 79 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
243 | 243 | | |
244 | 244 | | |
245 | 245 | | |
| 246 | + | |
| 247 | + | |
246 | 248 | | |
247 | 249 | | |
248 | 250 | | |
| |||
0 commit comments