Skip to content

Commit b4f14ee

Browse files
committed
feat(cuda): make CUDA-arch expansion ADDITIVE — feature table, capability threading, tactic registry, queried smem ceiling
Closes the four seam-gaps the breadth-sweep audit named in .agents/specs/breadth-sweep-plan.md §A.2, so adding a CUDA architecture vLLM supports becomes a table-row edit plus a tactic registration instead of a scattered multi-file edit. Structural/mechanical only: behavior-preserving on GB10 (sm_121a) by construction, no new kernel, no numerics change. GAP #1 — per-arch build FEATURE TABLE. The four hardcoded `MATCHES "12[01]a"` guards (VT_FP4_MMA_SM120A, the CUTLASS NVFP4/FP8 TUs, the vendored Marlin MoE) were whole-list substring tests: they could not say WHICH arch supports what, enabled every feature globally for archs that cannot run them, and silently dropped fp4/fp8/Marlin for legitimate spellings such as "121" or vLLM's own "12.1a". Replaced by VT_CUDA_FEATURE_TABLE in the new cmake/CudaArchFeatures.cmake, resolved per arch and REPORTED at configure time. `cuda_archs_loose_intersection()` is a 1:1 port of vllm/cmake/utils.cmake:376-485 @ e24d1b24; the per-feature `<F>_ARCHS` + `if(<F>_ARCHS)` idiom mirrors vLLM CMakeLists.txt:949-953,963 (FP4_SM120), :775-787 (sm120 SCALED_MM), :556-558 (MARLIN). Recorded deviation: the intersection is post-filtered to archs literally requested, because the `a` suffix is load-bearing for the fp4 MMA. GAP #4 — capability threaded to the kernel layer. New cuda_device_caps.h caches ONE probe (compute capability, opt-in smem ceiling, SM count, residency attributes); cuda_backend.cu's registrar now uses it and the backend CARRIES the capability via new vt::Backend::DeviceCapabilityMajor/Minor(). Mirrors vllm/platforms/cuda.py::CudaPlatform.get_device_capability. GAP #2 — runtime SM-dispatch seam. New cuda_arch_tactics.{h,cu}: a type-erased per-family tactic registry with capability predicates and selection counters. LaunchFp4Fp4 now takes the capability and dispatches through it. Exactly ONE tactic is registered — the existing sm_12x native fp4 path — so on GB10 this reduces to the old `#if VT_FP4_MMA_SM120A && NativeFp4MmaEnabled()` test: same kernel, same grid, same stream. A future arch ADDS a tactic from its own TU; the launcher is never edited. Shape mirrors FlashInfer's per-arch tactic registry (fp4_gemm_cutlass_template_sm120.h:187-220). GAP #3 — queried smem ceiling. cuda_paged_attn.cu's 100 KiB opt-in ceiling was a comment assumption with nothing checking it; seven open-coded opt-in blocks became SetDynamicSmemOptIn(), which enforces the cached, QUERIED cudaDevAttrMaxSharedMemoryPerBlockOptin (101376 on GB10 — every existing tile already fits, so this is behavior-preserving) and names the device on a shortfall instead of failing opaquely in the driver. The d==256 shape gate is deliberately untouched. Positive signal (a passing gate does not prove a new path ran): the registry counts selections/fallbacks and VT_ARCH_TACTIC_STATS=1 announces the chosen tactic; new tests assert the counters MOVE and that the backend reports the driver's capability. This does NOT make any untested architecture supported. One tactic is registered, no row moves off INVENTORIED, and a heterogeneous fat build still needs per-source gencode narrowing plus per-arch kernel bodies — a hardware-blocked kernel campaign, not an additive drop-in. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-opus-4-8 [ClaudeCode]
1 parent c56ab28 commit b4f14ee

20 files changed

Lines changed: 1414 additions & 88 deletions

.agents/backend-matrix.md

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,20 @@ it has a native build, ported tests, hardware execution, and a trace showing the
1111
intended dispatch. Performance claims additionally require same-workload runs
1212
under [benchmark-protocol.md](benchmark-protocol.md).
1313

14+
**How an architecture is added (the additive contract, `BACKEND-CUDA-ARCH-ADDITIVITY`).**
15+
Since [arch-additivity](specs/cuda-arch-additivity.md) landed, arch support is
16+
declared as DATA in two places and nowhere else: widen the relevant cells of
17+
`VT_CUDA_FEATURE_TABLE` in [cmake/CudaArchFeatures.cmake](../cmake/CudaArchFeatures.cmake)
18+
(each feature is resolved per arch against the requested target list, so a fat
19+
build can no longer silently drop a capability), and register the arch's kernel
20+
tactic from its own translation unit via
21+
[cuda_arch_tactics.h](../src/vt/cuda/cuda_arch_tactics.h) (host launchers select
22+
at runtime from the cached device capability and are never edited). This makes
23+
the expansion mechanical; it does NOT confer support. Exactly ONE tactic is
24+
registered today — the existing `sm_12x` fp4 path — and every row below that is
25+
not `sm_121` stays `INVENTORIED` because no hardware exists here to execute it.
26+
Cross-family bring-up remains a kernel campaign, not an additive drop-in.
27+
1428
## CUDA compiler branches
1529

1630
The global target list is selected by the CUDA compiler version at
@@ -50,7 +64,7 @@ it does not mean the global list contains that numeric target.
5064
| `BACKEND-CUDA-SM103` | CUDA compute 10.3 | `>=12.8,<13` globally; family via `f` on 13 | `CMakeLists.txt:105-118`; loose intersection `cmake/utils.cmake:393-481` | configurable target [CMakeLists.txt:37](../CMakeLists.txt#L37), applied globally at [CMakeLists.txt:64](../CMakeLists.txt#L64) | - | [CUDA inventory](specs/cuda-architecture-inventory.md) | `INVENTORIED` | - |
5165
| `BACKEND-CUDA-SM110` | CUDA compute 11.0 | `>=13` | `CMakeLists.txt:105-118`; SM10/11 family rules at `:806-837,892-914` | configurable target [CMakeLists.txt:37](../CMakeLists.txt#L37), applied globally at [CMakeLists.txt:64](../CMakeLists.txt#L64) | - | [CUDA inventory](specs/cuda-architecture-inventory.md) | `INVENTORIED` | - |
5266
| `BACKEND-CUDA-SM120` | CUDA compute 12.0 | `>=12.8` | `CMakeLists.txt:105-118`; SM12 FP4 at `:940-970` | SM12 fast paths [CMakeLists.txt:68](../CMakeLists.txt#L68); discrete/UMA classification `src/vt/cuda/cuda_backend.cu:251-273` requires both pageable access and `cudaDevAttrIntegrated` | Local RTX 5070 Ti sm_120a production build; CUDA attribute gate `tests/vt/test_cuda_backend.cpp:153-172` asserts `UnifiedMemory == pageable && integrated` and passes 3/3; plain Qwen3.5-4B direct-load gate `tests/vllm/models/test_qwen35_plain_weights.cpp:162-196` passes 1664/1664. Corrected 18-leg root `/tmp/qwen35-transplant-4b-647a2a6a` proves direct ON peak/stable PSS **2.606/0.727 GiB** vs OFF **8.559/8.558 GiB** with 128/128 paired output IDs; graph-node traces captured. Full component-family coverage remains open | [CUDA inventory](specs/cuda-architecture-inventory.md) | `PARTIAL` - local loader/backend leaf only, not full family coverage | - |
53-
| `BACKEND-CUDA-SM121` | CUDA compute 12.1 / current `121a` gate | `>=12.8,<13` globally; family via `f` on 13 | `CMakeLists.txt:105-118`; SM12 FP4 at `:940-970` | default target [CMakeLists.txt:37](../CMakeLists.txt#L37); runtime [cuda_backend.cu:20](../src/vt/cuda/cuda_backend.cu#L20); SM12 fast paths [CMakeLists.txt:68](../CMakeLists.txt#L68) | [CUDA backend tests](../tests/vt/test_cuda_backend.cpp#L31); [27B gate](../tests/parity/test_qwen27_paged_engine.cpp#L110); [35B gate](../tests/parity/test_qwen36_paged_engine.cpp#L78); [ledger trace](parity-ledger.md#L284) | [CUDA inventory](specs/cuda-architecture-inventory.md) | `PARTIAL` - gate workload only, not full family coverage | - |
67+
| `BACKEND-CUDA-SM121` | CUDA compute 12.1 / current `121a` gate | `>=12.8,<13` globally; family via `f` on 13 | `CMakeLists.txt:105-118`; SM12 FP4 at `:940-970` | default target [CMakeLists.txt:37](../CMakeLists.txt#L37); runtime [cuda_backend.cu:20](../src/vt/cuda/cuda_backend.cu#L20); SM12 fast paths now resolved per arch through the FEATURE TABLE [CMakeLists.txt:68](../CMakeLists.txt#L68), [CudaArchFeatures.cmake:1](../cmake/CudaArchFeatures.cmake#L1); the sole registered runtime tactic [cuda_arch_tactics.h:1](../src/vt/cuda/cuda_arch_tactics.h#L1) | [CUDA backend tests](../tests/vt/test_cuda_backend.cpp#L31); [27B gate](../tests/parity/test_qwen27_paged_engine.cpp#L110); [35B gate](../tests/parity/test_qwen36_paged_engine.cpp#L78); [ledger trace](parity-ledger.md#L284) | [CUDA inventory](specs/cuda-architecture-inventory.md); [arch additivity](specs/cuda-arch-additivity.md) | `PARTIAL` - gate workload only, not full family coverage | - |
5468

5569
## CUDA component target rules
5670

@@ -59,6 +73,7 @@ on every listed target.
5973

6074
| ID | Item | Upstream target/capability rule | Our code | Tests/evidence | Spike/spec | State | Owner |
6175
|---|---|---|---|---|---|---|---|
76+
| `BACKEND-CUDA-ARCH-ADDITIVITY` | Arch-expansion SEAMS: per-arch build FEATURE TABLE, device-capability probe threaded to the kernel layer, runtime SM-dispatch tactic registry, queried opt-in smem ceiling | loose intersection `cmake/utils.cmake:376-485`; per-feature `<F>_ARCHS` gating `CMakeLists.txt:949-953,963` (FP4 SM120), `:775-787` (sm120 scaled-mm), `:556-558` (Marlin); capability seam `vllm/platforms/cuda.py::get_device_capability` | feature table [CudaArchFeatures.cmake:1](../cmake/CudaArchFeatures.cmake#L1), wired at [CMakeLists.txt:64](../CMakeLists.txt#L64); capability [cuda_device_caps.h:1](../src/vt/cuda/cuda_device_caps.h#L1) + [backend.h:45](../include/vt/backend.h#L45) + [cuda_backend.cu:57](../src/vt/cuda/cuda_backend.cu#L57); tactic registry [cuda_arch_tactics.h:1](../src/vt/cuda/cuda_arch_tactics.h#L1), [cuda_arch_tactics.cu:1](../src/vt/cuda/cuda_arch_tactics.cu#L1), consumed by [cuda_matmul_nvfp4.cu:2636](../src/vt/cuda/cuda_matmul_nvfp4.cu#L2636); queried smem ceiling [cuda_paged_attn.cu:93](../src/vt/cuda/cuda_paged_attn.cu#L93) | capability carried [test_cuda_backend.cpp:1](../tests/vt/test_cuda_backend.cpp#L1); registry registered/consulted/SELECTED [test_ops_nvfp4_fp4.cpp:1](../tests/vt/test_ops_nvfp4_fp4.cpp#L1); behavior preservation 27B [235/235](../tests/parity/test_qwen27_paged_engine.cpp#L110) + 35B [315/315](../tests/parity/test_qwen36_paged_engine.cpp#L78) + Coder [6/6](../tests/vllm/models/test_qwen3coder_paged_engine.cpp#L1); [ledger](parity-ledger.md) | [arch additivity](specs/cuda-arch-additivity.md) | `ACTIVE` - seams landed and gated on sm_121a; ONE tactic registered, per-source gencode narrowing and every cross-family tactic body remain HW-blocked | `CLAIM-CUDA-ARCH-ADDITIVITY` |
6277
| `BACKEND-CUDA-COMP-CORE` | Stable core ops: activation, quant, RoPE, norm, sampler, SSM, cache, collectives | all filtered targets; `CMakeLists.txt:375-406,536-538` | CUDA source set [CMakeLists.txt:220](../CMakeLists.txt#L220) | [CUDA op tests](../tests/vt/test_cuda_ops.cpp#L206) on SM121 only | [CUDA inventory](specs/cuda-architecture-inventory.md) | `PARTIAL` | - |
6378
| `BACKEND-CUDA-COMP-COOP-TOPK` | Cooperative top-k | CUDA >=12; `90a`; CUDA 13 uses SM10/11/12 `f` families; `:408-424,540-546` | - | - | [CUDA inventory](specs/cuda-architecture-inventory.md) | `INVENTORIED` | - |
6479
| `BACKEND-CUDA-COMP-MARLIN` | Marlin dense and MoE | FP16 `80+PTX`, Turing `75`; BF16 adds `90+PTX`; FP8 input `89`; SM12 `a/f`; `:548-679,1168-1274` | NVFP4 SM12 subset [CMakeLists.txt:101](../CMakeLists.txt#L101) | [grouped-MoE tests](../tests/vt/test_ops_moe_grouped.cpp#L160) | [CUDA inventory](specs/cuda-architecture-inventory.md) | `PARTIAL` | - |

.agents/coordination.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,8 @@ capture/replay-mode scope remains). `benchmark_binding=false`, no speed credit
243243
payoff measured by the c2/c8 probe + the next authorized exact grid. Rescan §1
244244
items a-runner/b stay with the async/GDN `runner.cpp` owners.
245245

246+
| `CLAIM-CUDA-ARCH-ADDITIVITY` | `BACKEND-CUDA-ARCH-ADDITIVITY` | Claude Code (opus-4-8) | worktree `agent-a19d3c1e79b99c4f9`; dgx scratch `~/work/vllm.cpp-arch-additivity` | worktree branch, base `c56ab28` | `cmake/CudaArchFeatures.cmake`, the CUDA block of `CMakeLists.txt`, `src/vt/cuda/cuda_device_caps.h`, `src/vt/cuda/cuda_arch_tactics.{h,cu}`, the `LaunchFp4Fp4` seam in `src/vt/cuda/cuda_matmul_nvfp4.cu`, the smem opt-in helper in `src/vt/cuda/cuda_paged_attn.cu`, the capability virtuals in `include/vt/backend.h`, the registrar in `src/vt/cuda/cuda_backend.cu`, `tests/vt/test_cuda_backend.cpp`, `tests/vt/test_ops_nvfp4_fp4.cpp`, `.agents/specs/cuda-arch-additivity.md` | `IMPLEMENTED` — the four §A.2 seam-gaps landed, behavior-preserving on GB10 by construction; ONE tactic registered. Per-source gencode narrowing and every cross-family tactic body remain HW-blocked (`ROAD-V1-D1`). Does NOT move any arch off `INVENTORIED`. | 2026-07-21 |
247+
246248
## Handoff queue
247249

248250
| Priority | Row/block | Dependency | Next handoff | State |

0 commit comments

Comments
 (0)