The vt reference-tier fallback SEGFAULTS on device tensors instead of refusing by name, and a CUTLASS-less build reaches it silently

[localai-bot]

Claim

When an op has no native CUDA kernel registered, vt falls back to the portable CPU implementation and announces it:

[vt reference-tier] op=QuantFp8Static device=cuda has NO native kernel; running the PORTABLE CPU fallback (correct but slow)

That message says "correct but slow". It is not correct — the fallback is then handed tensors whose data pointers are device allocations, dereferences them on the host, and the process dies with SIGSEGV.

Measured on dgx.casa (GB10, sm_121), Release, -DVLLM_CPP_CUDA=ON -DVLLM_CPP_CUDA_ARCHITECTURES=121, running a CPU-vs-CUDA comparison test for vt::QuantFp8Static:

[vt reference-tier] op=QuantFp8Static device=cuda has NO native kernel; running the PORTABLE CPU fallback (correct but slow)
TEST CASE:  G2: CPU QuantFp8Static == CUDA QuantFp8Static, byte for byte
FATAL ERROR: test case CRASHED: SIGSEGV - Segmentation violation signal

[doctest] test cases:  2 |  1 passed | 1 failed | 2 skipped
[doctest] assertions: 43 | 43 passed | 0 failed |
[doctest] Status: FAILURE!

Process exit 139.

Two distinct defects

1. The fallback must refuse, not crash. A portable CPU kernel receiving DeviceType::kCUDA tensors cannot execute correctly under any circumstance. It should refuse by name — naming the op, the device, and the build feature that would have provided the native kernel — rather than dereferencing device memory. "Correct but slow" is an actively misleading claim in that message when the device is not CPU.

2. The build silently reaches that state. kQuantFp8Static's CUDA registration lives in src/vt/cuda/cuda_matmul_fp8_cutlass.cu:359, so it exists only when the CUTLASS fp8 feature is compiled. Without CUTLASS the configure log says:

CUDA feature cutlass-fp8: DISABLED (no requested arch in [121] provides it)
CUTLASS not found / no CUTLASS-dependent feature for this arch; NVFP4 GEMM + FA2 disabled
  (set -DVLLM_CPP_CUTLASS_DIR=<cutlass> or -DVLLM_CPP_CUTLASS_FETCH=ON)

The configure output is honest. What is missing is that nothing at run time connects the crash to it: a CUDA build without CUTLASS produces a binary that looks fully functional, announces a "correct but slow" fallback, and then dies. The operator has to already know that QuantFp8Static is CUTLASS-gated to interpret the SIGSEGV.

Why this matters beyond one op

This is the shape of defect this project keeps paying for: an instrument that reports a state it is not in. A gate written to compare "CPU vs CUDA" on a CUTLASS-less build is in fact comparing CPU against the CPU fallback — and had it not segfaulted, it would have passed, byte for byte, while proving nothing at all. The crash is the lucky outcome. The silent pass is the dangerous one, and it is available to any op whose native kernel is feature-gated.

Note the doctest shape too: assertions: 43 | 43 passed | 0 failed printed beside Status: FAILURE!. Anything grepping the assertions line alone reads this as green.

What done looks like

1. The reference-tier dispatch refuses by name when the tensors' device is not CPU, naming the op, the device, and the missing build feature. No dereference.
2. The "correct but slow" wording is conditioned on the device actually being CPU.
3. A test that a feature-gated op refuses rather than crashes when its native kernel is absent — driven through the real dispatch, not a hand-built call.
4. Consider whether a CUDA build lacking CUTLASS should warn at engine construction, not only at configure time, since the configure log is long gone by the time anyone runs a binary.

Found while running the owed G2 gate for #468 / #842 on GB10. That gate's result is VOID, not FAIL — it never reached a CUDA kernel; it is being re-run with -DVLLM_CPP_CUTLASS_FETCH=ON.