Skip to content

ROCm d=128 decode: the VT_ATTN_DECODE_D128 flag-ON ctest registration cannot show the new kernel ran, and runs 0 assertions off ROCm #1134

Description

@localai-bot

The VT_ATTN_DECODE_D128=1 ctest registration added by #767 exists to prove the opt-in d=128 ROCm decode arm is gated rather than silently skipped. It cannot do that, for two independent reasons, and the two compose so that there is no machine in this project on which the registration means anything.

1. RegisteredDevices() excludes kCPU, so the case is empty on every CPU runner.

// tests/vt/test_backend_cross_device.cpp:84-96
std::vector<DeviceType> RegisteredDevices() {
  std::vector<DeviceType> out;
  for (DeviceType t : {DeviceType::kCUDA, DeviceType::kMETAL, DeviceType::kVULKAN,
                       DeviceType::kXPU, DeviceType::kROCM}) {

Every case in that file loops over this list, including the new "paged attention at Qwen3 geometry (bf16, GQA 2, head_dim 128) matches the CPU oracle". On a CPU-only runner the list is empty, the loop body never executes, and the run reports test cases: 1 | 1 passed, assertions: 0, exit 0. That is the shape #463 named: a SKIP wearing a pass. CI has no ROCm hardware, so this is what CI actually measures — for both registrations.

2. Even on ROCm, declines cannot distinguish the two registrations.

The case's only backend-side assertion is OpProviderStats::declines == 0. OpProviderStats counts at provider granularity, not kernel granularity, and both PagedAttnOnline and PagedAttnDecodeGqaBf16/PagedAttnDecodeOptBf16T live behind the same ROCm provider. So declines == 0 holds identically with VT_ATTN_DECODE_D128 set and unset. The flag-ON registration is byte-identical in its observable assertions to the flag-OFF one; the only thing that differs is the environment, and nothing reads it back.

The NMSE bound does not close the gap either: the arm is correctness-complete, so both kernels pass it. A gate that passes on either kernel does not tell you which one ran.

Consequence. .agents/specs/rocm-decode-attn-d128.md §9 stop condition 2 — "stop if the flag-ON arm cannot be shown to reach the new kernel; confirm selection counts, not just tokens" — is open, and #767 lands with it open and says so. This issue is the record of that, so the debt is not carried only inside a spec section.

What closing it needs. A kernel-selection counter in src/vt/rocm/rocm_paged_attn.hip, readable from the test, asserted to differ between the two ctest registrations: nonzero d=128 decode-opt launches with the flag set, zero with it unset. Independently, the CPU-runner half wants either kCPU added to RegisteredDevices() (so the case has a non-vacuous arm everywhere) or an explicit non-zero-assertion floor, per #463.

Not a duplicate of:

Found during review of #767 (ROCm head_dim=128 decode arm). Filed rather than fixed in flow because closing it means adding an instrumentation seam to rocm_paged_attn.hip and re-gating on gfx1200 hardware, which is its own change with its own review — not the small-and-obvious shape AGENTS.md scopes the in-flow rule to.

Owned by BACKEND-ROCM.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions