[BUG] Fix hybrid kvcache kernel page size issue

[vadiklyutiy]

Purpose

The following

VLLM_USE_TRTLLM_ATTENTION=0 vllm serve Qwen/Qwen3-Next-80B-A3B-Instruct -tp 4 --enable-expert-parallel --no-enable-prefix-caching --async-scheduling

lm_eval --model local-completions --model_args model=Qwen/Qwen3-Next-80B-A3B-Instruct,base_url=http://localhost:8000/v1/completions -t gsm8k --num_fewshot 5 --batch_size 250

Produced

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.0197|±  |0.0038|
|     |       |strict-match    |     5|exact_match|↑  |0.0000|±  |0.0000|

The issue was with intermix of kv_manager_block_size and kernel_block_size we passed kv_manager_block_size instead of kernel_block_size that cause garbage loads in attn kernel.

Test Result

After fix

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.8514|±  |0.0098|
|     |       |strict-match    |     5|exact_match|↑  |0.8089|±  |0.0108|

[gemini-code-assist]

Code Review

This pull request effectively resolves a critical bug in the hybrid KV cache for the FlashInfer backend, where an incorrect block size was being passed to the attention kernel, leading to significant performance degradation. The fix correctly determines the kernel_block_size and uses it for the FlashInfer page_size, which is the right approach. The accompanying refactoring, which moves the logic for finding compatible block sizes into AttentionSpec, is a good design improvement that centralizes the logic and removes code duplication from GPUModelRunner. The changes are well-executed and appear robust.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[vadiklyutiy]

Because the root came from #24486
CC @zhiyuan1i @heheda12345

[heheda12345]

FYI for the discussion related to this bug #26936

[vadiklyutiy]

FYI for the discussion related to this bug #26936

Here at least 2 bugs :)
Seems there are a lot of different bugs discussed in #26936. But anyway some of them should be fixed with this PR

[vadiklyutiy]

@pavanimajety @heheda12345
Could you please take a look at this PR?

[tdoublep]

I have solved this is a slightly different way in this PR: #27753

I opted to change the way we intialize things so we can pass the kernel block sizes to the constructor of the attention metadata builders.

[heheda12345]

can you refactor _select_common_block_size a bit so that you can call something like GPUModelRunner._select_common_block_size? I don't think it's a good idea to move to self.kv_cache_spec.find_compatible_kernel_block_sizes

[vadiklyutiy]

To avoid misunderstand I'd like to double check do you mean find_compatible_kernel_block_sizes or _select_common_block_size?

[heheda12345]

Depends on you. Just call GPUModelRunner.xxxx() here and minimize the changes to gpu model runner

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @vadiklyutiy.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[vadiklyutiy]

I'm not going to play this game