CUDA: support ncols > 1024 in bitonic argsort/top-k The original bito…

[Geramy]

Overview

This PR changes the argsort / top-k max cols to the maximum of the gpu available LDS by changing the kernel to use the maximum amount of X elements per thread. This was brought on due to the following issue:
#26399 (comment)

This mainly targets HIP devices due to there being no ggml_cuda_op_top_k implementation on hip.

Additional information

Please note drift is not yet calculated in the below table.

┌──────┬───────────┬──────────┬────────┐
│ n_kv │ TG before │ TG after │   %    │
├──────┼───────────┼──────────┼────────┤
│ 1004 │ 15.64     │ 15.13    │ −3.3%  │
├──────┼───────────┼──────────┼────────┤
│ 2004 │ 16.48     │ 16.29    │ −1.2%  │
├──────┼───────────┼──────────┼────────┤
│ 3004 │ 16.41     │ 16.24    │ −1.0%  │
├──────┼───────────┼──────────┼────────┤
│ 4004 │ 16.36     │ 16.18    │ −1.1%  │
├──────┼───────────┼──────────┼────────┤
│ 4504 │ 15.24     │ 16.05    │ +5.3%  │
├──────┼───────────┼──────────┼────────┤
│ 5004 │ 15.20     │ 16.08    │ +5.8%  │
├──────┼───────────┼──────────┼────────┤
│ 5504 │ 14.28     │ 16.02    │ +12.2% │
└──────┴───────────┴──────────┴────────┘

Requirements

1. 2* AMD GPU
2. VRAM > 83GB
3. Deepseek-V4-Flash

• I have read and agree with the contributing guidelines Yes
• AI usage disclosure: YES, to run benchmarks.

[Geramy]

@pwilkin

[Geramy]

This would close issue #26399 (comment)