[Bug] ROCm MI355 HiCache Broken - Poor Performance for Realistic Agentic Workload

[functionstackx]

Checklist

• I searched related issues but found no solution.
• The bug persists in the latest version.
• Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback.
• If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
• Please use English. Otherwise, it will be closed.

Describe the bug

Hi @HaiShaw @1am9trash ,

this hicache bug doesnt seem exclusive to just qwen3.5 but affects lots of models

AI analysis below

Both HiCache arms attempted on v0.5.17-rocm720-mi35x-20260811 (run 31579113764) died in scheduler init:

在 v0.5.17-rocm720-mi35x-20260811 上尝试的两个 HiCache 分支均在调度器初始化阶段失败：

File "sglang/srt/mem_cache/kv_cache_builder.py", line 81, in maybe_register_hicache_draft
File "sglang/srt/mem_cache/hybrid_cache/hybrid_pool_assembler.py", line 936, in build_full_draft_pools
if pool.layer_num == 0:
AttributeError: 'HybridLinearKVPool' object has no attribute 'layer_num'
The hybrid stack itself is healthy on gfx950 — the same logs show 93.37 GB target KV and 50.65 GB Mamba host pools allocating per rank under page_first, then Attached hybrid pool stack to UnifiedRadixCache: pools=KV + MAMBA, transfer_layer_num=60. Only the MTP draft pool crashes.

Reproduction

v0.5.17-rocm720-mi35x-20260811

Environment

v0.5.17-rocm720-mi35x-20260811

[1am9trash]

Hi, @functionstackx
It seems that there already has a PR dealing this bug.
PR: #34560

[functionstackx]

Hi @1am9trash do u happen to know when this PR will get merged

[1am9trash]

I saw the community has reviewed and approved this PR.
I think may be merged soon.