You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Send same prompt request again, output is never ending "!!!!!!!!!!"
Steps for suggested fix on AITER kernel level:
After launching the container, handle aiter change with sh aiter_fix.sh
Run a one-line hotfix (sorryy) for import as such: sed -i 's|from aiter.ops.triton.unified_attention import unified_attention|from aiter.ops.triton.hybrid_attention_integration import enhanced_unified_attention as unified_attention|g' /app/vllm-os-mini/vllm/v1/attention/backends/triton_attn.py
Try serving and sending requests again with this command: VLLM_FORCE_HYBRID_ATTENTION=1 VLLM_ROCM_USE_AITER=1 VLLM_USE_AITER_UNIFIED_ATTENTION=1 VLLM_ROCM_USE_AITER_MHA=0 vllm serve openai/gpt-oss-120b --compilation-config '{"full_cuda_graph": true}' &