MiniMax-M3 spec decoding Blackwell - #694
Conversation
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Code Review
This pull request introduces hardware overrides for the 'blackwell' architecture in the MiniMax-M3 model configuration, specifying speculative decoding and attention settings. The review feedback correctly identifies that the CLI arguments --attention_config.backend and --attention_config.use_trtllm_attention are non-standard in vLLM and may cause server startup failures, recommending standard alternatives.
| - "--attention_config.backend" | ||
| - "FLASHINFER" | ||
| - "--attention_config.use_trtllm_attention" | ||
| - "true" |
There was a problem hiding this comment.
The CLI arguments --attention_config.backend and --attention_config.use_trtllm_attention are not standard vLLM arguments and will likely cause the server to fail with an unrecognized arguments error.
--attention_config.backendshould be--attention-backend.--attention_config.use_trtllm_attentionis not a standard vLLM CLI argument. Please verify if this is a custom argument supported by your specific Docker image (vllm/vllm-openai:minimax-m3) or if it should be passed as an environment variable instead.
- "--attention-backend"
- "FLASHINFER"
- "--attention_config.use_trtllm_attention"
- "true"
Use Eagle3-GQA draft head and enable FLASHINFER + TRT-LLM attention for spec_decoding on Blackwell (B200/B300/GB200/GB300). Non-Blackwell NVIDIA stays on the standard Eagle3 head; AMD stays on TRITON_ATTN. Based on SemiAnalysisAI/InferenceX#2328 and SemiAnalysisAI/InferenceX#2337.