[NV] minimax-m3-b200/b300-mtp: update fp8 MTP to vLLM nightly / [NV] minimax-m3-b200/b300-mtp：将 FP8 MTP 更新至 vLLM nightly - #2337
Conversation
…ASH_ATTN, stream-interval 32 Single-node MTP (minimaxm3-fp8-b200-vllm-mtp, minimaxm3-fp8-b300-vllm-mtp): - image -> vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 - EAGLE3 draft head -> Inferact/MiniMax-M3-EAGLE3-GQA - speculative attention_backend TRITON_ATTN -> FLASH_ATTN; num_speculative_tokens kept at 3 - consolidate --attention-config with indexer_kv_dtype fp8; --stream-interval 32 - drop the sparse_attention_msa contiguity patch (shipped in the nightly image)
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — this is a mechanical MTP recipe update (nightly image bump, EAGLE3-GQA draft head, TRITON_ATTN→FLASH_ATTN, consolidated --attention-config, dropped now-fixed contiguity patch) applied consistently to both the b200/b300 scripts and their config entries. The two findings are non-blocking nits (stale descriptive comments in nvidia-master.yaml, missing bilingual PR title/section) that don't affect runtime behavior.
Extended reasoning...
Overview
Touches two benchmark recipe scripts (minimaxm3_fp8_b200_mtp.sh, minimaxm3_fp8_b300_mtp.sh), the corresponding configs/nvidia-master.yaml entries, and perf-changelog.yaml. Changes: bump the vLLM image to a pinned nightly, rename the EAGLE3 draft head to the GQA variant, switch the speculative attention backend from TRITON_ATTN to FLASH_ATTN, consolidate two --attention-config flags into one JSON object, bump --stream-interval to 32, and delete an inline Python monkey-patch for a contiguity bug now fixed upstream in the nightly image.
Security risks
None. This is benchmark harness/config code with no auth, crypto, or user-facing surface — it only affects how a benchmark server is launched and reports perf numbers.
Level of scrutiny
Low-to-moderate is appropriate: this is CI benchmark infrastructure, not production-serving code, and the change follows an established pattern already used by sibling h200/h100 mtp recipes in the same file (same GQA/FLASH_ATTN combination). I checked that the --attention-config consolidation preserves the same effective keys as the previous split-flag form used in the non-mtp sibling scripts, and that the removed contiguity patch is justified by the PR description (fix now ships in the nightly image) rather than silently dropped.
Other factors
Both bug-hunter findings are nits: stale prose comments in configs/nvidia-master.yaml that were correctly updated in the .sh files but missed in the YAML, and a missing bilingual PR title/description per this repo's AGENTS.md convention. Neither affects the actual recipe behavior or benchmark correctness, so they don't block approval.
| # precedent: spec decode pays off at low/mid concurrency while acceptance | ||
| # dilutes in big batches, and the draft weights + draft KV shave headroom. | ||
| minimaxm3-fp8-b200-vllm-mtp: | ||
| image: vllm/vllm-openai:minimax-m3-0618-x86_64-cu130 | ||
| image: vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 | ||
| model: MiniMaxAI/MiniMax-M3-MXFP8 | ||
| model-prefix: minimaxm3 | ||
| runner: b200-dgxc |
There was a problem hiding this comment.
🟡 The config comments above minimaxm3-fp8-b200-vllm-mtp (~line 7458-7459) and minimaxm3-fp8-b300-vllm-mtp (~line 7513) still say the drafter is 'Inferact/MiniMax-M3-EAGLE3' pinned to 'TRITON_ATTN', but this PR switches both recipes to 'Inferact/MiniMax-M3-EAGLE3-GQA' with 'FLASH_ATTN'. Please update these two comment blocks to match, similar to how the h200/h100 mtp entries already describe the FLASH_ATTN/GQA rationale.
Extended reasoning...
configs/nvidia-master.yaml contains a hand-written comment block directly above each minimaxm3-fp8-*-vllm-mtp config entry explaining the EAGLE3 speculative-decoding setup for that recipe. For minimaxm3-fp8-b200-vllm-mtp the comment (around lines 7456-7459) reads: "...adds the Inferact/MiniMax-M3-EAGLE3 draft head... The target uses the FlashInfer TRT-LLM attention path. The EAGLE3 drafter is pinned separately to TRITON_ATTN." For minimaxm3-fp8-b300-vllm-mtp the comment (around lines 7511-7513) similarly references the plain Inferact/MiniMax-M3-EAGLE3 draft head with no attention-backend caveat.
This PR changes exactly what those comments describe: the draft head becomes Inferact/MiniMax-M3-EAGLE3-GQA and the speculative attention_backend moves from TRITON_ATTN to FLASH_ATTN (visible in the matching .sh recipe diffs, where the header comments were correctly updated: "Inferact/MiniMax-M3-EAGLE3-GQA draft head" and "The EAGLE3-GQA drafter is pinned separately to FLASH_ATTN."). However, the diff hunk touching configs/nvidia-master.yaml only replaces the image: line inside each block (lines 7461-7467 for b200, 7517-7523 for b300) — the descriptive comment lines sitting just above those keys were left untouched.
The result is that after this PR merges, the YAML comments for the b200 and b300 mtp entries misdescribe the actual recipe: they name the wrong draft-head repo (missing "-GQA") and, for b200, explicitly claim the drafter is "pinned to TRITON_ATTN" when it is now pinned to FLASH_ATTN. This is a real inconsistency the PR introduces/worsens, not a pre-existing one, since the PR's own script changes are what make the comments wrong. Contrast this with the sibling h200/h100 mtp entries further down in the same file (~lines 7539-7546 and 7565-7570), which already correctly explain the FLASH_ATTN/GQA rationale — those are the template this PR's two comment blocks should be brought in line with.
Step-by-step proof: (1) Before this PR, b200 mtp comment says draft head = Inferact/MiniMax-M3-EAGLE3, backend = TRITON_ATTN, matching the old script's DRAFT_MODEL="Inferact/MiniMax-M3-EAGLE3" and "attention_backend": "TRITON_ATTN". (2) This PR's diff to minimaxm3_fp8_b200_mtp.sh changes DRAFT_MODEL to "Inferact/MiniMax-M3-EAGLE3-GQA" and attention_backend to "FLASH_ATTN", and updates that script's own header comment to match. (3) The configs/nvidia-master.yaml diff for the same recipe only touches the image: line — the comment block above it is byte-for-byte identical to before. (4) Therefore post-merge, the YAML comment and the actual recipe behavior diverge: reading the config comment alone, a maintainer would believe the drafter still uses TRITON_ATTN with the non-GQA head, which is now false. The same divergence applies to the b300 comment (missing "-GQA" and no FLASH_ATTN mention).
This is comment-only and has zero effect on runtime behavior — the scripts themselves (the actual source of truth for how the benchmark runs) are correct. So while it's a genuine, PR-introduced inconsistency worth fixing to keep the config self-documenting, it doesn't block anything at merge. Fix: update the two comment blocks in configs/nvidia-master.yaml to say "Inferact/MiniMax-M3-EAGLE3-GQA draft head" and "pinned separately to FLASH_ATTN" (b200) / add the same GQA/FLASH_ATTN mention (b300), mirroring the phrasing already used in the h200/h100 mtp comments.
|
|
||
| - config-keys: | ||
| - minimaxm3-fp8-b200-vllm-mtp | ||
| - minimaxm3-fp8-b300-vllm-mtp | ||
| description: | ||
| - "Bump image to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9; set --stream-interval 32; consolidate --attention-config with indexer_kv_dtype fp8; drop the sparse_attention_msa contiguity patch (shipped in the nightly image)" |
There was a problem hiding this comment.
🟡 This PR's title and description are English-only, but AGENTS.md mandates bilingual PR titles/descriptions (title format <English title> / <中文标题> and a mirroring ## 中文说明 body section) for every PR. Please add the Chinese title suffix and a Chinese translation section before merging.
Extended reasoning...
AGENTS.md line 7 states, as a top-level mandatory rule (not subjective style): "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary...). This applies to every PR and every issue." CLAUDE.md forwards all repo guidance to AGENTS.md, so this rule governs review of this PR.
The violation: The PR title is [NV] minimax-m3-b200/b300-mtp: update fp8 MTP to vLLM nightly — English only, with no / <中文标题> suffix as the mandated format requires. The PR description's Summary section (bump image, EAGLE3-GQA draft head, FLASH_ATTN backend, consolidated --attention-config, dropped contiguity patch) has no mirroring ## 中文说明 section translating those same points into Simplified Chinese.
Why this isn't caught elsewhere: The only automated enforcement visible in the PR timeline is the codeowner-signoff-verify bot, which checks for the English sign-off phrase As a PR reviewer and CODEOWNER, I have reviewed this and have — it does not check PR title/body bilinguality. AGENTS.md's own exception list (CODEOWNER sign-off template stays English-verbatim; bot-generated comments follow their own templates) does not cover the human-authored title/description, so no carve-out applies here.
Step-by-step proof:
- Read AGENTS.md line 7: rule requires
<English title> / <中文标题>format and a## 中文说明mirror in the body, applying to "every PR." - Read the PR metadata: title =
[NV] minimax-m3-b200/b300-mtp: update fp8 MTP to vLLM nightly— no/separator, no Chinese characters anywhere. - Read the PR description: only a
## Summarysection in English with five bullet points; no## 中文说明or any Chinese text follows it. - Cross-check the exceptions in AGENTS.md (CODEOWNER sign-off verbatim text, bot templates) — neither applies to this human-authored title/body — so the rule is violated with no exemption.
Impact and fix: This is a metadata/process gap, not a code defect — nothing breaks at runtime, no benchmark or config behavior is affected, and the fix is simply editing the PR title and appending a ## 中文说明 section (following the translation quality bar in AGENTS.md, e.g. matching vllm-ascend's README.zh.md style) — no code changes or re-review of the diff are required. Given that, while it's a genuine, explicit repo requirement worth flagging, it doesn't rise to a blocking correctness issue.
…00-vllm-mtp-nightly
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30208331604 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30208331604 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30216094952 |
|
/reuse-sweep-run |
Ankur-singh
changed the title
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
❌❌❌ REJECTED ❌❌❌@Ankur-singh — blocking: this PR serves the ✅ Check 0 (CODEOWNER): PASS — |
3 participants
Summary
Single-node MTP (EAGLE3) MiniMax-M3 MXFP8 on B200/B300 (
minimaxm3-fp8-b200-vllm-mtp,minimaxm3-fp8-b300-vllm-mtp):vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9Inferact/MiniMax-M3-EAGLE3→Inferact/MiniMax-M3-EAGLE3-GQAattention_backendTRITON_ATTN→FLASH_ATTN;num_speculative_tokenskept at 3--attention-configwithindexer_kv_dtype: fp8; set--stream-interval 32sparse_attention_msacontiguity patch — the fix ships in the nightly image中文说明
在 B200/B300 上更新单节点 MTP（EAGLE3）MiniMax-M3 MXFP8 配置（
minimaxm3-fp8-b200-vllm-mtp、minimaxm3-fp8-b300-vllm-mtp）：vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9Inferact/MiniMax-M3-EAGLE3更新为Inferact/MiniMax-M3-EAGLE3-GQAattention_backend从TRITON_ATTN切换为FLASH_ATTN；num_speculative_tokens保持为 3--attention-config并设置indexer_kv_dtype: fp8；同时设置--stream-interval 32sparse_attention_msa内存连续性补丁——该修复已包含在 nightly 镜像中