[NV] minimax-m3-gb300-dynamo: nightly image + 2xDEP8 decode to 1xDEP16 / [NV] minimax-m3-gb300-dynamo：nightly 镜像 + 2×DEP8 解码合并为 1×DEP16

[xinli-sw]

Summary

GB300 disaggregated (minimaxm3-fp8-gb300-dynamo-vllm) — same treatment as the GB200 PR:

• Bump image to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 and dynamo to 1.3.0.dev20260710
• Drop the minimax-m3-gb300-vllm-fixes.sh setup patch and its launcher wiring — the fix ships in the nightly image
• Decode topology: the one 2×DEP8 decode arm (3p2d, conc 512) collapses into a single DEP16 worker — decode_workers 2→1, gpus_per_decode 8→16, data-parallel-size 8→16 (decode still 4 nodes / 16 GPU). Recipe renamed 3p2d-dep2-dep8 → 3p1d-dep2-dep16. The remaining decodes are 1×DEP8 or TEP and are unchanged.

中文说明

GB300 分离式推理配置（minimaxm3-fp8-gb300-dynamo-vllm）——采用与 GB200 PR 相同的调整：

• 将镜像升级至 vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9，并将 Dynamo 升级至 1.3.0.dev20260710
• 移除 minimax-m3-gb300-vllm-fixes.sh setup patch 及其启动器接线——相关修复已包含在 nightly 镜像中
• 解码拓扑： 原有的一个 2×DEP8 解码分支（3p2d，并发度 512）合并为单个 DEP16 worker——decode_workers 2→1、gpus_per_decode 8→16、data-parallel-size 8→16（解码侧仍使用 4 个节点 / 16 张 GPU）。配方由 3p2d-dep2-dep8 重命名为 3p1d-dep2-dep16。其余解码分支仍为 1×DEP8 或 TEP，保持不变。

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

Beyond the inline nit, I also checked whether dropping minimax-m3-gb300-vllm-fixes.sh (the FlashInfer all-reduce force_oneshot_support patch and the MSA sparse_attention_msa.py .contiguous() workaround) removes a fix still needed at runtime — ruled out, since this PR bumps to a newer nightly vLLM image that the PR description states now ships both patches upstream.

Extended reasoning...

configs/nvidia-master.yaml is CODEOWNER-owned (per .github/CODEOWNERS: @Ankur-singh @kedarpotdar-nv @InferenceX/core), so per policy this isn't auto-approved regardless of how mechanical the change looks. Beyond the inline nit finding, I read the diff for the vllm-fixes.sh removal (the launcher's --setup-script wiring in runners/launch_gb300-nv.sh is cleanly removed alongside it, and no other recipe references the script) and confirmed there's no orphaned reference or now-broken code path.

[claude]

🟡 In benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/3p1d-dep2-dep16-8k1k.yaml, the descriptive comment on line 3 still says '2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each)', describing the pre-PR topology. This PR collapsed decode to a single DEP16 worker (decode_workers 2->1, gpus_per_decode 8->16, data-parallel-size 8->16) and correctly updated the name: field and the matching comment in configs/nvidia-master.yaml, but missed this in-file comment. It's a documentation-only inconsistency with no runtime effect — worth a quick fix before merge for clarity.

Extended reasoning...

What's wrong: Line 3 of 3p1d-dep2-dep16-8k1k.yaml (renamed from 3p2d-dep2-dep8-8k1k.yaml) reads:

# 3P DEP2 prefill (TP1 DP2 EP, 2 GPU/worker) + 2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each).

This describes the pre-PR decode topology: two DEP8 decode workers, each spanning 2 nodes (2 x 8 GPU = 16 GPU total across 2 workers). That topology no longer exists in this file.

What actually changed: The PR's decode-collapse diff for this file is:

   prefill_workers: 3
-  decode_workers: 2
+  decode_workers: 1
   gpus_per_prefill: 2
-  gpus_per_decode: 8
+  gpus_per_decode: 16
   ...
-      data-parallel-size: 8
+      data-parallel-size: 16

So post-PR the decode side is a single DEP16 worker (TP1 DP16 EP, 16 GPU spanning 4 nodes) — not 2D DEP8. The comment on line 3 is now self-contradictory with the resources: block a few lines below it (decode_workers: 1, gpus_per_decode: 16) and with the vllm_config.decode block (data-parallel-size: 16).

Why this wasn't caught elsewhere: the author clearly intended to keep documentation in sync — they updated the name: field from ...3p2d-dep2-dep8... to ...3p1d-dep2-dep16..., and they updated the analogous comment in configs/nvidia-master.yaml from # 3p2d DEP2+DEP8, 6n to # 3p1d DEP2+DEP16, 6n. Both of those sibling annotations were fixed, which is strong evidence this in-file comment on line 3 was simply missed rather than intentionally left as-is.

Proof walkthrough:

1. Pre-PR: decode_workers: 2, gpus_per_decode: 8 → 2 workers x 8 GPU = 16 GPU, each worker = 8/4 = 2 nodes. Comment ('2D DEP8 ... 2 nodes each') matches.
2. Post-PR diff: decode_workers 2->1, gpus_per_decode 8->16, data-parallel-size 8->16.
3. Post-PR: 1 worker x 16 GPU = 16 GPU total, spanning 16/4 = 4 nodes for that single worker.
4. Line 3 comment still says '2D DEP8 decode ... 8 GPU/worker = 2 nodes each' — describing state (1), not the actual state (3).
5. Line 5 ('Nodes: 2 prefill + 4 decode') is unaffected and remains correct, since total decode node count (4) didn't change — only the fix on line 3 is needed.

Impact: This is a comment-only defect — the executable resources: and vllm_config: blocks are correct, so the recipe runs with the intended 1xDEP16 topology regardless. The only harm is to a future reader (or another automation adapting this recipe) who trusts the comment over the config and misunderstands the decode topology.

Fix: Update line 3 to match the pattern already used in configs/nvidia-master.yaml, e.g.:

# 3P DEP2 prefill (TP1 DP2 EP, 2 GPU/worker) + 1D DEP16 decode (TP1 DP16 EP, 16 GPU/worker = 4 nodes).

Severity is nit since it has zero effect on execution or benchmark results — it is purely a stale explanatory comment.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30208366514
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30208366514

[xinli-sw]

/reuse-sweep-run

[Ankur-singh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Run Sweep 30208366514
• Verified that this PR passes evals. Run Sweep 30208366514
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.

Additional detail section:

• This is a multi-node, disaggregated STP submission, not a speculative-decoding or agentic workload. Those checklist items are not applicable.
• Exact-head Run Sweep 30208366514 passed all 9 applicable throughput jobs and 8 applicable eval jobs. GSM8K scores ranged from 0.9538 to 0.9598 against the 0.90 threshold, using vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9.
• The general-code-quality item is intentionally unchecked. Accepted nonfunctional caveats: router.version metadata remains 1.3.0.dev20260614 while the recipes and runtime install Dynamo 1.3.0.dev20260710; and the renamed DEP16 recipe retains a stale explanatory comment describing the previous 2×DEP8 topology. The executable topology and tested runtime are correct.
• The changed entry is the existing Dynamo-orchestrated upstream-vLLM disaggregated submission, not a newly introduced TRT-LLM, ATOM, or other additional-engine entry.
• The single-node recipe requirement does not apply. Informational upstream recipe work: vLLM Recipes PR #687 is open and explicitly covers MiniMax-M3 disaggregated serving on B200/B300/GB200/GB300 and cites this InferenceX PR.
• This PR removes the prior serving-stack patch and launcher wiring; the exact-head run used the upstream image with SRTCTL_SETUP_SCRIPT empty.
• The existing authorized bare /reuse-sweep-run applies to the selected run.
• The PR remains CONFLICTING / DIRTY due to stale-main/perf-changelog cleanup. This approval does not resolve or misrepresent mergeability.
• The PR title and description are bilingual. The supported reuse merge path squash-merges, so the bilingual title supplies the landing commit subject.

Signed: Ankur-singh

[Klaud-Cold]

✅✅✅ Verdict: PASS ✅✅✅

✅ Check 0 (CODEOWNER): PASS — Ankur-singh is a listed owner of configs/nvidia-master.yaml; remaining paths fall under the catch-all.
✅ Check 1 (passing sweep on in-PR commit): PASS — head fa24e0b has green executed multi-node 8k1k / (9) and multi-node eval / (8) check-runs from run 30208366514; single-node lanes correctly skipped for this multi-node PR.
✅ Check 2 (evals pass): PASS — 8 GSM8K results in that run's eval artifact, 0.9538–0.9598 vs the 0.90 threshold (utils/evals/thresholds.yaml), run on this PR's image vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9, including the new DEP16 decode config.
➖ Check 3 (recipe link): N/A — disaggregated/multi-node submission (multinode: true, disagg: true, all recipes under benchmarks/multi_node/**); the recipe-link requirement applies to single-node recipes only.
✅ Check 4 (reuse command): PASS — authorized /reuse-sweep-run posted by xinli-sw (COLLABORATOR).
✅ Check 5 (latest checklist): PASS — all current-template items present; unchecked items (code quality, spec-decode, agentic, single-node recipe) are each explained in the additional detail section.
✅ Check 6 (upstream image / engine-first): PASS — image is upstream vllm/vllm-openai:nightly-...; no new non-vLLM/SGLang config entry added (existing dynamo-orchestrated upstream-vLLM entry modified).
✅ Check 7 (no architecture hacks): PASS — change is an image/dynamo bump plus a parallelism reshuffle (2×DEP8 → 1×DEP16); no --hf-overrides or FLOPs-reducing knobs.
➖ Check 8 (spec-decode chat templates): N/A — no speculative-decoding changes.
✅ Check 9 (no engine patches): PASS — the PR removes the prior inline patch (minimax-m3-gb300-vllm-fixes.sh) and its launcher wiring; the nightly image runs as shipped.
➖ Check 10 (agentic golden AL): N/A — no agentic speculative-decoding changes.

[Ankur-singh]

/reuse-sweep-run

[Klaud-Cold]

✅✅✅ Verdict: PASS ✅✅✅

✅ Check 0 (CODEOWNER): PASS — Ankur-singh is a listed owner of configs/nvidia-master.yaml; the remaining changed paths fall under the catch-all.
✅ Check 1 (passing sweep on in-PR commit): PASS — run 30208366514 executed 9 green multi-node 8k1k / and 8 green multi-node eval / jobs on in-PR commit fa24e0b0 (multi-node PR, so the single-node */ / eval / lanes are legitimately not applicable). Note: the sign-off calls this run "exact-head" — it ran on fa24e0b0, an in-PR commit two ahead of which the head 77fa834 sits; this still satisfies the in-PR requirement.
✅ Check 2 (evals pass): PASS — aggregated eval JSON has 8 GSM8K results, 0.9538–0.9598 vs the 0.90 default threshold (no minimaxm3 override in utils/evals/thresholds.yaml), all on vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 matching this PR's config, including the new 3P×1D DEP16 decode lane.
➖ Check 3 (recipe linked & merged): N/A — disaggregated/multi-node submission (all benchmark files under benchmarks/multi_node/srt-slurm-recipes/**, master entry multinode: true / disagg: true); the recipe-link requirement applies to single-node recipes only. The open vLLM recipes PR #687 is informational only.
✅ Check 4 (reuse command): PASS — authorized bare /reuse-sweep-run comments from xinli-sw and Ankur-singh (both COLLABORATOR).
✅ Check 5 (latest checklist template): PASS — all current-template items present; the unchecked items (code quality, spec-decode, agentic AL, single-node recipe) are each explained in the additional detail section.
✅ Check 6 (upstream image / engine-first): PASS — image moves to upstream vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9; the PR modifies the existing dynamo-orchestrated upstream-vLLM entry and adds no new non-vLLM/SGLang entry.
✅ Check 7 (no architecture hacks): PASS — changes are image/dynamo bumps and a decode-parallelism reshape (2×DEP8 → 1×DEP16); no --hf-overrides or FLOP-reducing knobs.
➖ Check 8 (spec-decode chat templates): N/A — no speculative-decoding changes (spec_decoding: none throughout).
✅ Check 9 (no engine patches): PASS — the PR removes the prior inline engine patch (minimax-m3-gb300-vllm-fixes.sh heredoc) and its launcher wiring; no patching remains.
➖ Check 10 (agentic golden AL): N/A — no agentic speculative-decoding changes.