Skip to content

[NV] minimax-m3-gb300-dynamo: nightly image + 2xDEP8 decode to 1xDEP16 / [NV] minimax-m3-gb300-dynamo：nightly 镜像 + 2×DEP8 解码合并为 1×DEP16 #2340

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Ankur-singh merged 4 commits into from
Jul 29, 2026
+35 −70
Merged

[NV] minimax-m3-gb300-dynamo: nightly image + 2xDEP8 decode to 1xDEP16 / [NV] minimax-m3-gb300-dynamo：nightly 镜像 + 2×DEP8 解码合并为 1×DEP16 #2340

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c79991c
[NV] minimax-m3-gb300-dynamo: vLLM nightly + dynamo bump; collapse 2x…
xinli-sw Jul 26, 2026
5dbc379
[NV] minimax-m3-gb300-dynamo: add perf-changelog pr-link
xinli-sw Jul 26, 2026
fa24e0b
Merge remote-tracking branch 'origin/main' into minimaxm3-fp8-gb300-d…
xinli-sw Jul 26, 2026
77fa834
Merge remote-tracking branch 'origin/main' into pr-2340-reuse-11657
Ankur-singh Jul 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 0 additions & 38 deletions benchmarks/multi_node/srt-slurm-recipes/configs/minimax-m3-gb300-vllm-fixes.sh
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions ...arks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/1p1d-dep2-dep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb300-1p1d-dep2-dep8-fp8-8k1k"

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand Down
4 changes: 2 additions & 2 deletions ...arks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/1p1d-dep2-tep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb300-1p1d-dep2-tep8-fp8-8k1k"

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand Down
4 changes: 2 additions & 2 deletions ...arks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/1p2d-dep2-tep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb300-1p2d-dep2-tep8-fp8-8k1k"

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand Down
4 changes: 2 additions & 2 deletions ...arks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/2p1d-dep2-dep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb300-2p1d-dep2-dep8-fp8-8k1k"

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand Down
4 changes: 2 additions & 2 deletions ...arks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/2p2d-dep2-tep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb300-2p2d-dep2-tep8-fp8-8k1k"

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand Down
4 changes: 2 additions & 2 deletions ...arks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/2p4d-dep2-tep4-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb300-2p4d-dep2-tep4-fp8-8k1k"

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand Down
12 changes: 6 additions & 6 deletions ...3-gb300-fp8/8k1k/3p2d-dep2-dep8-8k1k.yaml → ...-gb300-fp8/8k1k/3p1d-dep2-dep16-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
name: "minimax-m3-vllm-disagg-gb300-3p2d-dep2-dep8-fp8-8k1k"
name: "minimax-m3-vllm-disagg-gb300-3p1d-dep2-dep16-fp8-8k1k"

# 3P DEP2 prefill (TP1 DP2 EP, 2 GPU/worker) + 2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each).

@claude claude Bot Jul 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 In benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/3p1d-dep2-dep16-8k1k.yaml, the descriptive comment on line 3 still says '2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each)', describing the pre-PR topology. This PR collapsed decode to a single DEP16 worker (decode_workers 2->1, gpus_per_decode 8->16, data-parallel-size 8->16) and correctly updated the name: field and the matching comment in configs/nvidia-master.yaml, but missed this in-file comment. It's a documentation-only inconsistency with no runtime effect — worth a quick fix before merge for clarity.

Extended reasoning...

What's wrong: Line 3 of 3p1d-dep2-dep16-8k1k.yaml (renamed from 3p2d-dep2-dep8-8k1k.yaml) reads:

# 3P DEP2 prefill (TP1 DP2 EP, 2 GPU/worker) + 2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each).

This describes the pre-PR decode topology: two DEP8 decode workers, each spanning 2 nodes (2 x 8 GPU = 16 GPU total across 2 workers). That topology no longer exists in this file.

What actually changed: The PR's decode-collapse diff for this file is:

   prefill_workers: 3
-  decode_workers: 2
+  decode_workers: 1
   gpus_per_prefill: 2
-  gpus_per_decode: 8
+  gpus_per_decode: 16
   ...
-      data-parallel-size: 8
+      data-parallel-size: 16

So post-PR the decode side is a single DEP16 worker (TP1 DP16 EP, 16 GPU spanning 4 nodes) — not 2D DEP8. The comment on line 3 is now self-contradictory with the resources: block a few lines below it (decode_workers: 1, gpus_per_decode: 16) and with the vllm_config.decode block (data-parallel-size: 16).

Why this wasn't caught elsewhere: the author clearly intended to keep documentation in sync — they updated the name: field from ...3p2d-dep2-dep8... to ...3p1d-dep2-dep16..., and they updated the analogous comment in configs/nvidia-master.yaml from # 3p2d DEP2+DEP8, 6n to # 3p1d DEP2+DEP16, 6n. Both of those sibling annotations were fixed, which is strong evidence this in-file comment on line 3 was simply missed rather than intentionally left as-is.

Proof walkthrough:

  1. Pre-PR: decode_workers: 2, gpus_per_decode: 8 → 2 workers x 8 GPU = 16 GPU, each worker = 8/4 = 2 nodes. Comment ('2D DEP8 ... 2 nodes each') matches.
  2. Post-PR diff: decode_workers 2->1, gpus_per_decode 8->16, data-parallel-size 8->16.
  3. Post-PR: 1 worker x 16 GPU = 16 GPU total, spanning 16/4 = 4 nodes for that single worker.
  4. Line 3 comment still says '2D DEP8 decode ... 8 GPU/worker = 2 nodes each' — describing state (1), not the actual state (3).
  5. Line 5 ('Nodes: 2 prefill + 4 decode') is unaffected and remains correct, since total decode node count (4) didn't change — only the fix on line 3 is needed.

Impact: This is a comment-only defect — the executable resources: and vllm_config: blocks are correct, so the recipe runs with the intended 1xDEP16 topology regardless. The only harm is to a future reader (or another automation adapting this recipe) who trusts the comment over the config and misunderstands the decode topology.

Fix: Update line 3 to match the pattern already used in configs/nvidia-master.yaml, e.g.:

# 3P DEP2 prefill (TP1 DP2 EP, 2 GPU/worker) + 1D DEP16 decode (TP1 DP16 EP, 16 GPU/worker = 4 nodes).

Severity is nit since it has zero effect on execution or benchmark results — it is purely a stale explanatory comment.

# GB300 has 4 GPUs/node. Adapted from NV B300 PR #1863.
# Nodes: 2 prefill + 4 decode (+ head/infra).

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand All @@ -31,9 +31,9 @@ resources:
prefill_nodes: 2
decode_nodes: 4
prefill_workers: 3
decode_workers: 2
decode_workers: 1
gpus_per_prefill: 2
gpus_per_decode: 8
gpus_per_decode: 16

frontend:
type: dynamo
Expand Down Expand Up @@ -84,7 +84,7 @@ backend:
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
tensor-parallel-size: 1
data-parallel-size: 8
data-parallel-size: 16
data-parallel-rpc-port: 13345
enable-expert-parallel: true
trust-remote-code: true
Expand Down
4 changes: 2 additions & 2 deletions ...arks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/3p1d-dep2-dep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb300-3p1d-dep2-dep8-fp8-8k1k"

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand Down
4 changes: 2 additions & 2 deletions ...arks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8/8k1k/6p1d-dep2-dep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb300-6p1d-dep2-dep8-fp8-8k1k"

model:
path: "minimax-m3-mxfp8"
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"


dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

health_check:
max_attempts: 720
Expand Down
12 changes: 6 additions & 6 deletions configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7289,7 +7289,7 @@ minimaxm3-fp4-b300-dynamo-vllm-mtp-legacy-dep4:
# All prefill DEP2 (TP1 DP2 EP, 2 GPU/worker). Decode: DEP4, TEP8, DEP8, TEP4.
# 4 GPU/node (GB300 NVL72). kv-cache-dtype=fp8. srun_options mem=0 required.
minimaxm3-fp8-gb300-dynamo-vllm:
image: vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223
image: vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9
model: MiniMaxAI/MiniMax-M3-MXFP8
model-prefix: minimaxm3
runner: gb300-nv
Expand Down Expand Up @@ -7409,19 +7409,19 @@ minimaxm3-fp8-gb300-dynamo-vllm:
ep: 8
dp-attn: true

# 3p2d DEP2+DEP8, 6n: conc 512
# 3p1d DEP2+DEP16, 6n: conc 512
- conc-list: [512]
prefill:
num-worker: 3
tp: 2
ep: 2
dp-attn: true
additional-settings:
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb300-fp8/8k1k/3p2d-dep2-dep8-8k1k.yaml"
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb300-fp8/8k1k/3p1d-dep2-dep16-8k1k.yaml"
decode:
num-worker: 2
tp: 8
ep: 8
num-worker: 1
tp: 16
ep: 16
dp-attn: true

# 6p1d DEP2+DEP8, 5n: conc 2048
Expand Down
7 changes: 7 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5223,3 +5223,10 @@
- "Bump image to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 and dynamo to 1.3.0.dev20260710; drop the minimax-m3-gb200-vllm-fixes.sh setup patch (shipped in the nightly image)"
- "Collapse every 2xDEP8 decode into a single DEP16 worker (decode_workers 2->1, gpus_per_decode 8->16, data-parallel-size 8->16); recipes renamed Xp2d-dep4-dep8 -> Xp1d-dep4-dep16 (1p2d/2p2d/3p2d/5p2d)"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2339

- config-keys:
- minimaxm3-fp8-gb300-dynamo-vllm
description:
- "Bump image to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 and dynamo to 1.3.0.dev20260710; drop the minimax-m3-gb300-vllm-fixes.sh setup patch (shipped in the nightly image)"
- "Collapse the 2xDEP8 decode into a single DEP16 worker (decode_workers 2->1, gpus_per_decode 8->16, data-parallel-size 8->16); recipe renamed 3p2d-dep2-dep8 -> 3p1d-dep2-dep16 (conc 512). The remaining decodes are 1xDEP8 or TEP and are unchanged."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2340
4 changes: 0 additions & 4 deletions runners/launch_gb300-nv.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,10 +212,6 @@ elif [[ $FRAMEWORK == "dynamo-vllm" && $MODEL_PREFIX == "minimaxm3" ]]; then
git checkout sa-submission-q2-2026
mkdir -p recipes/vllm/minimax-m3-gb300-fp8
cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb300-fp8" recipes/vllm/minimax-m3-gb300-fp8
SRTCTL_SETUP_SCRIPT="minimax-m3-gb300-vllm-fixes.sh"
cp \
"$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/configs/$SRTCTL_SETUP_SCRIPT" \
"configs/$SRTCTL_SETUP_SCRIPT"
elif [[ $FRAMEWORK == "dynamo-vllm" && $MODEL_PREFIX == "kimik2.5" && $PRECISION == "fp4" ]]; then
git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR"
cd "$SRT_REPO_DIR"
Expand Down