Skip to content

[NV] minimax-m3-B300-dynamo: use vLLM nightly image / [NV] minimax-m3-B300-dynamo：使用 vLLM nightly 镜像 #2338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Ankur-singh merged 4 commits into from
Jul 29, 2026
Merged

[NV] minimax-m3-B300-dynamo: use vLLM nightly image / [NV] minimax-m3-B300-dynamo：使用 vLLM nightly 镜像 #2338

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d9abf7d
[NV] minimax-m3-b300-dynamo: vLLM nightly + dynamo bump; swap conc-40…
xinli-sw Jul 26, 2026
6ed7131
[NV] minimax-m3-b300-dynamo: add perf-changelog pr-link
xinli-sw Jul 26, 2026
666f505
Merge remote-tracking branch 'origin/main' into minimaxm3-fp8-b300-dy…
xinli-sw Jul 26, 2026
712f52c
Merge remote-tracking branch 'origin/main' into pr-2338-reuse-6403
Ankur-singh Jul 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 0 additions & 47 deletions benchmarks/multi_node/srt-slurm-recipes/configs/minimax-m3-vllm-fixes.sh
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions ...multi_node/srt-slurm-recipes/vllm/minimax-m3/b300-fp8/8k1k/1p1d-dep2-tp4-marlin-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "minimax-m3-vllm-disagg-b300-1p1d-fp8-dep2-tp4-marlin-8k1k"

model:
path: "MiniMaxAI/MiniMax-M3-MXFP8"
container: "vllm/vllm-openai:minimax-m3-0618-x86_64-cu130"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"

resources:
Expand All @@ -17,7 +17,7 @@ resources:

dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

frontend:
type: dynamo
Expand Down
4 changes: 2 additions & 2 deletions ...marks/multi_node/srt-slurm-recipes/vllm/minimax-m3/b300-fp8/8k1k/1p2d-dep2-tep4-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "minimax-m3-vllm-disagg-b300-1p2d-fp8-dep2-tep4-8k1k"

model:
path: "MiniMaxAI/MiniMax-M3-MXFP8"
container: "vllm/vllm-openai:minimax-m3-0618-x86_64-cu130"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"

resources:
Expand All @@ -17,7 +17,7 @@ resources:

dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

frontend:
type: dynamo
Expand Down
4 changes: 2 additions & 2 deletions ...marks/multi_node/srt-slurm-recipes/vllm/minimax-m3/b300-fp8/8k1k/1p4d-dep2-tep4-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "minimax-m3-vllm-disagg-b300-1p4d-fp8-dep2-tep4-8k1k"

model:
path: "MiniMaxAI/MiniMax-M3-MXFP8"
container: "vllm/vllm-openai:minimax-m3-0618-x86_64-cu130"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"

resources:
Expand All @@ -17,7 +17,7 @@ resources:

dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

frontend:
type: dynamo
Expand Down
4 changes: 2 additions & 2 deletions ...marks/multi_node/srt-slurm-recipes/vllm/minimax-m3/b300-fp8/8k1k/1p4d-dep2-tep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "minimax-m3-vllm-disagg-b300-1p4d-fp8-dep2-tep8-8k1k"

model:
path: "MiniMaxAI/MiniMax-M3-MXFP8"
container: "vllm/vllm-openai:minimax-m3-0618-x86_64-cu130"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"

resources:
Expand All @@ -17,7 +17,7 @@ resources:

dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

frontend:
type: dynamo
Expand Down
4 changes: 2 additions & 2 deletions ...marks/multi_node/srt-slurm-recipes/vllm/minimax-m3/b300-fp8/8k1k/2p2d-dep2-dep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "minimax-m3-vllm-disagg-b300-2p2d-fp8-dep2-dep8-8k1k"

model:
path: "MiniMaxAI/MiniMax-M3-MXFP8"
container: "vllm/vllm-openai:minimax-m3-0618-x86_64-cu130"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"

resources:
Expand All @@ -17,7 +17,7 @@ resources:

dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

frontend:
type: dynamo
Expand Down
4 changes: 2 additions & 2 deletions ...marks/multi_node/srt-slurm-recipes/vllm/minimax-m3/b300-fp8/8k1k/2p2d-dep2-tep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "minimax-m3-vllm-disagg-b300-2p2d-fp8-dep2-tep8-8k1k"

model:
path: "MiniMaxAI/MiniMax-M3-MXFP8"
container: "vllm/vllm-openai:minimax-m3-0618-x86_64-cu130"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"

resources:
Expand All @@ -17,7 +17,7 @@ resources:

dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

frontend:
type: dynamo
Expand Down
20 changes: 11 additions & 9 deletions ...m3/b300-fp8/8k1k/4p2d-dep2-tep4-8k1k.yaml → ...m3/b300-fp8/8k1k/3p2d-dep2-dep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
name: "minimax-m3-vllm-disagg-b300-4p2d-fp8-dep2-tep4-8k1k"
name: "minimax-m3-vllm-disagg-b300-3p2d-fp8-dep2-dep8-8k1k"

model:
path: "MiniMaxAI/MiniMax-M3-MXFP8"
container: "vllm/vllm-openai:minimax-m3-0618-x86_64-cu130"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"

resources:
gpu_type: "b300"
gpus_per_node: 8
prefill_nodes: 1
decode_nodes: 1
prefill_workers: 4
decode_nodes: 2
prefill_workers: 3
decode_workers: 2
gpus_per_prefill: 2
gpus_per_decode: 4
gpus_per_decode: 8

dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

frontend:
type: dynamo
Expand Down Expand Up @@ -55,7 +55,9 @@ backend:
max-num-batched-tokens: 16384

decode:
tensor-parallel-size: 4
tensor-parallel-size: 1
data-parallel-size: 8
data-parallel-rpc-port: 13345
enable-expert-parallel: true
trust-remote-code: true
no-enable-prefix-caching: true
Expand All @@ -67,7 +69,7 @@ backend:
max-model-len: 9472
language-model-only: true
stream-interval: 32
max-num-seqs: 1024
max-num-seqs: 1024 # Per DP rank: 2 workers x DP8 = 16 ranks.
max-num-batched-tokens: 16384
max-cudagraph-capture-size: 4096

Expand All @@ -79,5 +81,5 @@ benchmark:
type: "sa-bench"
isl: 8192
osl: 1024
concurrencies: "4096"
concurrencies: "1024x2048"
req_rate: "inf"
4 changes: 2 additions & 2 deletions ...marks/multi_node/srt-slurm-recipes/vllm/minimax-m3/b300-fp8/8k1k/4p2d-dep2-dep8-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "minimax-m3-vllm-disagg-b300-4p2d-fp8-dep2-dep8-8k1k"

model:
path: "MiniMaxAI/MiniMax-M3-MXFP8"
container: "vllm/vllm-openai:minimax-m3-0618-x86_64-cu130"
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
precision: "fp8"

resources:
Expand All @@ -17,7 +17,7 @@ resources:

dynamo:
install: true
version: 1.3.0.dev20260614
version: 1.3.0.dev20260710

frontend:
type: dynamo
Expand Down
14 changes: 7 additions & 7 deletions configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6885,7 +6885,7 @@ minimaxm3-fp4-b200-dynamo-vllm:
# baseline above. The image includes vLLM PR #46380, so no runtime patch is
# needed.
minimaxm3-fp8-b300-dynamo-vllm:
image: vllm/vllm-openai:minimax-m3-0618-x86_64-cu130
image: vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9
model: MiniMaxAI/MiniMax-M3-MXFP8
model-prefix: minimaxm3
runner: b300
Comment thread
Oseltamivir marked this conversation as resolved.
Expand Down Expand Up @@ -6952,19 +6952,19 @@ minimaxm3-fp8-b300-dynamo-vllm:
tp: 4
ep: 1
dp-attn: false
- conc-list: [4096]
- conc-list: [1024, 2048]
prefill:
num-worker: 4
num-worker: 3
tp: 2
ep: 2
dp-attn: true
additional-settings:
- "CONFIG_FILE=recipes/vllm/minimax-m3/b300-fp8/8k1k/4p2d-dep2-tep4-8k1k.yaml"
- "CONFIG_FILE=recipes/vllm/minimax-m3/b300-fp8/8k1k/3p2d-dep2-dep8-8k1k.yaml"
decode:
num-worker: 2
tp: 4
ep: 4
dp-attn: false
tp: 8
ep: 8
dp-attn: true
- conc-list: [16, 32, 64, 128]
prefill:
num-worker: 1
Expand Down
7 changes: 7 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5209,3 +5209,10 @@
- "Use configurations from concurrency 1 through 1024 with spread worker placement for CI transport validation."
- "Image: vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9; Dynamo: 1.3.0.dev20260710."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2230

- config-keys:
- minimaxm3-fp8-b300-dynamo-vllm
description:
- "Bump image to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 and dynamo to 1.3.0.dev20260710; drop the minimax-m3-vllm-fixes.sh setup patch (shipped in the nightly image)"
- "conc 4096: drop the 4p2d-dep2-tep4 arm; add a 3p2d-dep2-dep8 arm for conc [1024, 2048] (prefill 3xDEP2, decode 2xDEP8)"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2338
3 changes: 0 additions & 3 deletions runners/launch_b300-nv.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,6 @@ elif [[ $FRAMEWORK == "dynamo-vllm" && $MODEL_PREFIX == "minimaxm3" && ( $PRECIS
git checkout sa-submission-q2-2026
mkdir -p recipes/vllm/minimax-m3
cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3" recipes/vllm/minimax-m3
if [[ $PRECISION == "fp8" ]]; then
SRTCTL_SETUP_SCRIPT="minimax-m3-vllm-fixes.sh"
fi
# NVIDIA/srt-slurm#38
git show 22d46ba9971615016d2339c9ffbc7b4597accfad --format= -- src/srtctl/core/ip_utils/get_node_ip.sh | git apply - || exit 1
if [[ -n "$SRTCTL_SETUP_SCRIPT" ]]; then
Expand Down