Minimax M3 agg

[faradawn]

Set the MSA indexer KV cache to FP8 on the MiniMax-M3 NVFP4 and MXFP8 variants for B200/B300 serving. Based on SemiAnalysisAI/InferenceX#2120 and SemiAnalysisAI/InferenceX#2337.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
vllm-recipes	Ready	Preview, Comment	Jul 27, 2026 10:35pm

[gemini-code-assist]

Code Review

This pull request updates the speculative decoding configuration for the MiniMax-M3 model in models/MiniMaxAI/MiniMax-M3.yaml. It restructures the spec_decoding feature to support multiple modes, introducing a standard mode for single-node and multi-node serving, and a new gqa mode recommended for disaggregated serving. Both modes include hardware overrides for AMD to use the Triton attention backend. No review comments were provided, so there is no additional feedback.

[Ankur-singh]

Thanks for adding --attention_config.indexer_kv_dtype fp8 to the nvfp4 variant — that matches what we run on the single-node NVFP4 lanes exactly.

One suggestion while this is open, since it also adds the GQA draft head "recommended for disaggregated (pd_cluster) serving": our disaggregated NVFP4 recipes set two more keys in the same attention config that the current extra_args doesn't cover:

{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}

So the split we see in practice is:

Deployment	attention config
Single-node NVFP4 (B200/B300)	indexer_kv_dtype: fp8 only — i.e. exactly what this PR adds
Disaggregated P/D (pd_cluster, B200 + B300)	additionally backend: FLASHINFER + use_trtllm_attention: true

If that reflects your intent for the pd_cluster path too, it might be worth folding the two extra keys in here (dotted form, matching your style):

      - "--attention_config.backend"
      - "FLASHINFER"
      - "--attention_config.use_trtllm_attention"
      - "true"

Possibly gated to the disaggregated strategy rather than set unconditionally on the variant, since the single-node lanes don't pass them.

Evidence — these are the InferenceX configs the values come from (all three still in review, not yet merged, so treat as FYI rather than settled):

• B200 disagg: [NV] feat(minimaxm3): add B200 disaggregated configs / [NV] feat(minimaxm3)：新增 B200 分离式配置 SemiAnalysisAI/InferenceX#2230 (minimaxm3-fp4-b200-dynamo-vllm) — green sweep, gsm8k 0.948–0.953 across six concurrency points
• B300 disagg: [NV] Refresh MiniMax-M3 NVFP4 B300 8k1k disaggregated Dynamo-vLLM / 更新 MiniMax-M3 NVFP4 B300 8k1k 分离式 Dynamo-vLLM 配置 SemiAnalysisAI/InferenceX#2310 (minimaxm3-fp4-b300-dynamo-vllm-8k1k-tp1)
• Single-node B200/B300: Update vLLM MiniMax-M3 image tags / 更新 vLLM MiniMax-M3 镜像标签 SemiAnalysisAI/InferenceX#2120 — uses --attention_config.indexer_kv_dtype fp8 only, consistent with this PR

Happy to open a follow-up PR with just those two keys after this merges, if you'd rather keep this one scoped to the GQA/indexer change.

[Ankur-singh]

Thanks — the current head 20fa5198b5c46eaf05be8da5fe60f43ecf1cd29f now faithfully includes the non-speculative MiniMax-M3 NVFP4 B300 pd_cluster server overlap from SemiAnalysisAI/InferenceX#2310: indexer_kv_dtype=fp8 plus the Blackwell-only FLASHINFER / use_trtllm_attention=true overrides. The GQA draft-mode addition remains correctly sourced from SemiAnalysisAI/InferenceX#2314. This does not claim that the recipe UI reproduces the exact #2310 4xTP1-prefill + 1xTEP4-decode colocated topology; that benchmark topology remains outside the current command-builder shape.

I reproduced the Vercel failure. The PR is still based on 5c3b63b69d3d232c81892376b6d86031bf7246b0, so node scripts/build-recipes-api.mjs fails on the unrelated GLM-5.2 MXFP4 mode validation that current main already fixes. Rebasing these four commits onto current main ae7c7b1a43b190ea40bfc87bcf8b2fc4e8454fec is clean; on that rebased tree both node scripts/build-recipes-api.mjs and the full pnpm build pass. Please rebase/update the branch; no additional MiniMax recipe changes are needed for this failure.

中文：当前提交 20fa5198b5c46eaf05be8da5fe60f43ecf1cd29f 已准确覆盖 SemiAnalysisAI/InferenceX#2310 中非投机 MiniMax-M3 NVFP4 B300 pd_cluster 服务端的共同配置：indexer_kv_dtype=fp8，以及仅用于 Blackwell 的 FLASHINFER / use_trtllm_attention=true 覆盖。GQA 草稿模式仍正确来源于 SemiAnalysisAI/InferenceX#2314。这里不声称 recipe UI 能复现 #2310 的 4 个 TP1 预填充 worker 加 1 个 TEP4 解码 worker 的同节点拓扑；该基准测试拓扑目前仍超出命令生成器的表达范围。

我已复现 Vercel 失败。该 PR 仍基于 5c3b63b69d3d232c81892376b6d86031bf7246b0，因此 node scripts/build-recipes-api.mjs 会在与本 PR 无关的 GLM-5.2 MXFP4 模式校验处失败，而当前 main 已修复此问题。将这 4 个提交干净地 rebase 到当前 main ae7c7b1a43b190ea40bfc87bcf8b2fc4e8454fec 后，node scripts/build-recipes-api.mjs 和完整的 pnpm build 均通过。请更新分支；此次失败不需要额外修改 MiniMax recipe。

[Ankur-singh]

Validated this as the existing upstream artifact for SemiAnalysisAI/InferenceX#2120; no duplicate PR is needed.

• Source PR current head: d9924c6e8295cdb2f3d4e36b9612691446869e22.
• Supporting sweep: SemiAnalysisAI/InferenceX Actions run 29984401390, attempt 1, head 0b896e92f975d0ef8e374fdc284b06444fc95242 (green). The four MiniMax-M3 config blocks and four launcher scripts are byte-identical between that sweep SHA and the current source head.
• Target vllm-project/recipes main captured for validation: ae7c7b1a43b190ea40bfc87bcf8b2fc4e8454fec; PR head 20fa5198b5c46eaf05be8da5fe60f43ecf1cd29f merges cleanly.
• node scripts/build-recipes-api.mjs and the full pnpm build both pass on that current-main merge result. The promoted nvidia/MiniMax-M3-NVFP4 API has the correct model/precision metadata, VLLM_FLOAT32_MATMUL_PRECISION=high, --block-size 128, --trust-remote-code, and --attention_config.indexer_kv_dtype fp8. All four commits carry valid Signed-off-by trailers and DCO is green.

One command-parity issue remains before I can call every exposed combination supported: the exact non-EAGLE B200/B300 command normalizes cleanly, but the generated EAGLE3 command also inherits --attention_config.indexer_kv_dtype fp8 from variants.nvfp4.extra_args. Both validated InferenceX MTP launchers omit that flag; their only indexer dtype is the vLLM default. Please either (a) point to supporting runtime evidence for EAGLE3 + FP8 indexer cache, or (b) keep the FP8 indexer setting scoped to the non-EAGLE reproduction path instead of exposing the unverified cross-product.

Two metadata/CI finish items:

1. Please add Update vLLM MiniMax-M3 image tags / 更新 vLLM MiniMax-M3 镜像标签 SemiAnalysisAI/InferenceX#2120 and the single-node FP8 indexer-cache purpose to the PR body; it currently cites only #2314.
2. The Vercel failure is reproducible on the exact PR head: node scripts/build-recipes-api.mjs stops on the unrelated models/zai-org/GLM-5.2.yaml MXFP4 mode validation because this branch is still based on 5c3b63b69d3d232c81892376b6d86031bf7246b0. Current main already contains the fix. Please merge or rebase current main into the PR branch and push; the clean current-main merge result passes both the API generator and the full production build.

Once the command-scope question is resolved and the refreshed branch is green, this existing PR is the correct artifact to merge.

中文：

已确认本 PR 就是 SemiAnalysisAI/InferenceX#2120 对应的上游产物，无需再创建重复 PR。

• 源 PR 当前 head 为 d9924c6e8295cdb2f3d4e36b9612691446869e22。
• 支撑证据为 InferenceX Actions 运行 29984401390（attempt 1，head 0b896e92f975d0ef8e374fdc284b06444fc95242，已通过）。该扫描 SHA 与当前源 PR head 之间，四个 MiniMax-M3 配置块和四个启动脚本的内容完全一致。
• 本次验证所基于的 vllm-project/recipes main 为 ae7c7b1a43b190ea40bfc87bcf8b2fc4e8454fec；PR head 20fa5198b5c46eaf05be8da5fe60f43ecf1cd29f 可无冲突合并。
• 在上述基于当前 main 的合并结果上，node scripts/build-recipes-api.mjs 与完整的 pnpm build 均通过。生成的 nvidia/MiniMax-M3-NVFP4 API 包含正确的模型/精度元数据、VLLM_FLOAT32_MATMUL_PRECISION=high、--block-size 128、--trust-remote-code 和 --attention_config.indexer_kv_dtype fp8。四个提交均带有有效的 Signed-off-by，DCO 检查为绿色。

目前仍有一个命令等价性问题，解决前不能把所有暴露出的组合都视为已验证支持：非 EAGLE 的 B200/B300 命令可完成规范化对齐，但生成的 EAGLE3 命令也会从 variants.nvfp4.extra_args 继承 --attention_config.indexer_kv_dtype fp8。两份已验证的 InferenceX MTP 启动脚本都没有传入该参数，而是使用 vLLM 默认的 indexer dtype。请任选其一处理：(a) 提供 EAGLE3 + FP8 indexer cache 的运行证据；或 (b) 将 FP8 indexer 设置限制在非 EAGLE 的复现路径中，避免暴露尚未验证的组合。

还需要完成两项元数据/CI 工作：

1. 请在 PR 正文中补充 Update vLLM MiniMax-M3 image tags / 更新 vLLM MiniMax-M3 镜像标签 SemiAnalysisAI/InferenceX#2120 以及单节点 FP8 indexer cache 的用途；当前正文只引用了 #2314。
2. 已在 PR 的精确 head 上复现 Vercel 失败：由于该分支仍基于 5c3b63b69d3d232c81892376b6d86031bf7246b0，node scripts/build-recipes-api.mjs 会在与本 PR 无关的 models/zai-org/GLM-5.2.yaml MXFP4 模式校验处停止；当前 main 已包含修复。请将当前 main 合并或 rebase 到 PR 分支后推送。基于当前 main 的干净合并结果已同时通过 API 生成器和完整生产构建。

命令适用范围确认完成且更新后的分支变绿后，这个现有 PR 就是应当合并的正确产物。

[Ankur-singh]

Revalidated the new head 8014fd5c046dbebd0190a3f8653431315a0deef5: node scripts/build-recipes-api.mjs and the full pnpm build pass locally, the base and promoted MiniMax-M3 NVFP4 APIs render, normalized B300 pd_cluster argv contains the intended Blackwell attention flags, all five commits have valid Signed-off-by trailers, and the live checks are green.

One correction to my prior command-scope note: the exact InferenceX #2314 disaggregated GQA recipes do pass indexer_kv_dtype: fp8 in attention-config for both roles, so EAGLE3 GQA + FP8 indexer cache has matching source evidence.

There is still one exact pd_cluster parity mismatch before the GQA mode can be described as runnable/recommended from #2314:

• InferenceX #2314 prefill: num_speculative_tokens: 1
• InferenceX #2314 decode: num_speculative_tokens: 3
• Generated recipe command at this head: num_speculative_tokens: 3 for both prefill and decode, because the shared feature mode is applied symmetrically to both roles.

I do not see source evidence here for the generated 3/3 cross-product. Please either provide runtime evidence for 3/3, add a faithful per-role mode representation, or keep the exact GQA pd_cluster combination guide-only rather than calling it recommended. The non-speculative #2310 server overlap remains valid and exact (indexer_kv_dtype=fp8 plus Blackwell FLASHINFER / use_trtllm_attention=true), but its 4xTP1-prefill + 1xTEP4-decode colocated topology is also guide-only under the current command-builder schema. Please add SemiAnalysisAI/InferenceX#2310 to the PR body for that non-speculative provenance; the GQA provenance should remain SemiAnalysisAI/InferenceX#2314.

中文：已重新验证新 head 8014fd5c046dbebd0190a3f8653431315a0deef5：node scripts/build-recipes-api.mjs 与完整的 pnpm build 均在本地通过；MiniMax-M3 基础 API 和 NVFP4 派生 API 可正常生成；规范化后的 B300 pd_cluster argv 包含预期的 Blackwell attention 参数；5 个提交均带有有效的 Signed-off-by；线上检查也已全部变绿。

更正此前关于命令适用范围的一点：InferenceX #2314 的精确分离式 GQA 配置确实在预填充和解码两侧都通过 attention-config 设置了 indexer_kv_dtype: fp8，因此 EAGLE3 GQA 与 FP8 indexer cache 的组合有对应源配置证据。

但在将 GQA 模式描述为可运行或推荐的 pd_cluster 组合前，仍存在一个精确等价性问题：InferenceX #2314 的预填充侧使用 num_speculative_tokens: 1，解码侧使用 num_speculative_tokens: 3；而当前 recipe 生成的命令会在预填充和解码两侧都使用 3，因为同一个 feature mode 会对两个角色对称生效。目前没有看到 3/3 组合的运行证据。请任选其一处理：提供 3/3 的运行证据、增加可准确表达角色差异的模式，或将精确的 GQA pd_cluster 组合保留为仅文档说明，不要标为推荐。

#2310 的非投机服务端共同配置仍然准确有效，即 indexer_kv_dtype=fp8 加上 Blackwell 的 FLASHINFER / use_trtllm_attention=true；但其 4 个 TP1 预填充 worker 加 1 个 TEP4 解码 worker 的同节点拓扑在当前命令生成器 schema 下同样只能作为说明。请在 PR 正文中补充 SemiAnalysisAI/InferenceX#2310，明确这部分非投机配置的来源；GQA 部分仍应归因于 SemiAnalysisAI/InferenceX#2314。