feat(minimaxm3): refresh B300 AgentX Pareto points with EAGLE3-GQA / 使用 EAGLE3-GQA 刷新 MiniMax-M3 B300 AgentX 帕累托点 - #2610
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
2 similar comments
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31832767139 |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31832892782 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🟡
benchmarks/single_node/agentic/minimaxm3_fp4_b300_mtp.sh:46— sweep:TOTAL_CPU_DRAM_GB * 1000 * 1000 * 1000
The GB-to-bytes conversion is copy-pasted independently across 6 active scripts with no shared helper in benchmark_lib.sh, e.g. this file (line 46) and kimik3_fp4_b300_vllm.sh:106. Extracting a gb_to_bytes() helper into benchmark_lib.sh would make the unit conversion correct-by-construction everywhere and prevent the kind of 1024^3 drift that slipped into minimaxm3_fp4_b200_mtp.sh.Extended reasoning...
This finding is about duplication, not a functional bug in this PR: the corrected
TOTAL_CPU_DRAM_GB * 1000 * 1000 * 1000expression on line 46 is arithmetically right for decimal GB, matching the PR description's stated goal of using1000^3instead of1024^3.The problem is that this exact expression is hand-copied into at least six other active benchmark scripts (
kimik3_fp4_b300_vllm_mtp.sh:134,kimik3_fp4_mi355x_mtp.sh:136,kimik3_fp4_b300_vllm.sh:106,dsv4_fp4_b300_vllm_mtp.sh:131,dsv4_fp4_b200_vllm_mtp.sh:132), andbenchmarks/benchmark_lib.shonly validates thatTOTAL_CPU_DRAM_GBis a positive integer -- it never performs the GB-to-bytes conversion itself. Every one of those six sites has to get the1000vs1024base right independently, by eye, with no compiler or shared code to catch a mistake.That is exactly the failure mode that occurred on the sibling script
minimaxm3_fp4_b200_mtp.sh:131, which used1024 * 1024 * 1024instead of the decimal base and silently miscalculated the CPU KV-offload budget (see the sibling PR comment for that bug). Nothing in the current structure would have caught that divergence: there is no single source of truth for "how many bytes are inTOTAL_CPU_DRAM_GBgigabytes," so a copy-paste typo in the exponent base compiles, runs, and produces a plausible-looking (but wrong) byte count.Step-by-step proof of the risk this duplication creates:
- A developer adds a 7th script that also needs CPU KV-offload sizing and copies the pattern from one of the six existing sites.
- They fat-finger
1024instead of1000(or vice versa), which is a one-character-adjacent typo that is very easy to make and easy to miss in review, since both forms "look like a GB-to-bytes conversion." set -euo pipefaildoes not catch this: the arithmetic is well-formed,CPU_OFFLOAD_BYTESis a valid integer, and the script runs to completion.- The resulting
cpu_bytes_to_use/cpu_bytes_to_use_per_rankvalue silently under- or over-states the real CPU DRAM budget by ~7.4% (the ratio between1024^3and1000^3), which can cause the KV offload connector to be configured with the wrong capacity -- exactly what happened inminimaxm3_fp4_b200_mtp.sh:131. - This ships to CI/production runs and is only discovered by manual cross-script auditing (as happened here), not by any automated check.
If
benchmark_lib.shinstead exposed a singlegb_to_bytes()helper (e.g.gb_to_bytes() { echo $(( $1 * 1000 * 1000 * 1000 )); }), every call site would readCPU_OFFLOAD_BYTES=$(gb_to_bytes "$TOTAL_CPU_DRAM_GB")and the unit base would live in exactly one place. Note the six sites are not byte-identical downstream -- some divide byTP, others byGPU_COUNT, and they populate different JSON keys (cpu_bytes_to_usevscpu_bytes_to_use_per_rank) -- so the helper should only own the shared multiplier, with each caller applying its own per-rank division afterward. That is still a meaningful DRY win and is consistent with how these scripts already lean on shared helpers frombenchmark_lib.sh(require_agentic_kv_offload_backend,install_agentic_deps,wait_for_server_ready).This is a pure code-quality/reuse suggestion -- it does not change behavior in this PR and should not block merging, but fixing it would remove the exact structural gap that allowed the
1024^3bug to happen undetected in the first place. -
🟣
benchmarks/single_node/agentic/minimaxm3_fp4_b300_mtp.sh:46— Pre-existing: the same decimal-GB fix applied here (* 1000 * 1000 * 1000) is not mirrored in the b200 sibling script,benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.sh:131, which still computesCPU_OFFLOAD_BYTESusing binary GiB (* 1024 * 1024 * 1024) for the identical dram-utilization:0.683/tp:4/kv-offloading:dram point. That script's own comment now falsely claims b200 and b300 resolve to the same 1,024 GB budget, when b200 actually requests ~1,099.5 GB (a 7.37% overshoot).Extended reasoning...
This PR fixes
benchmarks/single_node/agentic/minimaxm3_fp4_b300_mtp.sh:46to computeCPU_OFFLOAD_BYTES = TOTAL_CPU_DRAM_GB * 1000 * 1000 * 1000(decimal GB), which correctly round-trips the value thatgenerate_sweep_configs.pyderives fromdram-utilization. That generator'sBYTES_PER_GBconstant is1_000_000_000(decimal), andTOTAL_CPU_DRAM_GBis computed asint(proportional_bytes / BYTES_PER_GB)— so any script that reconstructs bytes fromTOTAL_CPU_DRAM_GBmust multiply back by the same decimal1000^3, not the binary1024^3, or the resultingcpu_bytes_to_usewon't match what the generator intended.The b200 sibling script,
benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.sh:131, has the exact same line shape but was never updated:CPU_OFFLOAD_BYTES=$((TOTAL_CPU_DRAM_GB * 1024 * 1024 * 1024)). This script backs theminimaxm3-fp4-b200-vllm-agentic-mtpconfig (configs/nvidia-master.yaml~line 7452-7470), which shares the identicaldram-utilization: 0.683/tp: 4/kv-offloading: dramsearch point as the b300 config this PR touches. The comment directly above that config block explicitly asserts "0.683 maps ... to a 1,024 GB engine-level CPU KV budget" for both SKUs — but with the binary multiplier, b200 actually computes1024 * 1024^3 = 1,099,511,627,776bytes (~1,099.5 GB) instead of the intended1,024,000,000,000bytes, a 7.37% overshoot. This PR just proved that comment false by fixing the root cause in one file and leaving it in the other.The reason nothing else in the codebase catches this: every other active offload script (
kimik3_fp4_b300_vllm_mtp.sh,kimik3_fp4_mi355x_mtp.sh,kimik3_fp4_b300_vllm.sh,dsv4_fp4_b300_vllm_mtp.sh,dsv4_fp4_b200_vllm_mtp.sh) already uses the correct decimal1000^3, and nowminimaxm3_fp4_b300_mtp.shjoins them after this PR's fix. The b200 minimaxm3 script is the lone active straggler — there's no shared helper computing this byte conversion, so each script duplicates the arithmetic independently and one was simply missed when the family was fixed elsewhere (or predates the fix pattern being adopted).Concretely: take the documented 1,024 GB budget point at dram-utilization 0.683/tp:4. The generator computes
TOTAL_CPU_DRAM_GB = 1024(decimal GB, per its ownBYTES_PER_GB=1_000_000_000). The b300 script now correctly reconstructs1024 * 1000^3 = 1,024,000,000,000bytes and passes that ascpu_bytes_to_useto vLLM'sSimpleCPUOffloadConnector. The b200 script instead reconstructs1024 * 1024^3 = 1,099,511,627,776bytes — vLLM is told it can use ~75.5 GB more host DRAM for CPU KV offload than the generator's proportional-GPU cap intended, silently eating into headroom the sizing logic assumed was free.Impact is bounded (b200-dgxc's ~3246 GB installed DRAM comfortably absorbs the extra ~75.5 GB for a single engine), so this doesn't break anything today and isn't something this PR needs to fix. But it's the identical defect this PR just resolved in the sibling file, and the parity comment sitting right next to the config block now makes an assertion that is code-verifiably false. The fix, when someone gets to it, is the one-line change:
CPU_OFFLOAD_BYTES=$((TOTAL_CPU_DRAM_GB * 1000 * 1000 * 1000))inminimaxm3_fp4_b200_mtp.sh:131, matching what this PR already did for b300.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31833066181 |
|
/reuse-sweep-run |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:Scope. This PR refreshes the Validation and eval evidence. Run 31833066181 ran on the exact PR head Speculative decoding — acceptance length and the eval guard. The submission runs EAGLE3 with the GQA draft head Answering the creator lane's Single-node recipe publication — satisfied by an already-published recipe. This config is Model and scenario scope. MODELS.md lists MiniMax-M3 as active for Agentic coding, with Single-turn 1k1k and Single-turn 8k1k recorded as deprecated (removed 2026-08-04, #2493). This submission is agentic-coding only. MODELS.md also records the engine expectation for this model as the native/upstream vLLM engine with No engine or serving-stack patching. The diff is two YAML files. No Recorded exceptions — creator-preflight findings accepted as out of scope for this PR. The creator preflight reports Signed: |
Ankur-singh
changed the title
❌❌❌ REJECTED ❌❌❌@Ankur-singh — the sign-off uses a stale checklist template: #2613 added an ✅ Check 0 (CODEOWNER): PASS — |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:Scope. This PR refreshes the Validation and eval evidence. Run 31833066181 ran on the exact PR head Speculative decoding — acceptance length and the eval guard. The submission runs EAGLE3 with the GQA draft head Answering the creator lane's Single-node recipe publication — satisfied by an already-published recipe. This config is Model and scenario scope. MODELS.md lists MiniMax-M3 as active for Agentic coding, with Single-turn 1k1k and Single-turn 8k1k recorded as deprecated (removed 2026-08-04, #2493). This submission is agentic-coding only. MODELS.md also records the engine expectation for this model as the native/upstream vLLM engine with No engine or serving-stack patching. The diff is two YAML files. No Recorded exceptions — creator-preflight findings accepted as out of scope for this PR. The creator preflight reports Append-only item — not applicable to this PR. This submission does not set Why this sign-off was re-posted. This supersedes my earlier sign-off on this PR (comment 5299256551), which Signed: |
✅✅✅ Verdict: PASS ✅✅✅Re-verification of the re-posted sign-off (comment 5299320039); the prior rejection's only ground (stale checklist template) is resolved, and all other checks re-derive clean at head ✅ Check 0 (CODEOWNER): PASS — |
|
Usage: |
|
/stage-results 31833066181 |
|
@cquil11 staged run 31833066181: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-14~r31833066181 This run remains available across future |
5 participants
Description
Refresh the MiniMax-M3 NVFP4 B300 AgentX submission to eight EAGLE3-GQA Pareto
points and bump the pinned vLLM nightly.
dram-utilization: 1.0block, applying the proportional-policy cap.vllm/vllm-openai:nightly-ac7509e2b1db40fec2f03dde1ed4e9dfdc2338c9.Acceptance length stays pinned to the committed golden curve: 2.78 from
golden_al_distribution/minimaxm3_eagle3_gqa.yamlthinking_on[3], matchingNUM_SPEC_TOKENS=3and theInferact/MiniMax-M3-EAGLE3-GQAdraft head.Throughput jobs run synthetic acceptance; eval jobs run real verification
(
EVAL_ONLY=trueomitsrejection_sample_method).Scope note: this PR changes
configs/nvidia-master.yamlandperf-changelog.yamlonly. No benchmark script is modified.中文说明
将 MiniMax-M3 NVFP4 B300 AgentX 提交刷新为八个 EAGLE3-GQA 帕累托点，并升级所固定的
vLLM nightly 镜像。
dram-utilization: 1.0区块下新增 TP2 并发 24 的 SimpleCPU KV 卸载点，按比例策略上限配置。
vllm/vllm-openai:nightly-ac7509e2b1db40fec2f03dde1ed4e9dfdc2338c9。接受长度仍固定为已提交的黄金曲线值：取自
golden_al_distribution/minimaxm3_eagle3_gqa.yaml的thinking_on[3]= 2.78，与
NUM_SPEC_TOKENS=3及Inferact/MiniMax-M3-EAGLE3-GQA草稿头一致。吞吐作业使用合成接受率，评测作业使用真实验证（
EVAL_ONLY=true时不设置rejection_sample_method）。范围说明：本 PR 仅修改
configs/nvidia-master.yaml与perf-changelog.yaml，未改动任何基准测试脚本。
Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamlentry added forminimaxm3-fp4-b300-vllm-agentic-mtpminimaxm3_eagle3_gqa.yamlthinking_on[3]= 2.78)models/MiniMaxAI/MiniMax-M3.yaml