Add Kimi K2.6 NVFP4 B200 Dynamo vLLM configuration / 新增 Kimi K2.6 NVFP4 B200 Dynamo vLLM 配置 - #2360
Add Kimi K2.6 NVFP4 B200 Dynamo vLLM configuration / 新增 Kimi K2.6 NVFP4 B200 Dynamo vLLM 配置#2360RohitNagraj wants to merge 11 commits into
Conversation
Add seven 8K/1K disaggregated configuration points and load their recipes from NVIDIA/srt-slurm. 中文：新增 Kimi K2.6 B200 Dynamo vLLM 配置，注册七个 8K/1K 分离式推理配置项，并从 NVIDIA/srt-slurm 加载相应配方。
中文：在性能变更日志中补充 #2360 的 PR 链接。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Fetch the public recipe PR ref and check out its immutable commit before launching the configuration. 中文：获取公开配方 PR 引用，并在启动配置前检出其不可变提交。
Bring the configuration branch up to date with the latest public repository changes. 中文：将配置分支同步到公开仓库的最新 main。
Point the B200 launch script at the Kimi K2.6 NVFP4 checkpoint staged on cluster storage. 中文：将 B200 启动脚本指向集群存储中已准备好的 Kimi K2.6 NVFP4 模型路径。
|
Claude finished @RohitNagraj's task in 5m 16s —— View job PR Review
SummaryNo blocking issues found. One 🟡 warning posted inline: the srt-slurm pin in Everything else checks out:
Two things I could not verify from this environment (network-restricted) and which the sweep run will confirm:
Minor non-blocking note: |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| git fetch origin refs/pull/282/head | ||
| git checkout d94c8505136b1e021ade99f1e7cd3359fc491cb9 |
There was a problem hiding this comment.
🟡 WARNING: Pinned SHA d94c8505 is only reachable through refs/pull/282/head, and a failed checkout is not caught.
Why it matters: Unlike the sibling branches here (which pin a branch/tag or a SHA on main), this SHA lives only on an open PR's head ref. If NVIDIA/srt-slurm#282 is force-pushed or rebased before merge, git fetch origin refs/pull/282/head will no longer bring in d94c8505, the git checkout fails, and — since the script has no set -e — the run silently continues on main without recipes/vllm/kimi-k2.6/..., failing much later at the sed/srtctl apply step with a confusing error.
Fix: Fail fast on checkout so the root cause is obvious:
| git fetch origin refs/pull/282/head | |
| git checkout d94c8505136b1e021ade99f1e7cd3359fc491cb9 | |
| git fetch origin refs/pull/282/head | |
| git checkout d94c8505136b1e021ade99f1e7cd3359fc491cb9 || exit 1 |
Longer-term, once srt-slurm#282 merges, repoint this at a main-reachable SHA (like the dsr1-fp4 case above) so the pin can't be orphaned.
There was a problem hiding this comment.
LGTM — straightforward, well-patterned model onboarding config for Kimi K2.6 on B200.
What was reviewed: the new kimik2.6-fp4-b200-dynamo-vllm master-config entry (structure matches existing Kimi K2.5/GLM-5/DSv4 disaggregated dynamo-vllm entries), the perf-changelog addition, and the new MODEL_PATH/SRT_SLURM_MODEL_PREFIX + srt-slurm checkout branch in launch_b200-dgxc.sh. The bug hunting system flagged the /scratch/models vs /scratch/fsw/models path convention as a candidate issue and ruled it out; I don't have cluster access to independently confirm the staging path, but the launcher fails loudly (missing MODEL_PATH check + full-sweep CI) if it's wrong, so it will surface quickly rather than silently corrupting results.
Extended reasoning...
Overview
This PR adds a new model recipe: Kimi K2.6 NVFP4 disaggregated prefill/decode config for B200 using Dynamo vLLM. It touches three files: configs/nvidia-master.yaml (new top-level config block with 7 concurrency scenarios pointing to srt-slurm recipe YAMLs), perf-changelog.yaml (changelog entry), and runners/launch_b200-dgxc.sh (new MODEL_PATH resolution branch and a new srt-slurm checkout branch pinned to a specific commit on NVIDIA/srt-slurm PR #282).
Security risks
None. This is a benchmark configuration and CI launcher change — no user input handling, auth, or crypto involved. The srt-slurm checkout is pinned to a specific commit hash (not a floating branch), which is the safer pattern already used elsewhere in this file (e.g. the dsr1 sglang branch has an explicit comment about not floating on main).
Level of scrutiny
This is a standard 'add a new model+hardware recipe' PR, which is a well-trodden, high-frequency pattern in this repo (dozens of near-identical entries already exist for other models/precisions). The new master-config block and launcher branch are structurally identical to existing Kimi K2.5 / GLM-5 / DSv4 entries, so the risk of a subtle logic bug is low, and the PR review checklist already gates this via CODEOWNER sign-off and the full-sweep CI (which actually runs the job on hardware and would fail loudly on a bad MODEL_PATH or CONFIG_FILE reference). The main manual-review-only risk is the exact MODEL_PATH staging convention, which the bug hunting system already flagged and investigated.
Other factors
No unresolved review comments in the timeline. This is the first substantive review pass on this PR (only the automated recipe-reminder bot and a placeholder "working" comment precede it).
Stop the B200 launcher immediately when the pinned srt-slurm recipe commit cannot be checked out. 中文：当固定的 srt-slurm 配方提交无法检出时，立即终止 B200 启动流程。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30313613266 |
Pass --no-preflight only for Kimi K2.6 FP4 Dynamo-vLLM so srtctl can submit recipes whose model path is available on Slurm compute nodes. 中文：仅为 Kimi K2.6 FP4 Dynamo-vLLM 传递 --no-preflight，使 srtctl 能够提交模型路径仅在 Slurm 计算节点可用的配方。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30314088748 |
Point Kimi K2.6 NVFP4 at the shared Lustre path exposed on B200 Slurm nodes. 中文：将 Kimi K2.6 NVFP4 指向 B200 Slurm 节点可访问的共享 Lustre 路径。
|
sorry @RohitNagraj i am cancelling https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30314875577 for the day 0 k3 rn |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30314875577 |
5 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30314875577 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30314875577 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30314875577 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30314875577 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30314875577 |
Update the pinned srt-slurm recipe revision so multi-prefill workers rely on vLLM-generated engine IDs. 中文：更新固定的 srt-slurm 配方版本，使多预填充工作进程使用 vLLM 自动生成的 engine ID。
Preserve current main and re-append the Kimi K2.6 changelog entry at EOF. 中文：合并当前 main，并在文件末尾重新追加 Kimi K2.6 变更日志条目。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30418587119 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30418587119 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
❌❌❌ REJECTED ❌❌❌@Ankur-singh — blocking issue: this PR's pinned benchmark path patches the vLLM engine inside the pinned image at runtime, with no waiver at ✅ Check 0 (CODEOWNER): PASS — |
Add seven checked-in srt-slurm recipes, pin the current srt-slurm main commit, and run the vLLM 0.25.1 image without setup hooks or runtime patches. 中文：将 Kimi K2.6 配置同步到当前 main，加入七个随代码提交的 srt-slurm 配方，固定当前 srt-slurm main 提交，并直接运行 vLLM 0.25.1 镜像，不使用额外的 setup hook 或运行时补丁。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30495901633 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30495901633 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30495901633 |
4 participants
Summary
Recipe definitions: NVIDIA/srt-slurm#282
Recipe reference: Kimi-K2.6 vLLM recipe
中文说明
配方定义：NVIDIA/srt-slurm#282
配方参考：Kimi-K2.6 vLLM 配方