Add DeepSeek V4 GB300 Dynamo TensorRT-LLM agentic config / 添加 DeepSeek V4 GB300 Dynamo TensorRT-LLM Agentic 配置 - #2274
Conversation
Add eight disaggregated MTP topologies and the agentic routing, container-cache fallback, and GB300 launch support they require. 中文：添加八组分离式 MTP 拓扑，并补齐所需的 Agentic 会话路由、容器缓存回退及 GB300 启动支持。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Replace the placeholder with the published upstream PR URL. 中文：将性能变更日志中的占位链接更新为已发布的上游 PR URL。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I didn't find any bugs, but this PR touches shared infrastructure (benchmark_lib.sh's dynamo session-routing logic and the GB300 launcher's squash-import locking/fallback path) in addition to adding eight new agentic recipes, so it's worth a human look before merging.
What was reviewed: the new session-control branch in build_replay_cmd (X-Session-ID vs the legacy conv-aware-routing flags), the refactored squash-import lock/fallback flow in launch_gb300-nv.sh, the new dynamo-trt agentic dispatch branch, and the eight new recipe YAMLs plus the nvidia-master.yaml/perf-changelog.yaml entries.
Extended reasoning...
Overview
This PR adds eight new disaggregated MTP prefill/decode topology recipes for DeepSeek V4 Pro FP4 on GB300 using Dynamo TensorRT-LLM, plus supporting changes: a new session-routing branch in benchmarks/benchmark_lib.sh (opt-in X-Session-ID header vs legacy AIPerf conv-aware-routing flags), a new dynamo-trt/dsv4 dispatch branch in runners/launch_gb300-nv.sh that clones srt-slurm v1.0.30, a reworked squash-file import path (configurable primary dir + local-fallback with a proper write-probe and atomic tmp-then-rename), master-config and perf-changelog entries.
Security risks
No injection, auth, or credential-handling concerns. The squash-import rework uses srun ... bash -s -- with positional args (avoiding the prior string-interpolation-into-heredoc pattern), which is a modest security/robustness improvement. All values driving the launcher come from repo-controlled recipe config, not external user input.
Level of scrutiny
This PR mixes low-risk additive changes (new YAML recipes, master-config, changelog) with a shared-infrastructure change: the squash-import/lock rework and the new benchmark_lib.sh session-routing branch affect every GB300 job and every dynamo-* recipe respectively, not just the eight new configs. That combination — new recipes are easy to sanity-check, but the launcher/library changes are shared blast-radius — pushes this past the bar for a mechanical/config-only approval, even with zero reported bugs.
Other factors
The bug-hunting system found no bugs, and the four candidate issues raised by finders (batch_sizes vs max_batch_size, squash-lock move to /tmp, squash fallback dir not being shared storage, and the pyxis '#'-to-'/' image name handling, plus the literal-string 'None' env var) were all examined and ruled out this run. The recipe still carries the standard CODEOWNER sign-off requirement per the bot's reminder comment, which has not yet been completed in the visible timeline.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29657037702 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29663987749 |
4 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29663987749 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29663987749 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29663987749 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29663987749 |
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…cipes Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…nd (append-only) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30326998980 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30326998980 |
…GB300 agentic TRT-LLM recipes Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30408237861 |
中文：将 DeepSeek V4 配置与当前 main 分支同步，并保持性能变更日志的仅追加顺序。
|
@Ankur-singh the AgentX/AIPerf harness has been updated, please merge origin/main into your branch and refresh your submission. Additional tuning may be necessary depending on the config. I apologize for any inconvenience. This is an automated message. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30478794351 |
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…in dsv4 GB300 TRT-LLM recipes 中文：将 aiperf 固定到 abf55f9，并在 dsv4 GB300 TRT-LLM 配方中设置 MIMALLOC_PURGE_DELAY=-1 Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…onfig Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30731197609 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30731197609 |
a5fc766 to
b41d31c
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30760685148 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30760769162 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30760769162 |
4 participants
Summary
batch_1,benchmark, andsa-sharedcluster values.中文说明
batch_1、benchmark和sa-shared配置。