Add DSV4-Pro disagg frontier recipes (B200 + B300) on rc21 + Dynamo nightly - #274
Open
tanmayv25 wants to merge 1 commit into
Open
Add DSV4-Pro disagg frontier recipes (B200 + B300) on rc21 + Dynamo nightly#274tanmayv25 wants to merge 1 commit into
tanmayv25 wants to merge 1 commit into
Conversation
…ightly Pareto-throughput (sa-bench) frontier configs for DeepSeek-V4-Pro disaggregated serving under recipes/dsv4-sa-repro/8k1k_frontiers_only/ (SA-b200-20260713 [16] + SA-b300-20260713 [20], each with PERF.md). - Stock TRT-LLM release:1.3.0rc21 + Dynamo nightly wheel 1.3.0.dev2026071601 (dynamo.wheel; rc21 has DSV4-Pro support so no dsv4 wheel needed). - frontend: dynamo; enable_multiple_frontends true for prefill_workers >= 3 (validated: Dynamo otherwise bottlenecks on a single router; matches trtllm_serve within ~1.2% with the rule applied). - Offline EPLB load_balancer configs bundled under configs/ and referenced via /configs/ (ctx_ep4_384 for prefill, gen_ep8_slots384 for decode). - Autotuner-cache env dropped for portability; model.path stays dsv4-pro. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
tanmayv25
requested review from
alec-flowers,
csahithi,
hjjq,
ishandhanani,
kedarpotdar-nv,
kyleliang-nv,
nlevin-ui and
qiching
as code owners
July 20, 2026 19:36
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #274 +/- ##
=======================================
Coverage ? 67.80%
=======================================
Files ? 69
Lines ? 9106
Branches ? 0
=======================================
Hits ? 6174
Misses ? 2932
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds the DeepSeek-V4-Pro disaggregated frontier recipes (B200 + B300, 8k1k) under
recipes/dsv4-sa-repro/8k1k_frontiers_only/— the Pareto-throughput configs from our sweep, on stock TRT-LLMrelease:1.3.0rc21+ Dynamo nightly wheel1.3.0.dev2026071601.Contents (40 files)
PERF.mdper hardware.frontend: dynamo;enable_multiple_frontends: trueforprefill_workers >= 3— validated: Dynamo otherwise bottlenecks on a single router at high prefill-worker counts, and matches trtllm_serve within ~1.2% with the rule applied.dynamo.wheel.load_balancerconfigs bundled underconfigs/and referenced via/configs/(prefillctx_ep4_384, decodegen_ep8_slots384).Notes
model.path: dsv4-pro(cluster model-key) and a sa-benchdataset_path— cluster-local, resolved per running cluster (consistent with existing recipes).srtctl dry-runvalidated the b300 stack earlier. Not yet run end-to-end on B200/B300 GPU from the authoring env.🤖 Generated with Claude Code