feat(vllm): add per-node DP launch mode - #90
Conversation
f9e60ff to
b21248d
Compare
b21248d to
975e1e6
Compare
|
Is there a reason to mix the launch modes? Or is 1 way better than the other? If we have a reason to choose one way then I would prefer to be opinionated. |
975e1e6 to
1a0f9e3
Compare
|
Split as suggested: #90 now contains only the opt-in global |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #90 +/- ##
=======================================
Coverage ? 67.34%
=======================================
Files ? 69
Lines ? 9091
Branches ? 0
=======================================
Hits ? 6122
Misses ? 2969
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary
backend.dp_launch_mode: per_nodefor vLLM data-parallel endpoints.per_gpudefault.--data-parallel-size-local,--data-parallel-start-rank,--data-parallel-address, and DP-RPC settings from the allocated topology.Per-role mixed launch overrides were split into the separate stacked follow-up #271.
Motivation
The existing per-GPU launcher restricts every process to one GPU. DeepGEMM MegaMoE and CUDA symmetric-memory paths can require node-local DP ranks to share one process and CUDA namespace. The opt-in per-node layout provides that namespace without changing existing recipes.
Validation
make check865 passed, 2 skipped, 6 deselected.9700validated uniform per-node launch for3xPDEP8 + 1xDDEP16with decode AMXF4 MegaMoE:26.85 req/s,91,193 TPGSon 40 GPUs, p50 TTFT8.24s, and p50 TPOT58.73ms.