Skip to content

feat(vllm): support mixed DP launch modes - #271

Draft
esmeetu wants to merge 2 commits into
NVIDIA:mainfrom
esmeetu:yasong/vllm-mixed-dp-launch
Draft

feat(vllm): support mixed DP launch modes#271
esmeetu wants to merge 2 commits into
NVIDIA:mainfrom
esmeetu:yasong/vllm-mixed-dp-launch

Conversation

@esmeetu

@esmeetu esmeetu commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add role-specific prefill_dp_launch_mode, decode_dp_launch_mode, and aggregated_dp_launch_mode overrides for the vLLM backend.
  • Allow per-node and per-GPU DP process layouts to coexist in one deployment.
  • Share topology port allocation across both layouts so KV-event, NIXL, DP-RPC, HTTP, and system ports remain non-overlapping.
  • Derive Dynamo health expectations from the generated process topology.

Dependency

This is a stacked follow-up to #90. PR #90 now contains only the global opt-in dp_launch_mode: per_node implementation; this PR's intended review diff is the top commit, feat(vllm): support per-role DP launch modes.

Until #90 merges, GitHub will also show its prerequisite commit in this PR. After #90 lands, this branch will be rebased onto main so only the mixed-DP changes remain.

Motivation

Some vLLM roles need one process per node so node-local DP ranks share a CUDA namespace, while another role may need independent per-GPU Dynamo workers. The override is process-topology flexibility; it does not claim either layout has an inherent request-ingest performance advantage.

backend:
  type: vllm
  dp_launch_mode: per_node
  decode_dp_launch_mode: per_gpu
  vllm_config:
    prefill:
      data-parallel-size: 8
      data-parallel-hybrid-lb: true
    decode:
      data-parallel-size: 16
      data-parallel-hybrid-lb: true

Unset role overrides inherit the global dp_launch_mode.

Validation

  • make check
  • Ruff and formatting pass.
  • Full test suite: 867 passed, 2 skipped, 6 deselected.
  • Mixed prefill-per-node/decode-per-GPU launch has been exercised on a 3xPDEP8 + 1xDDEP16 DeepSeek V4 Pro deployment.

@esmeetu
esmeetu marked this pull request as ready for review July 18, 2026 02:31
@esmeetu
esmeetu marked this pull request as draft July 21, 2026 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant