Skip to content

feat(vllm): support direct aggregate serving - #278

Merged
ishandhanani merged 1 commit into
mainfrom
kylliang/direct-aggregate-vllm
Jul 28, 2026
Merged

feat(vllm): support direct aggregate serving#278
ishandhanani merged 1 commit into
mainfrom
kylliang/direct-aggregate-vllm

Conversation

@kyleliang-nv

@kyleliang-nv kyleliang-nv commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add frontend.type: vllm for aggregate jobs that launch vllm serve directly, without Dynamo
  • validate the supported topology: vLLM backend, aggregate-only, one worker on one node, and one frontend
  • wire direct-vLLM health checks, telemetry, AIPerf server metrics, profiling, and public-port ownership
  • preserve current main per-node DP behavior for Dynamo while keeping direct vLLM in the standard one-process topology

Motivation

Aggregate vLLM benchmarks do not need Dynamo as an orchestration layer. This provides a native direct-serving path while leaving existing Dynamo aggregate and disaggregated configurations unchanged.

Validation

  • uv run ruff check on all changed files
  • uv run pytest -q tests/test_configs.py tests/test_frontends.py (162 passed)
  • uv run pytest -q (893 passed, 2 skipped, 6 deselected)

Signed-off-by: kylliang <kylliang@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants