feat(benchmark): expose logical worker endpoints - #276
Merged
Conversation
cquil11
force-pushed
the
agent/expose-worker-endpoints
branch
from
July 21, 2026 00:16
36be8a9 to
3fe5508
Compare
cquil11
marked this pull request as ready for review
July 27, 2026 22:56
cquil11
requested review from
alec-flowers,
csahithi,
ishandhanani and
nlevin-ui
as code owners
July 27, 2026 22:56
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
cquil11
force-pushed
the
agent/expose-worker-endpoints
branch
from
July 28, 2026 21:49
aedc846 to
de78dfa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
benchmark.type: customan orderedAIPERF_SERVER_METRICS_URLSwithout requiring Slurm hostlist parsing or duplicated topology metadata.Motivation
srt-slurm already owns endpoint allocation, node placement, port assignment, heterogeneous-job topology, and network-interface-aware IP resolution. A custom benchmark currently cannot consume that authoritative runtime mapping. Consumers must reconstruct it from
SLURM_JOB_NODELIST, recipe-specific node counts, and the system-port allocation algorithm.That reconstruction is brittle and cannot distinguish a logical worker leader from a multinode follower rank. InferenceX PR #2291 is a concrete consumer: its custom AgentX/AIPerf replay needs SGLang worker metrics to drain requests correctly between concurrency points while running against an unmodified upstream srt-slurm release.
Environment contract
When a mode is present, custom benchmark processes receive:
SRT_PREFILL_IPSSRT_PREFILL_ENDPOINTSSRT_DECODE_IPSSRT_DECODE_ENDPOINTSSRT_AGG_IPSSRT_AGG_ENDPOINTSIP:portformat.DYN_SYSTEM_PORT; other frontends use the worker HTTP port.AIPERF_SERVER_METRICS_URLSas fully qualified/metricsURLs in the same order. Configured KVBM metrics URLs remain appended after worker URLs.benchmark.envis applied last, so recipes can override any generated value explicitly.This contract is computed from
backend_processes, the same cached topology used to launch workers. It does not hardcode SGLang recipe layouts or Slurm hostlist rules and therefore also covers aggregated/disaggregated layouts across upstream backends.Compatibility
This is additive for benchmark environments and requires no recipe schema change. Built-in AIPerf runners retain their historical physical-process URL list because vLLM data-parallel layouts rely on it; only custom commands use the new logical-worker AIPerf view.
Validation
uv run pytest tests/test_benchmarks.py::TestCustomBenchmarkRunner tests/test_node_placement.py -q— 15 passed.uv run ruff check src/srtctl/— passed.uv run ruff format --check src/srtctl/— passed.mainat38723f6and are macOS/environment baseline failures (sched_getaffinity, CPU allocation metadata, and the existing SA-Bench profiling-script path test).