Skip to content

fix(profiling): use Dynamo control/ engine routes for start/stop_profile - #264

Open
shyeh25 wants to merge 1 commit into
NVIDIA:mainfrom
shyeh25:fix_profiler_break
Open

fix(profiling): use Dynamo control/ engine routes for start/stop_profile#264
shyeh25 wants to merge 1 commit into
NVIDIA:mainfrom
shyeh25:fix_profiler_break

Conversation

@shyeh25

@shyeh25 shyeh25 commented Jul 14, 2026

Copy link
Copy Markdown

Dynamo moved its control-plane engine routes under a control/ namespace in ai-dynamo/dynamo#10347 (e6fd808c0f, 2026-06-15):

  • register_engine_route("start_profile", ...)
  • register_engine_route("control/start_profile", ...)

So any Dynamo build ≥ #10347 serves the profiler-trigger at /engine/control/start_profile (and /engine/control/stop_profile), not the old /engine/start_profile.

srt-slurm's profiling client (lib/profiling.sh) was still POSTing the old paths. Against a current Dynamo this silently 404s → the profiler is never armed → prefill/decode nsys
(and torch) profiles come out empty. There's no error surfaced because the call is best-effort (curl -f ... || Warning).

This patch points the dynamo frontend branch at the new control/ routes.

Scope / safety:

  • Only affects the dynamo frontend profiling path (any backend behind it: sglang, vllm). The sglang native-frontend branch (/start_profile) is unchanged.
  • TRT-LLM is unaffected (it profiles via TLLM_PROFILE_START_STOP, no HTTP call).
  • No effect on non-profiling runs — this code only runs when profiling: is enabled.
  • Requires Dynamo ≥ #10347. Older Dynamo served /engine/start_profile; pairing this with a pre-#10347 Dynamo would 404. In practice any Dynamo new enough to also carry the sglang ProfileReq body fix (fix(sglang): support profile request object API ai-dynamo/dynamo#11199, 2026-07-06) is already ≥ #10347, so the two go together.

Dynamo moved its control-plane engine routes under a control/ namespace in ai-dynamo/dynamo#10347 (e6fd808c0f, 2026-06-15):

  - register_engine_route("start_profile", ...)
  + register_engine_route("control/start_profile", ...)

  So any Dynamo build ≥ #10347 serves the profiler-trigger at /engine/control/start_profile (and /engine/control/stop_profile), not the old /engine/start_profile.

  srt-slurm's profiling client (lib/profiling.sh) was still POSTing the old paths. Against a current Dynamo this silently 404s → the profiler is never armed → prefill/decode nsys
  (and torch) profiles come out empty. There's no error surfaced because the call is best-effort (curl -f ... || Warning).

  This patch points the dynamo frontend branch at the new control/ routes.

  Scope / safety:
  - Only affects the dynamo frontend profiling path (any backend behind it: sglang, vllm). The sglang native-frontend branch (/start_profile) is unchanged.
  - TRT-LLM is unaffected (it profiles via TLLM_PROFILE_START_STOP, no HTTP call).
  - No effect on non-profiling runs — this code only runs when profiling: is enabled.
  - Requires Dynamo ≥ #10347. Older Dynamo served /engine/start_profile; pairing this with a pre-#10347 Dynamo would 404. In practice any Dynamo new enough to also carry the sglang
  ProfileReq body fix (ai-dynamo/dynamo#11199, 2026-07-06) is already ≥ #10347, so the two go together.
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@f6eb42a). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #264   +/-   ##
=======================================
  Coverage        ?   67.21%           
=======================================
  Files           ?       69           
  Lines           ?     9040           
  Branches        ?        0           
=======================================
  Hits            ?     6076           
  Misses          ?     2964           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants