Conversation
Add examples/aimd/ with demo resources for showcasing AIMD adaptive concurrency control in the batch processor. Includes a 50-request JSONL input file targeting sim-model-aimd, a REST Client demo.http with failure injection and recovery sequences, and a README with step-by-step instructions and Grafana monitoring guidance. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Lior Aronovich <lioraronpr@gmail.com>
Replace kubectl patch + deployment rollout with the simulator's new /admin/config endpoint (llm-d-inference-sim v0.9.1). Failure injection rate changes are now instant — no pod restart needed, making the AIMD sawtooth pattern clearly visible on Grafana in real time. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Lior Aronovich <lioraronpr@gmail.com>
Signed-off-by: Lior Aronovich <lioraronpr@gmail.com>
lioraron
changed the title
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Lior Aronovich <lioraronpr@gmail.com>
Remove duplicate file upload from sequence 2 — reuse the file ID from sequence 1 since it's the same input file. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Lior Aronovich <lioraronpr@gmail.com>
|
This PR is marked as stale after 21d of inactivity. After an additional 14d of inactivity (7d to become rotten, then 7d more), it will be closed. To prevent this PR from being closed, add a comment or remove the |
|
This PR is marked as stale after 21d of inactivity. After an additional 14d of inactivity (7d to become rotten, then 7d more), it will be closed. To prevent this PR from being closed, add a comment or remove the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
examples/aimd/with demo resources for the AIMD adaptive concurrency featuresim-model-aimd)demo.httpwith three sequences: observe AIMD drop under 429 failures, clear failures via runtime admin API and observe recovery, reset/admin/configendpoint (llm-d-inference-sim v0.9.1) for instant failure toggling — no pod restarts neededTest plan
make dev-deployand verifyvllm-sim-aimdpod is runningkubectl port-forward svc/vllm-sim-aimd 8888:8000 -n defaultdemo.http— verify AIMD concurrency drops on Grafana/admin/config, verify AIMD recovery on Grafana🤖 Generated with Claude Code