[Klaud Cold] dsv4-fp4-b200-vllm: nightly-700d39b image, prefill-schedule-interval 16 at CONC≥256 / 更新 B200 DSv4 镜像并在高并发下启用 prefill 调度间隔 - #2534
[Klaud Cold] dsv4-fp4-b200-vllm: nightly-700d39b image, prefill-schedule-interval 16 at CONC≥256 / 更新 B200 DSv4 镜像并在高并发下启用 prefill 调度间隔#2534xinli-sw wants to merge 16 commits into
Conversation
…o nightly-700d39b
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — straightforward image bump and a scoped CLI flag addition, consistent with prior recipe-tuning PRs in this repo.
Extended reasoning...
Overview
This PR touches a single benchmark recipe (dsv4_fp4_b200_vllm.sh), its master-config image tag (configs/nvidia-master.yaml), and the changelog (perf-changelog.yaml). It bumps the vLLM image to a nightly build and adds an unconditional --prefill-schedule-interval 16 override for CONC >= 256, which takes precedence over the existing DP_ATTENTION-only interval of 4. This mirrors the standard 'nuke'-style image bump + perf-tuning pattern used throughout this repo's recipe files.
Security risks
None. This is a benchmark shell script and YAML config for internal perf sweeps — no user input, auth, or credential handling is touched. The image tag change points to an official vllm/vllm-openai nightly build referenced by immutable digest-like tag.
Level of scrutiny
Low. This is a narrow, mechanical change confined to a single model/hardware/engine recipe combination, following the same shape as many prior merged PRs (image bump + a single new conditional CLI flag). The bug-hunting system found no issues, and the one candidate concern raised (the CONC>=256 branch silently overriding the DP_ATTENTION interval-4 setting) was examined and ruled out — it reads as an intentional override for high-concurrency prefill scheduling, not an unintended clobber, and only affects this one recipe's own script.
Other factors
No outstanding reviewer comments to address. Change is fully self-contained (script + config + changelog kept in sync), and verification is enforced via the full-sweep-fail-fast label already applied, which will run the actual benchmark sweep against this recipe before merge.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31266541491 |
…ure, gmu 0.94; fix perf-changelog
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31280559933 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31280587045 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31283688198 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31296702954 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31297069605 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31298034747 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31320204256 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31320761695 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31327444049 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31426751542 |
1 participant
Summary
dsv4-fp4-b200-vllmimage tovllm/vllm-openai:nightly-700d39b55813fce6fe5339b334beb55d2a39f598--prefill-schedule-interval 16forCONC >= 256indsv4_fp4_b200_vllm.sh中文说明
dsv4-fp4-b200-vllm镜像升级至nightly-700d39b55813fce6fe5339b334beb55d2a39f598dsv4_fp4_b200_vllm.sh中，当CONC >= 256时启用--prefill-schedule-interval 16