[WIP][AMD][MI355x] dsv4-fp4-mi355x-vllm-agentic, install latest lmcache#2216
[WIP][AMD][MI355x] dsv4-fp4-mi355x-vllm-agentic, install latest lmcache#2216ruizhang0101 wants to merge 3 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
00897b8 to
4788c65
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29375661761 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
4788c65 to
7d1d4e2
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29375704484 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29378716513 |
7d1d4e2 to
9b66fa6
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29385533117 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29432613472 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29450233700 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29450233700 |
7dbae3d to
7f681e5
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29534525713 |
7f681e5 to
9b219b2
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29535114622 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29536630935 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29536630935 |
ffb8879 to
ae9a711
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29547186370 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29547585852 |
a8c16cf to
9761c32
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29609293689 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29611889601 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29611889601 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29611889601 |
9761c32 to
a185652
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29791171947 |
| ), | ||
| Fields.DISAGG.value: disagg, | ||
| Fields.SCENARIO_TYPE.value: "agentic-coding", | ||
| Fields.RUN_EVAL.value: False, |
There was a problem hiding this comment.
why is run eval default to false?
There was a problem hiding this comment.
I was not sure whether the eval part is ready. If it is ready now, I can flip to true.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29791171947 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29791171947 |
Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com>
…ault-on - amd-master.yaml: drop the lmcache DRAM arm from tp 8 to tp 4 (conc-list unchanged). Halves the LMCache L1 pool partition (TOTAL_CPU_DRAM_GB / (8/TP)) so the MP server finishes allocation inside the healthcheck window. - generate_sweep_configs.py: mark_eval_entries now includes agentic entries by default (include_agentic=True) - the highest-conc single-node agentic entry per (model, runner, framework, precision) group is marked run-eval in default invocations, matching --evals-only/--all-evals behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com>
a185652 to
091006b
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30062040756 |
Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30120665378 |
3 participants
Summary
CXX=hipcc BUILD_WITH_HIP=1)--no-separate-object-groupto the LMCache MP server launch commandvllm/vllm-openai-rocm:nightly-09663abde0f50944a8d5ea30120666024b503faa(rotated off Docker Hub retention, 404s) to the stablevllm/vllm-openai-rocm:v0.25.1release tagdf -h /dev/shm+free -hprinted before LMCache server launch (marked for removal before merge)--no-l1-use-lazy+--shm-name ""): even at a 30min timeout the eager L1 pool allocation didn't come up healthy. Back to LMCache's default lazy allocator (small init size, grows on demand) and the original 120-attempt healthcheck timeout.utils/matrix_logic/generate_sweep_configs.py: agentic entry dicts never stampedrun-evalat construction time, unlike their fixed-seq-len siblings — addedFields.RUN_EVAL.valuestamps so inline validation succeeds beforemark_eval_entriesoverwrites it (now defaulted toTrue, see below).utils/matrix_logic/validation.py: added the missingrun-eval/eval-onlyfields toMultiNodeAgenticMatrixEntry(the sibling single-node fix landed upstream on main separately, with a more complete design — dedicatedagentic_evalsbucket +Optional[bool]/exclude_none— deferred to that for the single-node side).[32, 40, 48]unchanged): halves the LMCache L1 pool partition (TOTAL_CPU_DRAM_GB / (8/TP)) so the MP server comes up inside the healthcheck window.generate_sweep_configs.py: agentic entry dicts now stampFields.RUN_EVAL.value: Trueat construction time (SWE-bench eval is ready — per review thread).mark_eval_entriesstill overwrites it in eval-marked invocations; the construction default only survives--no-evalsruns.