[Klaud Cold][agentic experiment][Variant G] Kimi-K3 B200 agg TP8xPP2 agentic — direct vllm serve + prefix-cache retention 0 / Kimi-K3 B200 聚合式 TP8xPP2 智能体实验——直接 vllm serve + 前缀缓存留存 0 - #2374

Closed

functionstackx wants to merge 21 commits into

mainfrom

klaud/kimik3-b200-agentic-direct-vllm-retention0

Commits on Jul 27, 2026

feat: add Kimi-K3 MXFP4 B200 aggregated TP8xPP2 Dynamo-vLLM agentic recipe

functionstackx
and
claude
committed
b4fd077
docs: link PR #2355 in changelog entry and MODELS rows

functionstackx
and
claude
committed
4dbbdc8
fix: pin agentic srt-slurm to NVIDIA v1.0.36, dynamo 1.2.1, conc-8 smoke test

functionstackx
and
claude
committed
e8d42a7
fix: drop OpenAI-frontend tool-choice flags from dynamo-vllm worker args

functionstackx
and
claude
committed
c1e2a56
fix: try dynamo wheel 1.2.0.dev20260426 for Kimi-K3 frontend tokenizer

functionstackx
and
claude
committed
ef35fd1
fix: pin dynamo to day-zero Kimi-K3 commit, restore kimi_k3 parser flags

functionstackx
and
claude
committed
be6c56e
fix: use dynamo namespaced --dyn-* kimi_k3 parser args on the worker

functionstackx
and
claude
committed
c6917e6
fix: disable aiperf conv-aware routing (session_control 400-rejected)

functionstackx
and
claude
committed
a26853a
fix: patch kimi-k3 image mamba_hybrid index_fill_ dtype via setup_script

functionstackx
and
claude
committed
f61eafb
feat: agentic experiment D — direct vllm serve via srt-slurm PR #278

functionstackx
and
claude
committed
8fd319c
docs: point changelog and MODELS rows at experiment PR #2359

functionstackx
and
claude
committed
c0ace4d
fix: drop gpu-memory-utilization to 0.90 (flashinfer MoE workspace OOM)

functionstackx
and
claude
committed
862024d
fix: expandable_segments allocator, drop NCCL_CUMEM_ENABLE (prefill OOM)

functionstackx
and
claude
committed
4370988
Revert "fix: expandable_segments allocator, drop NCCL_CUMEM_ENABLE (prefill OOM)"

functionstackx
and
claude
committed
0c5fe11
feat: add VLLM_PREFIX_CACHE_RETENTION_INTERVAL=32768

functionstackx
and
claude
committed
e675dd2
feat: widen agentic conc list to 1/8/16/32

functionstackx
and
claude
committed
4b0c3a4
fix: remove VLLM_PREFIX_CACHE_RETENTION_INTERVAL (K3 scheduler block)

functionstackx
and
claude
committed
479b74b

Commits on Jul 28, 2026

feat: agentic experiment G — variant D + prefix-cache retention 0

functionstackx
and
claude
committed
3824d0e
docs: point changelog and MODELS rows at experiment PR #2374

functionstackx
and
claude
committed
d002202
Merge remote-tracking branch 'origin/main' into klaud/kimik3-b200-agentic-direct-vllm-retention0
functionstackx
committed
58f1652
feat: variant G conc curve 1/2/4/8/16/32 (add 2 and 4)

functionstackx
and
claude
committed
f14afdc

Navigation Menu

Uh oh!

[Klaud Cold][agentic experiment][Variant G] Kimi-K3 B200 agg TP8xPP2 agentic — direct vllm serve + prefix-cache retention 0 / Kimi-K3 B200 聚合式 TP8xPP2 智能体实验——直接 vllm serve + 前缀缓存留存 0 - #2374

Commits on Jul 27, 2026

feat: add Kimi-K3 MXFP4 B200 aggregated TP8xPP2 Dynamo-vLLM agentic recipe

docs: link PR #2355 in changelog entry and MODELS rows

fix: pin agentic srt-slurm to NVIDIA v1.0.36, dynamo 1.2.1, conc-8 smoke test

fix: drop OpenAI-frontend tool-choice flags from dynamo-vllm worker args

fix: try dynamo wheel 1.2.0.dev20260426 for Kimi-K3 frontend tokenizer

fix: pin dynamo to day-zero Kimi-K3 commit, restore kimi_k3 parser flags

fix: use dynamo namespaced --dyn-* kimi_k3 parser args on the worker

fix: disable aiperf conv-aware routing (session_control 400-rejected)

fix: patch kimi-k3 image mamba_hybrid index_fill_ dtype via setup_script

feat: agentic experiment D — direct vllm serve via srt-slurm PR #278

docs: point changelog and MODELS rows at experiment PR #2359

fix: drop gpu-memory-utilization to 0.90 (flashinfer MoE workspace OOM)

fix: expandable_segments allocator, drop NCCL_CUMEM_ENABLE (prefill OOM)

Revert "fix: expandable_segments allocator, drop NCCL_CUMEM_ENABLE (prefill OOM)"

feat: add VLLM_PREFIX_CACHE_RETENTION_INTERVAL=32768

feat: widen agentic conc list to 1/8/16/32

fix: remove VLLM_PREFIX_CACHE_RETENTION_INTERVAL (K3 scheduler block)

Commits on Jul 28, 2026

feat: agentic experiment G — variant D + prefix-cache retention 0

docs: point changelog and MODELS rows at experiment PR #2374

Merge remote-tracking branch 'origin/main' into klaud/kimik3-b200-agentic-direct-vllm-retention0

feat: variant G conc curve 1/2/4/8/16/32 (add 2 and 4)