[Klaud Cold][agentic experiment][Variant E] Kimi-K3 B200 agg TP8xPP2 agentic — direct vllm serve + SimpleCPU KV offload / Kimi-K3 B200 聚合式 TP8xPP2 智能体实验——直接 vllm serve + SimpleCPU KV 卸载 - #2370

Closed

functionstackx wants to merge 21 commits into

mainfrom

klaud/kimik3-b200-agentic-direct-vllm-simplecpu

Commits on Jul 27, 2026

feat: add Kimi-K3 MXFP4 B200 aggregated TP8xPP2 Dynamo-vLLM agentic recipe

functionstackx
and
claude
committed
b4fd077
docs: link PR #2355 in changelog entry and MODELS rows

functionstackx
and
claude
committed
4dbbdc8
fix: pin agentic srt-slurm to NVIDIA v1.0.36, dynamo 1.2.1, conc-8 smoke test

functionstackx
and
claude
committed
e8d42a7
fix: drop OpenAI-frontend tool-choice flags from dynamo-vllm worker args

functionstackx
and
claude
committed
c1e2a56
fix: try dynamo wheel 1.2.0.dev20260426 for Kimi-K3 frontend tokenizer

functionstackx
and
claude
committed
ef35fd1
fix: pin dynamo to day-zero Kimi-K3 commit, restore kimi_k3 parser flags

functionstackx
and
claude
committed
be6c56e
fix: use dynamo namespaced --dyn-* kimi_k3 parser args on the worker

functionstackx
and
claude
committed
c6917e6
fix: disable aiperf conv-aware routing (session_control 400-rejected)

functionstackx
and
claude
committed
a26853a
fix: patch kimi-k3 image mamba_hybrid index_fill_ dtype via setup_script

functionstackx
and
claude
committed
f61eafb
feat: agentic experiment D — direct vllm serve via srt-slurm PR #278

functionstackx
and
claude
committed
8fd319c
docs: point changelog and MODELS rows at experiment PR #2359

functionstackx
and
claude
committed
c0ace4d
fix: drop gpu-memory-utilization to 0.90 (flashinfer MoE workspace OOM)

functionstackx
and
claude
committed
862024d
fix: expandable_segments allocator, drop NCCL_CUMEM_ENABLE (prefill OOM)

functionstackx
and
claude
committed
4370988
Revert "fix: expandable_segments allocator, drop NCCL_CUMEM_ENABLE (prefill OOM)"

functionstackx
and
claude
committed
0c5fe11
feat: add VLLM_PREFIX_CACHE_RETENTION_INTERVAL=32768

functionstackx
and
claude
committed
e675dd2
feat: widen agentic conc list to 1/8/16/32

functionstackx
and
claude
committed
4b0c3a4
fix: remove VLLM_PREFIX_CACHE_RETENTION_INTERVAL (K3 scheduler block)

functionstackx
and
claude
committed
479b74b

Commits on Jul 28, 2026

feat: agentic experiment E — SimpleCPUOffloadConnector CPU DRAM KV offload

functionstackx
and
claude
committed
77988f7
docs: point changelog and MODELS rows at experiment PR #2370

functionstackx
and
claude
committed
833d4e8
fix: disable cross-layer blocks in SimpleCPUOffloadConnector (K3 hybrid)

functionstackx
and
claude
committed
05eae6f
fix: expandable_segments on the SimpleCPU-offload variant (prefill OOM)

functionstackx
and
claude
committed
b99c9ce

Navigation Menu

Uh oh!

[Klaud Cold][agentic experiment][Variant E] Kimi-K3 B200 agg TP8xPP2 agentic — direct vllm serve + SimpleCPU KV offload / Kimi-K3 B200 聚合式 TP8xPP2 智能体实验——直接 vllm serve + SimpleCPU KV 卸载 - #2370

Commits on Jul 27, 2026

feat: add Kimi-K3 MXFP4 B200 aggregated TP8xPP2 Dynamo-vLLM agentic recipe

docs: link PR #2355 in changelog entry and MODELS rows

fix: pin agentic srt-slurm to NVIDIA v1.0.36, dynamo 1.2.1, conc-8 smoke test

fix: drop OpenAI-frontend tool-choice flags from dynamo-vllm worker args

fix: try dynamo wheel 1.2.0.dev20260426 for Kimi-K3 frontend tokenizer

fix: pin dynamo to day-zero Kimi-K3 commit, restore kimi_k3 parser flags

fix: use dynamo namespaced --dyn-* kimi_k3 parser args on the worker

fix: disable aiperf conv-aware routing (session_control 400-rejected)

fix: patch kimi-k3 image mamba_hybrid index_fill_ dtype via setup_script

feat: agentic experiment D — direct vllm serve via srt-slurm PR #278

docs: point changelog and MODELS rows at experiment PR #2359

fix: drop gpu-memory-utilization to 0.90 (flashinfer MoE workspace OOM)

fix: expandable_segments allocator, drop NCCL_CUMEM_ENABLE (prefill OOM)

Revert "fix: expandable_segments allocator, drop NCCL_CUMEM_ENABLE (prefill OOM)"

feat: add VLLM_PREFIX_CACHE_RETENTION_INTERVAL=32768

feat: widen agentic conc list to 1/8/16/32

fix: remove VLLM_PREFIX_CACHE_RETENTION_INTERVAL (K3 scheduler block)

Commits on Jul 28, 2026

feat: agentic experiment E — SimpleCPUOffloadConnector CPU DRAM KV offload

docs: point changelog and MODELS rows at experiment PR #2370

fix: disable cross-layer blocks in SimpleCPUOffloadConnector (K3 hybrid)

fix: expandable_segments on the SimpleCPU-offload variant (prefill OOM)