[KV Offload]: Tiered offloading + MoonCakeStore#2689

Merged

samsja merged 7 commits into

mainfrom

Jun 2, 2026

Commits on Jun 2, 2026

feat(inference): tiered KV cache offload (native + mooncake, CPU/disk)

S1ro1
and
claude
committed
chore(inference): drop unused mooncake disk num_bytes and metadata_server

S1ro1
and
claude
committed
Merge branch 'main' into mooncake-baby
S1ro1
authored
refactor(inference): launch Mooncake store in sbatch bash; add CHANGELOG

S1ro1
and
claude
committed
feat(inference): Mooncake shared distributed KV pool across nodes

S1ro1
and
claude
committed
Merge branch 'main' into mooncake-baby
S1ro1
authored
chore(inference): drop runtime mooncake-requires-slurm check

S1ro1
and
claude
committed