feat(mooncake): manage standalone store services

[00fish0]

Motivation

In a typical disaggregated SGLang deployment, prefill and decode nodes have different host-memory usage patterns.
Prefill nodes usually enable HiCache. Its L2 cache consumes part of the host DRAM, leaving less memory available for Mooncake Store. Decode nodes normally do not enable HiCache, so more of their remaining allocatable DRAM can be assigned to Mooncake Store.

As a result, the two roles need different Store capacities. For example:

• Prefill: HiCache L2 plus a smaller Mooncake Store segment.
• Decode: no HiCache L2 and a larger Mooncake Store segment.

The existing embedded-client configuration is insufficient for this deployment because the same Mooncake environment is injected into all inference workers. Running standalone Store services allows prefill and decode nodes to use separate MOONCAKE_GLOBAL_SEGMENT_SIZE values.

This also makes memory ownership explicit: the SGLang worker initializes its embedded Mooncake clients without contributing another global DRAM segment, while the standalone Store service owns the role-specific DRAM allocation.

Summary

This PR extends the existing Mooncake integration for both SGLang and vLLM by:

• Adding managed standalone Mooncake Store services on prefill, decode, or aggregated worker nodes.
• Allowing role-specific Store memory sizes, environment variables, and command-line arguments.
• Adding configurable mooncake_master tuning arguments through mooncake_kv_store.master_args.
• Showing the resolved Mooncake configuration in dry-run output.
• Documenting the configuration and lifecycle behavior.

Standalone Store services

A recipe can launch one standalone Store service per selected physical worker node:

backend:
  type: sglang

  prefill_environment:
    MOONCAKE_GLOBAL_SEGMENT_SIZE: "0"
  decode_environment:
    MOONCAKE_GLOBAL_SEGMENT_SIZE: "0"

  mooncake_kv_store:
    standalone:
      command: [python, -m, mooncake.mooncake_store_service]
      args: [--port, "8800", --max-wait-time, "120"]
      env:
        MOONCAKE_PROTOCOL: rdma
        MOONCAKE_DEVICE: "mlx5_0,mlx5_1"
        MC_GID_INDEX: "3"
        MOONCAKE_LOCAL_BUFFER_SIZE: "0"
      placements:
        prefill:
          env:
            MOONCAKE_GLOBAL_SEGMENT_SIZE: "100gb"
        decode:
          env:
            MOONCAKE_GLOBAL_SEGMENT_SIZE: "400gb"
      health_check:
        port: 8800
        timeout_seconds: 120

The services are started after mooncake_master becomes ready and before inference workers are launched.

Configurable master arguments

Recipes can now customize Mooncake master tuning parameters:

mooncake_kv_store:
  master_args:
    - --eviction_high_watermark_ratio=0.9
    - --default_kv_lease_ttl=10000
    - --rpc_thread_num=16

When master_args is specified, it replaces the default tunable argument list. Setting it to [] uses Mooncake's own defaults.

The RPC port, HTTP metadata server, and metrics endpoint remain managed by srt-slurm.

This also removes the unsupported --nof_eviction_high_watermark_ratio=0.9 argument from the default master command.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[ishandhanani]

pls rebase

[KaunilD]

Hi please rebase.