Skip to content

feat(docker): overlay mooncake structured object store module pending release#1784

Open
guapisolo wants to merge 3 commits into
mainfrom
mooncake-sos-overlay
Open

feat(docker): overlay mooncake structured object store module pending release#1784
guapisolo wants to merge 3 commits into
mainfrom
mooncake-sos-overlay

Conversation

@guapisolo

Copy link
Copy Markdown
Collaborator

Summary

Ship the mooncake structured object store API in the CUDA image for the mooncake rollout backend.

Motivation

miles' mooncake rollout-data object store backend needs the structured
object store API (FieldSchema, export_ref/import_ref, unified put/get,
release_result) that is merged upstream in kvcache-ai/Mooncake #2907,
#3013 and #3023 but missing from every released wheel: the v0.3.12
release branch was cut before those merges landed.

Usage

docker build -f docker/Dockerfile .
# inside the built image, the API the mooncake object-store backend imports resolves:
python3 -c "from mooncake.structured_object_store import FieldSchema, MooncakeBundleTransfer, export_ref, import_ref"

This unblocks --object-store-backend mooncake (PR #591) on images built from this Dockerfile.

Design Notes

  • Key choices: overlay the single pure-Python module structured_object_store.py at pinned upstream commit 4dbe5a4c over the installed package.
  • The upstream API commits touch only that module.
  • The module runs unchanged on the C++ core the sglang base image already ships.
  • The new RUN layer import-checks every required symbol, so an incomplete overlay fails the image build.
  • Alternatives considered: bumping to the v0.3.12 wheel was rejected because its release branch predates the API merges.
  • Rebuilding mooncake from source was rejected as too heavy for a temporary gap that one release will close.
  • docker/Dockerfile.rocm is unchanged: whether its base image ships mooncake is unverified.

Verification

  • Tests added: none; the RUN layer's import check gates every image build.
  • Local round-trip with the overlaid module passes against the exact wheel the current image ships (mooncake-transfer-engine-cuda13 0.3.11.post1).
  • The round-trip covers put with field schemas, ref export/import, get, release_result, cleanup_dataproto over tcp against a local mooncake_master.

Review Focus

  • Scrutinize the MOONCAKE_SOS_COMMIT pin: it must cover every symbol miles/utils/object_store.py imports when PR [feat]support rollout data transfer with mooncake #591 lands.
  • Scrutinize the import-check line in the new RUN layer; it is the only gate keeping an incomplete overlay out of runtime.
  • Scrutinize the drop condition: the layer comment binds removal to the first mooncake release containing the API.

miles' mooncake rollout-data object store backend needs the structured
object store API (FieldSchema, export_ref/import_ref, unified put/get,
release_result) that is merged upstream in kvcache-ai/Mooncake #2907,
#3013 and #3023 but missing from every released wheel: the v0.3.12
release branch was cut before those merges landed. The API is a single
pure-Python module compatible with the C++ core the sglang base image
already ships, so fetch that module at a pinned master commit over the
installed package instead of rebuilding mooncake from source. The layer
import-checks every required symbol so an incomplete overlay fails the
image build instead of surfacing at runtime.

Verified against mooncake-transfer-engine-cuda13 0.3.11.post1 (the
version the current image ships): with the overlaid module, the full
put(field_schemas) -> export_ref/import_ref -> get -> release_result ->
cleanup_dataproto round-trip passes over tcp against a local
mooncake_master. Drop this layer once a mooncake release contains the
API.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

guapisolo and others added 2 commits July 24, 2026 07:34
…dule

Supersedes the raw-file overlay: the pinned structured object store now
ships as a real wheel built by the miles-wheels `mooncake` step
(yueming-yuan/miles-wheels#8), and the wheels-release download already
lands it in /tmp/wheels. Installing the whole wheel replaces the base
image's identically named dist, so the C++ core, mooncake_master, and
the Python API stay one pinned build instead of a hybrid of 0.3.11
binaries with a newer overlaid module.

The layer requires the wheel on cu13/amd64 (the variant miles CI runs
mooncake on) and fails the build when the release lacks it; other
(cuda, arch) variants log and keep the base image's mooncake until
their wheel is built. The import self-check stays as the fail-loud
gate for an incomplete wheel.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The per-variant gating is readable from the shell branches; keep the
comment to the why and the drop condition.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants