Publish container images to GHCR from GitHub Actions (cuda, vulkan, cpu; rocm blocked; metal/mlx not containerizable)

[localai-bot]

Follow-up to the downloadable-binary work (#117, spec .agents/specs/release-binary-matrix.md, PR #141). An archive you have to download, unpack and match against a host ABI is the right primitive, but for most people the fastest path to a running server is docker run. This issue tracks publishing container images to GHCR from GitHub Actions.

Tracked as ROAD-V1-CONTAINERS / ENG-RELEASE-CONTAINERS, currently INVENTORIED. Nothing exists yet — no Dockerfile for the server, no workflow, no registry package. Per POL-SPIKE-FIRST the row cannot move until .agents/specs/container-images.md is written and accepted.

The image is the bundle

The container is not a second packaging format. It ships exactly the staged tree the release-binary matrix already defines:

bin/vllm-server          # linked against the static vllm core
VERSION
manifest (machine-readable: compiled CPU tiers / CUDA SMs)
SHA256, SPDX JSON SBOM, build-provenance attestation, third-party licenses

ENTRYPOINT ["vllm-server"]. No model weights, no tokenizer assets, no Python, no PyTorch, no Triton runtime, no compiler, no source or build directory. If the binary lane and the image lane grow separate layouts, that's a bug — they share one staging tree.

Lanes

Lane	Channel	Notes
cuda	with the binary lane's CUDA channel	One fat image covering every supported SM (80, 86, 87, 89, 90a, 100a, 103a, 110, 120a, 121a). Per-SM images, if any, are diagnostics — never the primary pull.
vulkan	preview	Mirrors the Vulkan binary channel. Needs the loader + ICD in the image; the GPU driver stays on the host.
cpu	stable-after-runtime-gate	One adaptive binary, conservative baseline (portable SSE2 on x86_64, no AVX2 assumption), runtime ISA dispatch.
rocm	blocked	Tracks its binary channel. The ROCm skeleton has never compiled on AMD hardware (#41, #132), so this lane is recorded, not built.

Metal and MLX get no images. There is no macOS container runtime and no Metal passthrough into a Linux VM, so these are not "not yet" — they are not possible. They stay static-binary-only lanes. Recorded as a boundary so nobody re-opens it as pending work.

Architectures

Every lane is a multi-arch manifest: linux/amd64 + linux/arm64, built on native runners, not QEMU.

arm64 is not an afterthought here. The project's own gate hardware is aarch64 — DGX Spark GB10 (sm_121a), Jetson AGX Thor (sm_110, #168), Orin (sm_87). A CUDA image that only exists for x86_64 misses the machines this project is actually measured on, and emulated CUDA builds don't finish in a runner's lifetime.

Tags

ghcr.io/mudler/vllm.cpp:v0.1.0-cuda
ghcr.io/mudler/vllm.cpp:v0.1.0-vulkan
ghcr.io/mudler/vllm.cpp:v0.1.0-cpu
ghcr.io/mudler/vllm.cpp:latest-cuda
ghcr.io/mudler/vllm.cpp:latest          -> cpu

One package, lane in the tag. Immutable version tags; latest-* moves.

Host boundary

The GPU driver and the container runtime are the host's, never the image's. CUDA images assume an NVIDIA driver plus nvidia-container-toolkit on the host; Vulkan images assume a host ICD. The image carries userspace runtime only, and the README must say so plainly — this is the single most common source of "it doesn't see my GPU" reports.

What the spike has to settle

• Base image per lane, and the resulting size/CVE surface (nvidia/cuda:*-runtime vs a slimmer hand-rolled base)
• Where the build happens: reuse the binary lane's staged artifact vs building inside the Dockerfile
• Multi-arch manifest assembly across two native runner jobs
• Provenance and SBOM attachment (cosign / GitHub attestations) so an image is verifiable the way an archive is
• Retention: untagged/PR images must not accumulate forever
• Least-privilege publish: packages: write scoped to the publish job only, no long-lived tokens, tag-triggered
• Whether PR builds push at all, or only build

Non-goals

Bundling weights. Shipping Python or a compiler. A "dev" image with the toolchain. Widening any backend, model, quantization or kernel support claim — an image lane exists only where the corresponding backend already has runtime evidence.