…NTAINERS)
FOLLOWING_AGENTS_PROTOCOL
Spike spec plus W1-W5 and W7 of ENG-RELEASE-CONTAINERS. The cpu lane is built
and gated end to end on a real image; vulkan builds and validates on hosted CI;
cuda is implemented and gated statically but has never been built. NOTHING is
published: the GHCR package does not exist and no lane has matching-hardware
runtime evidence (W6).
One docker/Dockerfile with three lane targets. Every builder stage calls the
SAME scripts/build-*-release.sh the release workflow calls, so the ten-SM
gencode list, the CPU tier gates and the staged layout are written down once.
Those scripts already end in validate-release-archive.py, so an image build
inherits the extracted-archive audit rather than reimplementing it -- which is
why the builders carry file/binutils, qemu-user and a SHA256-pinned Intel SDE,
and why the vulkan builder needs a software ICD: each is a real requirement of
the gate being inherited. `-j 2` was hardcoded in both release scripts and is
now a JOBS parameter.
New gates, each with a red-first mutation suite:
- check-container-matrix.py keeps release/container-matrix.json and the
Dockerfile agreeing on lanes, tags, digest-pinned bases and the driver
boundary (31/31). It scans RUN/COPY/ADD only, so the image can still DOCUMENT
in a comment and a label that libcuda.so.1 comes from the host.
- check-container-workflow.py holds containers.yml to its stages: no registry
write in the job a pull request runs, OIDC only in attest, publish tag-gated,
and validation BEFORE the push (28/28, ordering included).
- validate-container-image.py audits config and layout and, given a model,
boots the server and requires /health, /version, the image's own declared
healthcheck, and a clean SIGTERM.
MEASURED (x86_64, Docker 29.1.2, opt-125m-bf16-st): 783 MB cpu image, config +
layout + boot all green. Hosted CI: plan, verify(cpu, amd64) and
verify(vulkan, amd64) green, publish stages correctly skipped on a non-tag.
The boot gate found a real bug on its first run, which is why it boots the
server instead of running --help (#312, fixed here): vllm-server installed no
SIGTERM handler, and as container PID 1 the kernel does not apply default
signal dispositions, so the signal was IGNORED -- docker stop waited its full
30 s grace and then SIGKILLed, exit 137. A self-pipe handler now routes SIGTERM
and SIGINT into the same server.stop() the VT_BENCH_PROFILE_CONTROL FIFO
shutdown already used, at all three listen() sites. RED 137 after 30 s -> GREEN
exit 0 in 0.25 s. Every rolling update and compose down was hard-killing the
server; it was invisible outside a container because a non-PID-1 process dies
on the default disposition anyway.
Three further bugs, none of which could fail anywhere but in a real release:
- .dockerignore's `**/build*/` matched FILES as well as directories, because
Docker does not honour a trailing slash the way .gitignore does, so
scripts/build-*-release.sh were excluded from the build context.
- `docker push --quiet` prints the image REFERENCE, not the digest, so the
manifest stage would have joined garbage.
- The immutable-tag guard ran BEFORE login, where imagetools inspect fails for
lack of auth -- read as "tag does not exist", a fail-open that would have
overwritten an immutable version tag.
Pull requests build the lanes that opt in (cpu and vulkan on amd64) rather than
all six, because a ten-SM fat CUDA image does not fit a hosted runner budget per
push. That is a cost decision and NOT a publication hole: publish consumes the
full release matrix and revalidates each lane immediately before pushing it, and
the guard enforces that split.
scripts/check-pr-size.py FAILS CLOSED on unclassified paths, so docker/Dockerfile,
docker/healthcheck.sh and release/container-matrix.json had to be classified
before this change could pass the project's own size gate. Both checker pins
touched here gain a case locking why the pin is justified, not just its value.
The two red sanitizer lanes are NOT from this change: reproduced on pristine
origin/main, root-caused to PromRegistry::SeriesFor returning a Series& into a
std::vector it then push_backs into, and filed as #330 for the metrics row.
Issues: #170, #312, #330
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [Claude Code]
Implements
ENG-RELEASE-CONTAINERS— spike spec plus W1-W5 and W7. Closes #170 (image lanes), closes #312 (the bug the gate found).The cpu lane is built and gated end to end on a real image. Nothing is published: the GHCR package does not exist, and no lane has matching-hardware runtime evidence — that is W6 and it stays open.
What is here
One
docker/Dockerfile, three lane targets. Every builder stage calls the samescripts/build-*-release.shthe release workflow calls, so the ten-SM gencode list, the CPU tier gates and the staged layout are written down exactly once. Those scripts already end invalidate-release-archive.py, so an image build inherits the extracted-archive audit rather than reimplementing it.Runtime is
ubuntu:24.04pinned by digest for every lane; the cuda lane copies only the CUDA runtime libraries it links, and the driver stays on the host.ffmpegships in all three lanes — a deliberate, argued divergence from the archive contract's no-vendor rule, because an archive is extracted onto a host with aPATHto inherit and a container has none.Three new gates, each with a red-first mutation suite:
check-container-matrix.pycheck-container-workflow.pyvalidate-container-image.pyMeasured
x86_64, Docker 29.1.2, booted on
opt-125m-bf16-st:/health200,/version200, the image's own declared healthcheck passing inside the container, clean SIGTERMThe gate found a real bug on its first run (#312)
This is why the smoke boots the server instead of running
--help.vllm-serverinstalled noSIGTERMhandler. As container PID 1 the kernel does not apply default signal dispositions, so the signal was ignored outright:docker stopwaited its full 30 s grace, thenSIGKILL— exit 137. Every rolling update,compose downand systemd restart was hard-killing the server and dropping in-flight work. It was invisible outside a container because a non-PID-1 process dies on the default disposition anyway.Fixed with a self-pipe handler routing
SIGTERM/SIGINTinto the sameserver.stop()the existingVT_BENCH_PROFILE_CONTROLFIFO shutdown already used, installed at all threelisten()sites.RED: exit 137 after 30 s → GREEN: exit 0 in 0.25 s.
Also fixed, and silent until now
.dockerignore's**/build*/matched files as well as directories — Docker does not honour a trailing slash as directories-only the way.gitignoredoes — soscripts/build-*-release.shwere excluded from the build context and the image build failed withnot foundafter appearing to copy the whole tree. The builders also neededfileandbinutils, without which the inherited archive validator failed after a full compile.-j 2was hardcoded in both release scripts and is now aJOBSparameter.Not established, and not claimed
cudaandvulkanare gated statically but have never been built here.sm_110, Orinsm_87) is untouched.Records
Row →
ACTIVEwith anchors and a claim.docs/STATUS.mdlands 1 byte under its ratchet, which is lowered in this change with the reason attached. FEATURES gains container and graceful-shutdown rows; BENCHMARKS gains the image size and the shutdown number; USAGE documents pulling, running and validating an image.scripts/agent-preflight.shandscripts/agent-ready.pyare green on the merged tree, and the image was rebuilt and re-gated after merging main, since main touchedserver_main.cpp.🤖 Generated with Claude Code