What
vllm-server installs no SIGTERM handler, so it never shuts down cleanly on the signal every process supervisor sends first.
In a container this is not merely untidy, it is total: the server is PID 1, and the kernel does not apply default signal actions to PID 1. An unhandled SIGTERM is therefore ignored outright, so docker stop waits the full grace period and then SIGKILLs.
Evidence
Measured against the new cpu container image (row/ENG-RELEASE-CONTAINERS, PR #307), booted on opt-125m-bf16-st:
/health -> 200, /version -> 200, declared HEALTHCHECK passesdocker stop --timeout 30 -> exit 137 (SIGKILL), after a full 30 second wait
scripts/validate-container-image.py fails the image for this reason.
Impact
Every ordinary orchestrator action — a Kubernetes rolling update, docker compose down, a systemd restart — hard-kills the server after a pointless wait, dropping in-flight requests. Outside containers the process does die on SIGTERM by default, but still without draining.
Fix
Install a SIGTERM/SIGINT handler that stops the HTTP server through the same path the existing VT_BENCH_PROFILE_CONTROL FIFO shutdown already uses (server.stop()), via a self-pipe so the handler itself stays async-signal-safe.
Found while implementing ENG-RELEASE-CONTAINERS; the container gate is what surfaced it.
What
vllm-serverinstalls noSIGTERMhandler, so it never shuts down cleanly on the signal every process supervisor sends first.In a container this is not merely untidy, it is total: the server is PID 1, and the kernel does not apply default signal actions to PID 1. An unhandled
SIGTERMis therefore ignored outright, sodocker stopwaits the full grace period and thenSIGKILLs.Evidence
Measured against the new
cpucontainer image (row/ENG-RELEASE-CONTAINERS, PR #307), booted onopt-125m-bf16-st:/health-> 200,/version-> 200, declared HEALTHCHECK passesdocker stop --timeout 30-> exit 137 (SIGKILL), after a full 30 second waitscripts/validate-container-image.pyfails the image for this reason.Impact
Every ordinary orchestrator action — a Kubernetes rolling update,
docker compose down, a systemd restart — hard-kills the server after a pointless wait, dropping in-flight requests. Outside containers the process does die on SIGTERM by default, but still without draining.Fix
Install a
SIGTERM/SIGINThandler that stops the HTTP server through the same path the existingVT_BENCH_PROFILE_CONTROLFIFO shutdown already uses (server.stop()), via a self-pipe so the handler itself stays async-signal-safe.Found while implementing
ENG-RELEASE-CONTAINERS; the container gate is what surfaced it.