Handle SIGTERM

[gb-beng]

It would be great if the worker/container would gracefully terminate on SIGTERM. Kubernetes can send it e.g. when draining nodes.

---

To test this I used:

docker run \
  --rm \
  --detach \
  --env W_MODEL=tiny \
  --env W_DEVICE=cpu \
  --env MODEL_BASE_DIR=/model \
  --env DATA_BASE_DIR=/data \
  --name whisper-asr-worker ghcr.io/beeldengeluid/whisper-asr-worker:git-85fa5e3 --service=y
# d0378c37991498afe11038fa8528686cddb754d5ee7caa38fec042b93b75888a
docker ps
# CONTAINER ID   IMAGE                                                  COMMAND                  CREATED          STATUS          PORTS     NAMES
# d0378c379914   ghcr.io/beeldengeluid/whisper-asr-worker:git-85fa5e3   "./docker-entrypoint…"   4 seconds ago    Up 3 seconds              whisper-asr-worker
docker container kill --signal=SIGTERM whisper-asr-worker
# whisper-asr-worker
sleep 120 && docker ps
# CONTAINER ID   IMAGE                                                  COMMAND                  CREATED          STATUS              PORTS     NAMES
# d0378c379914   ghcr.io/beeldengeluid/whisper-asr-worker:git-85fa5e3   "./docker-entrypoint…"   2 minutes ago    Up About a minute             whisper-asr-worker

Note that SIGINT does work.

[greenw0lf]

@gb-beng would it be possible to use docker stop whisper-asr-worker instead of the kill command? From what I could find, it seems like the standard approach as it first sends SIGTERM, then after 10s it sends SIGKILL. But if kubernetes handles these operations through docker kill, then I'll keep investigating how to handle it because it is taking longer than expected to find a solution

Also, using docker container kill but with the SIGINT signal doesn't work for me (I'm not sure how you tested that)

[gb-beng]

@gb-beng would it be possible to use docker stop whisper-asr-worker instead of the kill command? From what I could find, it seems like the standard approach as it first sends SIGTERM, then after 10s it sends SIGKILL.

Kubernetes doesn't use docker commands to send these signals, I just used it for testing/illustration. There is an equivalent to docker stop, but the point is that I'd like to gracefully stop the process instead of killing it.

I'll keep investigating how to handle it because it is taking longer than expected to find a solution

I wouldn't spend too much time on it now, let's focus on the distributed/scalable deployment first, as it will affect this issue as well.

Also, using docker container kill but with the SIGINT signal doesn't work for me (I'm not sure how you tested that)

You're right SIGINT also doesn't work...