Docker: Embedding Issue & possible Fix

[adrianliechti]

When running the Docker Image (CPU or CUDA) and run an embedding model, I get this error

nomic-embed-text-1  | terminate called after throwing an instance of 'std::runtime_error'
nomic-embed-text-1  |   what():  locale::facet::_S_create_c_locale name not valid

When installing the locale pack on start with something like this, everythings works as expected

nomic-embed-text:
    image: ghcr.io/ggerganov/llama.cpp:server
    pull_policy: always
    entrypoint: ""
    command: /bin/bash -c "apt-get update && apt-get install locales && locale-gen en_US.UTF-8 && update-locale && /server --host 0.0.0.0 --port 8000 --log-disable --ctx-size 8192 --embedding --model /models/nomic-embed-text-v1.5.Q4_K_M.gguf"
    volumes:
      - ../../models:/models

curl http://localhost:8080/oai/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Hello!",
    "model": "nomic-embed-text"
  }'

maybe it is worth to install / update the locals in the base image (or change the code to adapt to invariant-culture?)
https://stackoverflow.com/questions/28405902/how-to-set-the-locale-inside-a-debian-ubuntu-docker-container

[phymbert]

Hello,

Yes I have fixed the server test but I did not update the public docker image.

Need to add language-pack-en AFAIK.

https://github.com/ggerganov/llama.cpp/blob/ddf65685105a39a57b1e7f80c3aa502a6313af24/.github/workflows/server.yml#L63

Please open a PR.

[ngxson]

Out of curiosity, but why do you need to change the locale of the container? Sorry I didn't read carefully, yes seems like this is a common problem

[javi22020]

Same problem here, did anyone fix it?

[time-less-ness]

I was having this issue compiling from source on Debian (GCP VM created today). When I git pull and get latest main commits then recompile, I do not get the error anymore.

[fyun89]

Just wanted to give a quick stab at this -- would appreciate someone with Nvidia or Intel hardware to test. I unfortunately have a Apple Silicon machine.