[BUG]: Backend Error when detecting Ollama instances

[t73biz]

How are you running AnythingLLM?

Docker (local)

What happened?

I am running anythingllm within a docker container on an Ubuntu 24 instance with ollama v0.5.7
The local ollama instance is accessible at http://localhost:11434

When running the following command to stand up the docker instance, (I left out the -d param to be able to debug), I get an error in the console.

Run Command:

export STORAGE_LOCATION=$HOME/AnythingLLMDesktop/storage && \
mkdir -p $STORAGE_LOCATION && \
touch "$STORAGE_LOCATION/.env" && \
docker run -p 3001:3001 \
--cap-add SYS_ADMIN \
--add-host=host.docker.internal:host-gateway \
-v ${STORAGE_LOCATION}:/app/server/storage \
-v ${STORAGE_LOCATION}/.env:/app/server/.env \
-e STORAGE_DIR="/app/server/storage" \
mintplexlabs/anythingllm

Error:

[backend] error: TypeError: fetch failed
    at node:internal/deps/undici/undici:12625:11
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ollamaAIModels (/app/server/utils/helpers/customModels.js:346:18)
    at async getCustomModels (/app/server/utils/helpers/customModels.js:49:14)
    at async /app/server/endpoints/system.js:960:35

Are there known steps to reproduce?

Visit the http://localhost:3001/settings/llm-preference endpoint and select Ollama as the LLM Provider

[t73biz]

ok after quite a bit of fussing, this was the command that allowed for ollama to be access from the host through to the docker container.

export STORAGE_LOCATION=$HOME/AnythingLLMDesktop/storage && \
mkdir -p $STORAGE_LOCATION && \
touch "$STORAGE_LOCATION/.env" && \
docker run --rm -d -p 3001:3001 \
--cap-add SYS_ADMIN \
--network=host \
-v ${STORAGE_LOCATION}:/app/server/storage \
-v ${STORAGE_LOCATION}/.env:/app/server/.env \
-e STORAGE_DIR="/app/server/storage" \
mintplexlabs/anythingllm

The difference here is the --network=host \ and then I was able to run with the ollama config url value of http://localhost:11434

I'm not sure if this is secure for production. My gut says no, but it does work locally

[timothycarambat]

That is the correct solution, otherwise the local network loopback will keep using the container network and not the host network (where Ollama is running)