LLM and embeddings

[thebrahman]

is it possible to run an LLM and embedding model simultaneously from the same docker container?

for example:

[9876691]

Yes. I'm currently running both in one container.

So when I run

curl http://localhost:8080/v1/models

You can see I have 2 models.

{"object":"list","data":[{"id":"text-embedding-ada-002","object":"model"},{"id":"ggml-gpt4all-j","object":"model"}]}

Then when calling the embeddings API I set the model to text-embedding-ada-002

[BradKML]

Wait is that part of the UX of LocalAI or some other software?