[FEAT]: Multilingual Native Embedder

[timothycarambat]

What would you like to see?

Currently, the built-in embedder uses the ONNX all-MiniLM-L6-v2 embedder, which does okay for most use cases and is much smaller to download.

There should be support for the larger multilingual-e5-large model (ONNX HERE) for multi-lingual support.

This should not be the default, but it should be something the user can opt to select. They may have to wait for the download to completely download for the embedder change to be saved as we cannot afford the latency to download the model at runtime.

[timothycarambat]

Also, we don't want to pre-pack the docker image with models people may not use, so we will not be doing that in the future to keep the docker image portable enough for a reasonable size.

[vlbosch]

I would also like the option to add another local embeddings model, like for example BGE-M3. I tried adding it in the models-folder myself, but couldn't get it to work yet, unfortunately. Hopefully this feature can be added on the short term, so that we don't need to really on OpenAI's models for multilingual documents. Thanks in advance! :-)

[oscar-7000]

bge-m3 would be nice

[sweco-nlmdek]

This would be a very welcome feature. i see in this thread : #645 someone tried multilingual-e5-large and it seems to help allot.

[Herz3h]

Is there a manual way to use a multilingual embedding model in the mean time ? or one from sentence-transformers?

[timothycarambat]

@Herz3h, yes, we support Ollama or LMStudio for embedder endpoints, where you can use any embedder you like to fit your use case.

[Herz3h]

Thanks, however neither of ollama/lmstudio seem to support sentence-transformers, is there a way to still use them? since some of them rank very high in https://huggingface.co/spaces/mteb/leaderboard

[timothycarambat]

Ollama and LMstudio both support embedders - you need to use the GGUF version since they cannot run tensorflow or pytorch models.

Ollamas select is certainly more limited since it relies on their registry, but you can import HF models directly into Ollama.
Example: https://huggingface.co/models?library=gguf&sort=downloads&search=embedding
Click on any of those and you will see the ability to pull it in Ollama or LMStudio.

Both of which support /embedding - so it works.

[Herz3h]

Oh didn't know there was an option from huggingface to pull anymodel in ollama. Thank you very much !