[Feature]: I’d like to be able to use models via API in addition to locally running models

[qdrddr]

I'd like to be able to utilize
OCR, Embeddings, and Reranking models via API with LiteLLM Client SDK.

This should improve accuracy when used with SotA models such as Qwen3-Embedding-8B and Qwen3-Rerank

[logan-markewich]

For now, embeddings are not configurable, and I plan to keep it that way. We use static embeddings to keep things very fast, and embed line-by-line to work with the limits of static embeddings. This provides a very good "fuzzy keyword search"

Reranking isn't really implemented. I don't really have any plans for this. It'd be nice to run some colbert-like model for reranking. If there's one that runs in rust I could look into it.

OCR is not configurable at the moment, but I have some plans to change this in the future.

[qdrddr]

To my understanding this project has implemented reranking with ColBert in rust. So maybe it'll be useful for you @logan-markewich

https://github.com/lightonai/next-plaid