Summary
GenieX currently only supports generative models (text-generation / VLM) — there's no way to run an embedding model to get vectors. Would be great to add embedding support.
Tried LiquidAI/LFM2.5-Embedding-350M (Q4_0 GGUF, lfm2 arch) on WoS ARM64 (X Elite), CLI v0.3.13:
pull detects it as llm
infer forces generation → degenerate output (password password password...)
serve's POST /v1/embeddings → 404 (server only has /v1/models and chat completions)
The same GGUF returns proper 1024-dim vectors via upstream llama.cpp's llama-embedding, and cosine similarity behaves correctly — so the model is fine and the lfm2 arch is already embeddable; GenieX just doesn't expose the path.
The underlying llama_cpp runtime should already have this (llama.cpp's llama_get_embeddings). Would be nice to:
- Detect embedding models on
pull
- Add a
/v1/embeddings endpoint (and/or a CLI/SDK embed API)
On-device RAG / semantic search is a common use case. Thanks!
Summary
GenieX currently only supports generative models (text-generation / VLM) — there's no way to run an embedding model to get vectors. Would be great to add embedding support.
Tried
LiquidAI/LFM2.5-Embedding-350M(Q4_0 GGUF, lfm2 arch) on WoS ARM64 (X Elite), CLI v0.3.13:pulldetects it asllminferforces generation → degenerate output (password password password...)serve'sPOST /v1/embeddings→ 404 (server only has/v1/modelsand chat completions)The same GGUF returns proper 1024-dim vectors via upstream llama.cpp's
llama-embedding, and cosine similarity behaves correctly — so the model is fine and the lfm2 arch is already embeddable; GenieX just doesn't expose the path.The underlying
llama_cppruntime should already have this (llama.cpp'sllama_get_embeddings). Would be nice to:pull/v1/embeddingsendpoint (and/or a CLI/SDK embed API)On-device RAG / semantic search is a common use case. Thanks!