Skip to content

Support embedding models #1149

Description

@ppl520zfl

Summary

GenieX currently only supports generative models (text-generation / VLM) — there's no way to run an embedding model to get vectors. Would be great to add embedding support.

Tried LiquidAI/LFM2.5-Embedding-350M (Q4_0 GGUF, lfm2 arch) on WoS ARM64 (X Elite), CLI v0.3.13:

  • pull detects it as llm
  • infer forces generation → degenerate output (password password password...)
  • serve's POST /v1/embeddings → 404 (server only has /v1/models and chat completions)

The same GGUF returns proper 1024-dim vectors via upstream llama.cpp's llama-embedding, and cosine similarity behaves correctly — so the model is fine and the lfm2 arch is already embeddable; GenieX just doesn't expose the path.

The underlying llama_cpp runtime should already have this (llama.cpp's llama_get_embeddings). Would be nice to:

  1. Detect embedding models on pull
  2. Add a /v1/embeddings endpoint (and/or a CLI/SDK embed API)

On-device RAG / semantic search is a common use case. Thanks!

Metadata

Metadata

Labels

No labels
No labels

Fields

No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions