Vector embeddings over a tiny HTTP contract.
On-device ONNX or any OpenAI-compatible API. The reference /embed server for CPersona.
Standalone repository — extracted from the (now private)
clotohub-serversmonorepo so it can be used on its own. ClotoCore users get this through the in-app marketplace (ClotoHub); everyone else can run it directly as described below.
A small server that turns text into vectors. It speaks a minimal HTTP contract so anything can call it — its primary consumer is CPersona, whose hybrid search uses it for the vector-similarity layer. It can run a model on-device via ONNX (no API key, no network) or proxy an OpenAI-compatible API.
It also exposes an MCP (stdio) surface and an optional persistent vector index (/index, /search), but the HTTP /embed endpoint is all CPersona needs.
POST /embed
Request: { "texts": ["string", ...] } # non-empty array, max 100 per batch
Response: { "embeddings": [[float, ...], ...], "dimensions": <int> }
Point any client (e.g. CPersona's CPERSONA_EMBEDDING_URL / generic EMBEDDING_HTTP_URL) at http://127.0.0.1:8401/embed.
Prerequisites: Python 3.10+
# Download a model into ./data/models (jina-v5-nano is what CPersona is tuned for)
uvx --from "cembedding[onnx]" cembedding-download-model --model jina-v5-nano
# Run the server (reads ./data/models from the current directory)
EMBEDDING_PROVIDER=onnx_jina_v5_nano uvx --from "cembedding[onnx]" cembeddingOr install it onto your PATH with pip install "cembedding[onnx]", then run
cembedding-download-model --model jina-v5-nano and cembedding.
From source (development):
git clone https://github.com/Cloto-dev/CEmbedding.git
cd CEmbedding
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install ".[onnx]"
python -m cembedding.download_model --model jina-v5-nano
EMBEDDING_PROVIDER=onnx_jina_v5_nano python -m cembedding # or: python server.pyYou should see HTTP embedding endpoint started on http://127.0.0.1:8401/embed. Verify it:
curl -s http://127.0.0.1:8401/embed \
-H 'content-type: application/json' \
-d '{"texts":["hello world"]}' | head -c 200No image is published: build it from this repository, at the revision you mean to run.
docker build -t cembedding .
# Fill the volume once. Left to itself the server downloads the weights on the
# first request instead -- with the port already accepting connections it cannot
# yet answer, and no progress visible to whoever is waiting on it.
docker run --rm -v cembedding-data:/data cembedding \
cembedding-download-model --model jina-v5-nano
docker run -d --name cembedding -p 8401:8401 -v cembedding-data:/data \
-e EMBEDDING_PROVIDER=onnx_jina_v5_nano \
-e CEMBEDDING_AUTH_TOKEN="$(openssl rand -hex 32)" \
cembeddingThe image sets EMBEDDING_HTTP_HOST=0.0.0.0, because inside a container the
server's default binds the container's own loopback: a published port then
forwards to a socket nothing is listening on, and the connection is refused in a
way that reads like a crash. That is a reachability decision and not a security
one — see Authentication, and set a token whenever the
port is published.
ONNX_MODEL_DIR is /data/model in the image, so the download command above and
the server look in the same place. One model per volume: the variable names a
directory, not a collection.
The model and the index live on the /data volume, which is what survives the
container. A bind-mounted host directory has to be writable by uid 10001 (the
image's user), or the run needs --user "$(id -u)".
Set EMBEDDING_PROVIDER:
| Value | Model | Notes |
|---|---|---|
onnx_jina_v5_nano |
jina-embeddings-v5-text-nano-retrieval (~212M params in the fp32 ONNX graph, 768d) | Local CPU, what CPersona is benchmarked against |
onnx_bge_m3 |
bge-m3 | Local CPU, larger / multilingual |
onnx_miniml |
all-MiniLM-L6-v2 (22M, 384d) | Local CPU, smallest |
mlx_bge_m3 |
bge-m3 (MLX) | Apple Silicon only — pip install ".[mlx]" |
auto_bge_m3 |
bge-m3 | Auto-selects MLX on Apple Silicon, ONNX elsewhere |
api_openai |
provider's model | OpenAI-compatible API; needs EMBEDDING_API_KEY (+ optional EMBEDDING_API_URL, EMBEDDING_MODEL) |
Download a local model with cembedding-download-model --model {miniml,jina-v5-nano,bge-m3} (or python -m cembedding.download_model ... from a source checkout; fetched from HuggingFace into ./data/models, not committed to this repo).
The jina-v5-nano repository ships the same graph in several precisions. EMBEDDING_MODEL_VARIANT selects one; cembedding-download-model --model jina-v5-nano --variant <v> fetches it ahead of time. Measured on CPU execution providers with a mixed English/Japanese corpus (55 texts, 22 queries), against the fp32 vectors as reference:
| variant | download | resident (laptop) | single query (laptop / 4-core x86) | cosine to fp32, median / worst | top-10 agreement |
|---|---|---|---|---|---|
fp32 (default) |
810 MB | 906 MB | 5.8 ms / 71 ms | — | — |
fp16 |
405 MB | 933 MB | 6.3 ms / — | 1.0000 / 1.0000 | 1.00 |
int8 |
236 MB | 628 MB | 46 ms / 319 ms | 0.9998 / 0.9981 | 0.97 |
fp16produces the same vectors asfp32. It halves the download and nothing else: CPU execution providers widen the weights back to fp32 at load.int8cuts resident memory by about a third and keeps retrieval quality (top-10 agreement 0.97 against fp32, 0.98 when int8 queries run against an fp32-indexed corpus), but single-query latency is 4-8x worse on every CPU measured, because activations are quantized at run time. Choose it when memory is the constraint and latency is not.- The 4-bit variants in the same repository are not offered: on this corpus their worst-case cosine to fp32 was 0.30.
Vectors already indexed with one precision stay usable with another (the mixed-precision agreement above), so switching does not require re-indexing, though re-indexing removes the residual difference.
| Env var | Default | Description |
|---|---|---|
EMBEDDING_PROVIDER |
api_openai |
Provider (see table above) |
EMBEDDING_HTTP_PORT |
8401 |
HTTP port for /embed |
EMBEDDING_HTTP_HOST |
127.0.0.1 |
Address /embed binds to. Loopback is right for a single host; a container has to bind an address its peers can reach (see Run it in a container). Moving it decides nothing about who may call — a token does |
EMBEDDING_INDEX_ENABLED |
true |
Enable the persistent vector index endpoints (/index, /search, /remove, /purge) |
EMBEDDING_INDEX_DB_PATH |
data/embedding_index.db |
SQLite file backing the vector index |
EMBEDDING_SEARCH_BACKEND |
numpy |
/search matmul backend. numpy (Accelerate BLAS) or mlx (Apple-GPU resident matrix; falls back to numpy when mlx is absent) |
EMBEDDING_SIDECAR |
auto |
Startup source for the resident vectors: auto uses the sidecar file (see below), off always reads them from SQLite |
EMBEDDING_SIDECAR_PATH |
<index db>.sidecar |
Where the sidecar file lives |
EMBEDDING_SIDECAR_MIN_ROWS |
10000 |
Smallest corpus that gets a sidecar; below it the in-memory index is cheap enough that a file adds nothing |
EMBEDDING_SIDECAR_MAX_TAIL |
0.25 |
Rebuild when the rows written since the last build exceed this fraction of the rows in the file |
ONNX_MODEL_DIR |
(auto) | Override the model directory for ONNX providers |
ONNX_EP_PREFERENCE |
(auto) | ONNX execution providers, comma-separated. Empty = auto (CoreML on macOS, DirectML on Windows, else CPU; CPU always ensured) |
ONNX_MAX_SEQ_LEN |
2048 |
Max tokenization length (1–8192; MiniLM clamped to 512 internally) |
EMBEDDING_MODEL_VARIANT |
fp32 |
Precision of jina-v5-nano to load: fp32 / fp16 / int8 (downloaded on first use). See Model precision before changing it |
ONNX_INTRA_OP_THREADS |
0 |
ONNX Runtime intra-op threads. 0 = runtime default (physical cores). Set it when the process runs under a CPU quota the runtime cannot see (container limit, shared host) |
ONNX_GRAPH_OPT_LEVEL |
all |
ONNX Runtime graph optimization: disable / basic / extended / all. Lower it only to compare against an un-fused graph |
EMBEDDING_MAX_BATCH |
64 |
Most texts one model run may carry when concurrent requests are merged into it. Local providers run one pass at a time, so requests that arrive while a pass is in flight share the next one instead of each paying for a pass (nothing waits for a batch to fill, and one request is never split). Results are bit-identical on the CPU provider; on accelerator providers the low bits (about 1e-6) can depend on the batch a text ran in, as they already did for multi-text requests. 0 disables merging |
EMBEDDING_API_KEY |
— | Required for api_openai |
EMBEDDING_API_URL |
https://api.openai.com/v1/embeddings |
API endpoint for api_openai |
CEMBEDDING_AUTH_TOKEN |
— | Inbound bearer token. Unset = no authentication (see below) |
CEMBEDDING_REQUIRE_AUTH |
false |
Refuse to start when no token is configured |
Loading the index from SQLite copies every stored blob into a matrix, which
costs time and memory proportional to the corpus at every start. With
EMBEDDING_SIDECAR=auto (the default) the same rows are also kept in a file
laid out the way the search matrix already is, and a start maps it instead of
decoding the corpus. At 100,000 vectors of 768 dimensions on an Apple laptop
that is 0.09s to the first search instead of 0.22s, at 403 MiB of peak resident
memory instead of 1.1 GiB.
SQLite stays the durable record and decides every disagreement. The file is
written at startup once the corpus reaches EMBEDDING_SIDECAR_MIN_ROWS, and
again when the rows written since the last build outgrow
EMBEDDING_SIDECAR_MAX_TAIL; rows written after a build are read from SQLite
and are never invisible. Deleting the file costs the next start a full read and
nothing else, and a file that is truncated, foreign or unreadable is ignored
with the reason logged.
cembedding-sidecar status --db data/embedding_index.db # present? usable? how stale?
cembedding-sidecar build --db data/embedding_index.db # write one nowBoth HTTP surfaces — the REST endpoints (/embed, /index, /search,
/remove, /purge) and the Streamable HTTP MCP transport — accept an inbound
bearer token:
CEMBEDDING_AUTH_TOKEN=$(openssl rand -hex 32)With the token set, every request must carry Authorization: Bearer <token>;
a missing header, a wrong scheme and a wrong token are all rejected with 401.
Comparison is constant-time. With the token unset, requests are served exactly
as in earlier versions and a warning is logged — set CEMBEDDING_REQUIRE_AUTH=true
to turn that warning into a startup error instead. Requiring a token is opt-in
in this release so existing deployments keep working; a later release will make
it the default.
Do not treat the bind address as the security boundary. The REST surface
binds loopback and the MCP transport binds 0.0.0.0 by default, but a tunnel or
reverse proxy forwards to loopback all the same, so a loopback bind is no
evidence that requests are local. If the process is reachable through a tunnel,
a proxy, or any non-loopback interface, configure a token.
Run this server, then tell CPersona to use it:
# CPersona MCP config env
CPERSONA_EMBEDDING_MODE=http
CPERSONA_EMBEDDING_URL=http://127.0.0.1:8401/embedWithout an embedding server CPersona still works (FTS5 + keyword search); adding one enables the vector-similarity layer.
To serve CPersona's remote vector search (CPERSONA_VECTOR_SEARCH_MODE=remote),
this server's /index + /search endpoints hold the vectors. v0.6.0 searches a
per-namespace resident matrix (one matmul per query, ~21x faster than v0.5.0 at
237k x 384: 131 ms -> 6 ms/query), so a full-corpus semantic recall stays fast at
memory-corpus scale. When flipping an existing CPersona deployment to remote mode,
first migrate its already-stored vectors:
python scripts/backfill_embedding_index.py \
--cpersona-db ~/.claude/cpersona.db \
--index-db data/embedding_index.db \
--expect-dim 768 # your embedding model's dimensionthen restart this server so it reloads the index. Skipping the backfill silently drops every pre-flip memory from semantic recall (the remote branch only falls back to local search on HTTP errors, not on empty results).
Open an issue — bug report or feature request.
Reports are welcome even when you are not certain it is a bug. If it turns out to be a configuration problem, that is still useful signal — it means the documentation was unclear, which is a defect of its own. Security vulnerabilities are the one exception: please report those privately through GitHub Security Advisories rather than in a public issue.
MIT — see LICENSE.