Conversation
Add a local sqlite-vec resolver that can load a real vec0 shared object through QMD instead of assuming the upstream sqlite-vec npm package supports the host platform directly. This keeps the existing supported-platform path intact while opening a system-library path for FreeBSD. It also makes sqlite-vec failures actionable by routing the unavailable state through platform-aware diagnostics and tests.
Stop hard-coupling node-llama-cpp into install-time and import-time paths. This keeps non-LLM commands usable on FreeBSD even when the native backend is unavailable, allows runtime-specific error messages, and keeps the immediate upstream lane conservative by defaulting FreeBSD to a non-building node-llama-cpp policy until the adjacent upstream fix lands.
Add FreeBSD operator documentation and a host-side smoke harness that covers both first-run bring-up and ongoing maintenance flows. This records the validated source-checkout path explicitly and gives the port a repeatable verification entrypoint instead of relying on manual shell history.
There was a problem hiding this comment.
Pull request overview
This PR introduces an upstream-friendly FreeBSD runtime path for QMD’s “core” search workflow by switching sqlite-vec loading to a real loadable extension (vec0.so) and making the node-llama-cpp backend optional/conservative on FreeBSD.
Changes:
- Add a platform-aware sqlite-vec resolver (
QMD_SQLITE_VEC_PATH, FreeBSD system probing, npm binary fallback) plus improved diagnostics. - Make
node-llama-cppoptional and add a runtime loader + error type so LLM features fail fast (especially on FreeBSD). - Add FreeBSD documentation, smoke harness, and new/updated tests around sqlite-vec resolution and LLM backend availability.
Reviewed changes
Copilot reviewed 12 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/sqlite-vec.test.ts | Adds unit tests for sqlite-vec resolution, FreeBSD probe paths, and diagnostic/error formatting. |
| test/llm.test.ts | Adds tests for node-llama-cpp availability/error surfacing and gates integration tests behind an opt-in runtime check. |
| test/freebsd-smoke.sh | Adds a FreeBSD smoke script with --quick vs --full validation paths. |
| src/store.ts | Switches sqlite-vec error creation to platform-aware diagnostics. |
| src/platform/sqlite-vec.ts | New platform helper to resolve vec0 loadable extension paths and produce platform-specific hints/errors. |
| src/platform/node-llama-cpp.ts | New optional runtime loader + error type for node-llama-cpp. |
| src/llm.ts | Refactors to use the runtime loader and adds FreeBSD-specific build policy behavior. |
| src/db.ts | Switches sqlite-vec loading to loadable-extension paths and improves error reporting. |
| src/cli/qmd.ts | Skips device probing by default on FreeBSD and makes pull fail cleanly when LLM backend is unavailable. |
| README.md | Documents FreeBSD support posture and points to the FreeBSD guide. |
| pnpm-lock.yaml | Moves node-llama-cpp under optionalDependencies and updates lock metadata accordingly. |
| package.json | Makes node-llama-cpp an optional dependency. |
| docs/FREEBSD.md | Adds FreeBSD bring-up instructions, vec0.so build steps, and smoke test guidance. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| build: process.platform === "freebsd" ? "never" : "autoAttempt", | ||
| logLevel: nodeLlamaCpp.LlamaLogLevel.error, | ||
| gpu, | ||
| skipDownload: process.platform !== "freebsd", |
There was a problem hiding this comment.
skipDownload is currently set to process.platform !== "freebsd", which makes it true on macOS/Linux/Windows. That appears inverted (and it conflicts with the new unit test expecting skipDownload to be false). This could prevent node-llama-cpp from performing its normal download behavior on supported platforms. Flip the condition (or omit skipDownload so the library default applies), and keep the FreeBSD behavior aligned with the intended “no auto-build” policy.
| skipDownload: process.platform !== "freebsd", | |
| skipDownload: process.platform === "freebsd", |
| override readonly cause: unknown; | ||
|
|
||
| constructor(cause?: unknown) { | ||
| super(formatNodeLlamaCppUnavailableMessage(cause)); | ||
| this.name = "NodeLlamaCppUnavailableError"; | ||
| this.cause = cause; |
There was a problem hiding this comment.
NodeLlamaCppUnavailableError declares override readonly cause, but Error.cause is not readonly in the standard lib typings. This is likely to fail TypeScript structural compatibility checks under strict mode. Make cause writable (drop readonly), or rely on the built-in Error cause by calling super(message, { cause }) and not redeclaring the property.
| override readonly cause: unknown; | |
| constructor(cause?: unknown) { | |
| super(formatNodeLlamaCppUnavailableMessage(cause)); | |
| this.name = "NodeLlamaCppUnavailableError"; | |
| this.cause = cause; | |
| constructor(cause?: unknown) { | |
| super(formatNodeLlamaCppUnavailableMessage(cause), { cause }); | |
| this.name = "NodeLlamaCppUnavailableError"; |
| for cmd in node corepack git cmake ninja python3 gmake bash sqlite3 envsubst; do | ||
| require_cmd "$cmd" | ||
| done | ||
|
|
There was a problem hiding this comment.
The script requires cmake and ninja unconditionally, even in --quick mode. That contradicts the stated goal of --quick being BM25 + sqlite-vec only (and the FreeBSD docs list cmake/ninja as needed only for the later LLM follow-up). Consider only requiring cmake/ninja when MODE=full (or when an LLM backend is actually needed).
| for cmd in node corepack git cmake ninja python3 gmake bash sqlite3 envsubst; do | |
| require_cmd "$cmd" | |
| done | |
| for cmd in node corepack git python3 gmake bash sqlite3 envsubst; do | |
| require_cmd "$cmd" | |
| done | |
| if [ "$MODE" = "full" ]; then | |
| for cmd in cmake ninja; do | |
| require_cmd "$cmd" | |
| done | |
| fi |
Summary
This PR adds the first upstreamable Node.js path for running QMD core search on FreeBSD.
The implementation keeps the existing supported-platform behavior in place, uses a real
vec0.soloadable extension for FreeBSD, and avoids depending on a downstream-vendorednode-llama-cpppatch.What changed
QMD_SQLITE_VEC_PATHnode-llama-cppoptional and conservative on FreeBSD so LLM commands fail fast instead of trying to auto-build through an unsupported path--quickcore path from the later--fullLLM follow-upWhy this shape
FreeBSD is closer to the existing macOS system-library path than to the Linux prebuilt-binary path. The immediate upstream lane therefore prefers:
vec0.so)node-llama-cppsupport lands cleanlyprocess.platform === "freebsd"checksValidation
Validated on a real FreeBSD 15.0 amd64 host:
pnpm install --frozen-lockfilesqlite3loading a FreeBSD-builtvec0.soqmd statustest/freebsd-smoke.sh --quickqmd embedfailing withnode-llama-cpp is unavailableinstead of falling through tospawn npm ENOENTKnown limits
test/freebsd-smoke.sh --fullremain gated on the adjacent upstreamnode-llama-cppFreeBSD fixRelated upstream dependency: asg017/sqlite-vec#282