feat(embedder): zero-config startup with built-in default embedder; split .env.example (closes #940 #941 #949 #948 #)#945
Open
happyhust wants to merge 15 commits into
Conversation
Author
|
Update — added seekdb as the default database (sharing the OceanBase backend). Following the same zero-config principle as the embedder, this branch now also makes embedded SeekDB the default vector store. SeekDB and OceanBase are the same engine (same SQL surface, same What changed in this commit
Test plan
Now |
6 tasks
happyhust
added a commit
to happyhust/powermem
that referenced
this pull request
May 29, 2026
…both modes
Since the seekdb backend supports a remote-server mode in addition to its
embedded on-disk mode, splitting it into its own first-class provider was
overshoot — the OceanBase config surface already expresses both shapes
(empty OCEANBASE_HOST = embedded seekdb; non-empty = remote cluster).
This commit removes the SeekDBConfig / SeekDBGraphConfig classes, the
`seekdb` provider registration, the SEEKDB_* env namespace, the OCEANBASE_*
fallback isolation, the OCEANBASE_PATH rejection, the required-host
validator on OceanBaseConfig, and the dedicated .env block.
The zero-config startup story still holds — just delivered through a
single provider:
- DATABASE_PROVIDER default switches from "seekdb" to "oceanbase".
- MemoryConfig.vector_store default_factory becomes OceanBaseConfig
(which already defaulted to host="" → embedded seekdb at
./seekdb_data).
- core/memory.py storage_type fallback returns to "oceanbase".
- .env.example minimal block describes the single-provider default
("OceanBase with no host = embedded seekdb").
- .env.example.full collapses the previous separate seekdb + oceanbase
blocks into one OceanBase section that documents both modes inline,
with each variable still annotated with purpose / recommended /
alternatives.
- READMEs (en / cn / jp) match.
What is **kept** from the rest of PR oceanbase#945, since it stands independent of
the storage decision: the built-in default embedder (issue oceanbase#941), the
pyseekdb>=1.3.0 dep floor, the .env.example minimal/full split (issue
oceanbase#940), and the per-section purpose/recommended/alternatives doc style.
Tests rewritten to match: 189 passed in the full unit suite. Clean-env
smoke check confirms `MemoryConfig()` with no env vars produces an
OceanBaseConfig with host="" and ob_path="./seekdb_data" (embedded
seekdb mode), with the built-in local embedder.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes oceanbase#940, closes oceanbase#941. Previously a fresh MemoryConfig() defaulted to OpenAIEmbeddingConfig and text-embedding-3-small, which made the "default" path require an OPENAI_API_KEY before the system could run — no real zero-config story. This change introduces a built-in default embedder that mirrors pyseekdb's DefaultEmbeddingFunction (sentence-transformers/all-MiniLM-L6-v2 via ONNX, 384 dims) so PowerMem boots end-to-end with no API key and no external service. The model auto-downloads to the local cache on first use and runs entirely locally afterwards. Switching to a production embedder is a single config field. Alongside that, .env.example is split into a minimal version (just the LLM key block, ~5 vars) and .env.example.full (every existing knob, grouped by component) so first-time users aren't drowned in options. README, README_CN, and README_JP all point at both files. Changes: - New PyseekdbDefaultEmbedding wrapper + PyseekdbDefaultEmbeddingConfig registered under provider name "default" - MemoryConfig.embedder.default_factory now PyseekdbDefaultEmbeddingConfig - .env.example trimmed to the strictly required keys; full reference moved to .env.example.full with an updated header that points back - README / README_CN / README_JP updated to reflect zero-config defaults - Unit tests: embedder round-trip, factory registry, and zero-config MemoryConfig() (mock pyseekdb so no model download in CI) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…backend Adds `seekdb` as a first-class database provider name, registered to the same `OceanBaseVectorStore` class as `oceanbase` (SeekDB is OceanBase running embedded — same engine, same SQL surface, only configuration differs). Promotes seekdb to the default everywhere it matters: - MemoryConfig.vector_store.default_factory: SQLiteConfig -> SeekDBConfig - DatabaseSettings.provider env default: "sqlite" -> "seekdb" - core/memory.py storage_type fallback: "oceanbase" -> "seekdb" - deprecated create_config() / CreateConfigOptions default also updated `SeekDBConfig` subclasses `OceanBaseConfig` and only overrides what differs: provider name, embedded-mode defaults (empty host, on-disk `./seekdb_data`), and `SEEKDB_*` env var aliases (each falls back to the corresponding `OCEANBASE_*` alias, so users can flip `DATABASE_PROVIDER` between `seekdb` and `oceanbase` without rewriting their `.env`). Same treatment for `SeekDBGraphConfig` on the graph-store side. .env.example.full reorders the database section to lead with embedded SeekDB, and `.env.example` (minimal) now advertises SeekDB + the local embedder as the zero-config defaults. README/CN/JP follow suit. Tests: 7 new unit tests pin the contract — provider registration, shared class path between `seekdb` and `oceanbase`, embedded-mode defaults, SEEKDB_* env aliases, MemoryConfig() default, and DatabaseSettings() default. Full unit suite: 193 passed (up from 186). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per product branding, the canonical spelling is lowercase `seekdb`. This
commit normalizes every user-visible occurrence across the repo:
- Config templates: .env.example, .env.example.full
- READMEs: README.md, README_CN.md, README_JP.md
- Docs: docs/api/0002-async_memory.md
- Python docstrings, Field descriptions, comments, log/error strings
in storage/, utils/, cli/, user_memory/, core/memory.py, server/,
script/
Intentionally kept (these are code identifiers, not documentation):
- Class names: `SeekDBConfig`, `SeekDBGraphConfig` (renaming would break
the public API and violate PascalCase convention for Python classes)
- Env var prefixes: `SEEKDB_*` (matches the existing `OCEANBASE_*`
convention — ALL_CAPS for env variables is independent of the brand
spelling)
Tests still green: 193 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per product direction, when PowerMem uses seekdb it should default to the latest seekdb. The previous floor `>=0.1.0` was permissive enough to resolve to long-superseded releases on fresh installs; bump it to `>=1.3.0` so users get the current engine (matching what is already installed in CI / dev environments). Tests still green: 193 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous wording told users to `cp .env.example.full .env` when they wanted to tune anything, which throws away the curated minimal file and exposes the wall of knobs we deliberately hid. The new framing matches how the file is actually meant to be used: keep .env minimal, and additively pull individual blocks from .env.example.full when the environment offers stronger infrastructure (OceanBase cluster, hosted embedding LLM, rerank LLM, etc.) or you need more performance. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…alternatives
Restructures every section's annotations in both files so each variable now
documents three things explicitly:
1. What the config *does* — the actual effect, not just the field name.
2. A recommended value with a short reason.
3. Alternative options so the reader knows the recommendation is a default,
not a constraint.
In .env.example (minimal), the single LLM block now explains each variable
(provider / key / model) with named alternatives per provider (qwen-plus →
qwen-max / qwen-turbo / gpt-4o / claude-sonnet-4-6 / local Ollama models).
In .env.example.full, every numbered section gets the same treatment:
database providers (with the seekdb ↔ oceanbase symmetry), LLM, embedding,
rerank, agent scoping, Ebbinghaus decay, performance batches/caches, security
+ encryption, telemetry, audit, logging, skill store, graph store, sparse
embedding, query rewrite, HTTP server (bind / auth / rate limit / CORS),
custom prompts. Each section header also briefly states what subsystem the
block configures and when to bother touching it.
No code changes. 193/193 unit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Before this change, a user running DATABASE_PROVIDER=seekdb could configure the connection (SEEKDB_PATH, SEEKDB_DATABASE, …) and vector index (SEEKDB_ INDEX_TYPE, SEEKDB_VECTOR_METRIC_TYPE, SEEKDB_EMBEDDING_MODEL_DIMS) with SEEKDB_* keys, but had to drop down to OCEANBASE_TEXT_FIELD / VECTOR_FIELD / PRIMARY_FIELD / METADATA_FIELD / VIDX_NAME for the table column names. That asymmetry forced mixed-namespace .env files for anyone integrating with an existing schema under seekdb — exactly the friction the SEEKDB_* aliases were introduced to remove. This commit adds SEEKDB_TEXT_FIELD, SEEKDB_VECTOR_FIELD, SEEKDB_PRIMARY_ FIELD, SEEKDB_METADATA_FIELD, and SEEKDB_VIDX_NAME aliases on SeekDBConfig, each with the matching OCEANBASE_* alias kept as a fallback for migrations. .env.example.full documents them in the seekdb section, and the OceanBase section cross-references back to it. Tests pin both directions: SEEKDB_* primary wins, OCEANBASE_* fallback still resolves. Suite: 195 passed (up from 193). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… hybrid
Audit found four OCEANBASE_* env aliases with no SEEKDB_* counterpart:
- OCEANBASE_POOL_RECYCLE (pool_recycle)
- OCEANBASE_POOL_PRE_PING (pool_pre_ping)
- OCEANBASE_INCLUDE_SPARSE (include_sparse)
- OCEANBASE_ENABLE_NATIVE_HYBRID (enable_native_hybrid)
This forced a mixed-namespace .env for anyone tuning the connection pool,
enabling sparse vectors, or pushing hybrid ranking into the engine's native
SQL extension under DATABASE_PROVIDER=seekdb — exactly the inconsistency
the SEEKDB_* aliases were introduced to remove.
This commit adds the four missing aliases on SeekDBConfig, with the matching
OCEANBASE_* keys kept as fallbacks for migrations. Documented honestly in
.env.example.full:
- Pool knobs are no-ops in embedded mode (NullPool), useful when seekdb
points at a remote host.
- SEEKDB_INCLUDE_SPARSE is a shortcut for the SPARSE_VECTOR_ENABLE switch
in section 14 (both aliases still resolve).
- SEEKDB_ENABLE_NATIVE_HYBRID requires seekdb ≥1.3 or OceanBase ≥4.5.
The OceanBase section cross-references the same tunables instead of
duplicating the docs. After this commit, `diff` of OCEANBASE_* vs SEEKDB_*
aliases on the config class is empty — perfect parity.
Tests: two new — SEEKDB_* primary aliases bind, OCEANBASE_* fallback still
resolves. Suite: 197 passed (up from 195).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…configs Three related changes that lock in the contract "seekdb reads SEEKDB_*, oceanbase reads OCEANBASE_* — no cross-reading": 1. SeekDBConfig drops every OCEANBASE_* fallback alias. Previously a seekdb-configured deployment could silently inherit settings from OCEANBASE_* keys lying around in the env; now those keys are ignored entirely. The only cross-cutting alias kept is SPARSE_VECTOR_ENABLE, which is a generic feature toggle (not OceanBase-namespaced) shared by all providers. 2. OceanBaseConfig drops the ob_path field. OCEANBASE_PATH is a seekdb concept (the embedded on-disk data directory). Setting it while DATABASE_PROVIDER=oceanbase now raises a clear validation error pointing the user at DATABASE_PROVIDER=seekdb / SEEKDB_PATH instead. OceanBaseConfig.host's default flips from "" to "127.0.0.1" and a field validator rejects empty values, so there is no silent fall- through to embedded mode under the oceanbase provider. The validator is scoped to the OceanBaseConfig class itself so SeekDBConfig (which keeps host="" as the embedded-mode signal) is unaffected. 3. SEEKDB_ENABLE_NATIVE_HYBRID default flips from false to true. This branch already pins pyseekdb>=1.3.0, which ships the native hybrid SQL extension, so the new default matches what the engine actually supports out of the box. .env.example.full documents the new contract: the seekdb section calls out "namespace isolation" explicitly, the OceanBase section notes that OCEANBASE_PATH is rejected and that OCEANBASE_HOST is required. Tests cover: SeekDBConfig ignores OCEANBASE_* env keys, OceanBaseConfig rejects empty host, OceanBaseConfig rejects OCEANBASE_PATH env, the OCEANBASE_PATH rejection does NOT fire on the SeekDBConfig subclass, and SEEKDB_ENABLE_NATIVE_HYBRID defaults to True. Suite: 201 passed (up from 197). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…_* docs
Audit of the storage backend (oceanbase.py:200-220) confirms which of the
four shared knobs are actually live on each code path:
- POOL_RECYCLE / POOL_PRE_PING — read only inside `if host:` (the
remote-cluster branch). Embedded seekdb uses NullPool and never
reads them.
- INCLUDE_SPARSE / ENABLE_NATIVE_HYBRID — read in both branches; both
backends genuinely use them.
So the two pool knobs are oceanbase-only. This commit removes them from
the seekdb namespace end-to-end and adds active code-level rejection so
a misconfiguration surfaces loudly instead of silently doing nothing:
- SeekDBConfig now overrides pool_recycle / pool_pre_ping with no env
aliases (so OCEANBASE_POOL_* cannot bleed through), and a
model_validator on SeekDBConfig raises ValueError if either
SEEKDB_POOL_RECYCLE or SEEKDB_POOL_PRE_PING is set in the
environment — the error message points the user at the
OCEANBASE_POOL_* equivalents.
- .env.example.full drops the SEEKDB_POOL_* entries from the seekdb
block entirely.
- The OceanBase block flips its previously commented-out hint lines
into active, fully-documented settings (purpose / recommended /
other-options format) for OCEANBASE_POOL_RECYCLE,
OCEANBASE_POOL_PRE_PING, OCEANBASE_INCLUDE_SPARSE,
OCEANBASE_ENABLE_NATIVE_HYBRID.
INCLUDE_SPARSE and ENABLE_NATIVE_HYBRID stay on both providers — same
field, namespace-isolated aliases (SEEKDB_* vs OCEANBASE_*), with
provider-appropriate defaults (seekdb ships ≥1.3 with native hybrid on;
OceanBase defaults to off to remain safe for older clusters).
Tests: two new ones pin that SEEKDB_POOL_RECYCLE / SEEKDB_POOL_PRE_PING
env vars are rejected with a clear error. Suite: 203 passed (up from 201).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…both modes
Since the seekdb backend supports a remote-server mode in addition to its
embedded on-disk mode, splitting it into its own first-class provider was
overshoot — the OceanBase config surface already expresses both shapes
(empty OCEANBASE_HOST = embedded seekdb; non-empty = remote cluster).
This commit removes the SeekDBConfig / SeekDBGraphConfig classes, the
`seekdb` provider registration, the SEEKDB_* env namespace, the OCEANBASE_*
fallback isolation, the OCEANBASE_PATH rejection, the required-host
validator on OceanBaseConfig, and the dedicated .env block.
The zero-config startup story still holds — just delivered through a
single provider:
- DATABASE_PROVIDER default switches from "seekdb" to "oceanbase".
- MemoryConfig.vector_store default_factory becomes OceanBaseConfig
(which already defaulted to host="" → embedded seekdb at
./seekdb_data).
- core/memory.py storage_type fallback returns to "oceanbase".
- .env.example minimal block describes the single-provider default
("OceanBase with no host = embedded seekdb").
- .env.example.full collapses the previous separate seekdb + oceanbase
blocks into one OceanBase section that documents both modes inline,
with each variable still annotated with purpose / recommended /
alternatives.
- READMEs (en / cn / jp) match.
What is **kept** from the rest of PR oceanbase#945, since it stands independent of
the storage decision: the built-in default embedder (issue oceanbase#941), the
pyseekdb>=1.3.0 dep floor, the .env.example minimal/full split (issue
oceanbase#940), and the per-section purpose/recommended/alternatives doc style.
Tests rewritten to match: 189 passed in the full unit suite. Clean-env
smoke check confirms `MemoryConfig()` with no env vars produces an
OceanBaseConfig with host="" and ob_path="./seekdb_data" (embedded
seekdb mode), with the built-in local embedder.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rides Some deployments (corporate proxies, private gateways, self-hosted ollama/vllm endpoints) need to override the default base URL for the chosen LLM provider. Those settings deliberately live in .env.example.full, not in the minimal file. Add a short note in .env.example's LLM block telling readers exactly where to look and which variable names to cherry-pick (QWEN_LLM_BASE_URL, OPENAI_LLM_BASE_URL, etc.). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…fault
The full reference advertised qwen as the embedding provider, but the
zero-config default the code actually ships is the built-in local
`default` provider (PyseekdbDefaultEmbedding, all-MiniLM-L6-v2, 384d).
Having the docs disagree with the wired default was confusing — copying
the full file would silently switch users from local to cloud
embeddings and require an API key.
This commit aligns the block with the code:
- EMBEDDING_PROVIDER=default
- EMBEDDING_MODEL=all-MiniLM-L6-v2
- EMBEDDING_DIMS=384 (and matches the OCEANBASE_EMBEDDING_MODEL_DIMS
default in section 1)
- EMBEDDING_API_KEY commented out (the default needs none; uncomment
when switching providers)
- Recommended / Other options re-ordered: `default` is now first with
the rationale, cloud / self-hosted providers listed as upgrades.
Smoke-checked: `EMBEDDING_PROVIDER=default` + `EMBEDDING_MODEL=
all-MiniLM-L6-v2` + `EMBEDDING_DIMS=384` resolves end-to-end through
EmbeddingSettings.to_config().
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ll & docs - embedder: default EMBEDDING_PROVIDER to the built-in local all-MiniLM-L6-v2 embedder (384 dims, no API key) so PowerMem starts with true zero config; set EMBEDDING_PROVIDER to switch to a cloud provider (oceanbase#941) - server: change default listening port 8000 -> 8848 across core config, CLI help, Makefile, Docker (Dockerfile/compose/entrypoint), claude-code-plugin hook, examples, VS Code extension and regression tests (oceanbase#949) - claude-code-plugin: add one-click SETUP.md / UNINSTALL.md, marketplace.json and refreshed hooks for install-and-go agent wiring (oceanbase#942) - docs: add per-agent setup guide (docs/integrations/claude_code.md, overview) and refresh README / README_CN / README_JP / getting-started (oceanbase#943) - chore: ignore seekdb_data/; black-reformat config_loader.py and utils.py Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
happyhust
force-pushed
the
feat/zero-config-default-embedder
branch
from
1b7793a to
e82e576
Compare
|
|
happyhust
changed the title
longdafeng
force-pushed
the
feat/zero-config-default-embedder
branch
from
6cbc071 to
afe18b0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR resolves #940 (config simplification) and #941 (built-in default embedder so zero-config startup actually works) in a single, narrowly-scoped change.
What changes
1. Built-in default embedder (#941)
MemoryConfig().embedderpreviously fell back toOpenAIEmbeddingConfig+text-embedding-3-small, which made the "default" path requireOPENAI_API_KEYbefore the system could run — there was no real zero-config story.This PR adds a built-in default embedder that mirrors the exact model used by pyseekdb's
DefaultEmbeddingFunction:sentence-transformers/all-MiniLM-L6-v2via ONNX runtime, 384 dimensions. The model auto-downloads to~/.cache/pyseekdb/onnx_models/on first use and runs entirely locally afterwards.PyseekdbDefaultEmbeddingwrapper classPyseekdbDefaultEmbeddingConfigregistered under provider name"default"MemoryConfig.embedder.default_factoryswitched fromOpenAIEmbeddingConfigtoPyseekdbDefaultEmbeddingConfigpyseekdb>=0.1.0inpyproject.toml), so this adds no new transitive dependenciesAfter this change,
MemoryConfig()constructs successfully with no env vars and no API key. Acceptance criteria from #941:embedderis not configured2.
.env.examplesplit (#940)The 384-line
.env.examplewas the "wall of options" #940 calls out. This PR:.env.example.full(every existing knob, grouped by component, unchanged content) with a new header explaining when to copy it vs the minimal version..env.exampledown to the strictly-required keys: just the LLM provider block (LLM_PROVIDER,LLM_API_KEY,LLM_MODEL). Everything else now defaults sanely: SQLite at./powermem.db, local default embedder, no graph store, no reranker, etc.README.md,README_CN.md, andREADME_JP.mdto point at both files and to call out the new zero-config defaults.Acceptance criteria from #940:
.envderived from.env.example(only required keys set).env.examplecontains only required variables, with explanatory comments.env.example.fullcontains all variables, grouped and commentedWhat does not change
.envthat setsEMBEDDING_PROVIDER=openai(or qwen / etc.) is unaffected — the new default only kicks in when nothing else is configured.pyproject.toml.Test plan
pytest tests/unit/test_pyseekdb_default_embeddings.py— 8 new tests covering: embed round-trip (mocked, no model download in CI), embed_batch, empty input, None rejection, default model/dim match pyseekdb, provider registry, factory resolution, and the headlineMemoryConfig()zero-config invariant.pytest tests/unit— 186 tests pass (full unit suite, no regressions from this branch).env -i python -c "from powermem.configs import MemoryConfig; cfg = MemoryConfig(); assert cfg.embedder.provider == 'default' and cfg.embedder.api_key is None"succeeds with no.env, no env vars.~/.cache/pyseekdb/...matches expectations on Windows / macOS / Linux.Notes for reviewers
.env.exampleis intentionally minimal — only the LLM API key remains, because the LLM is the one component without a credential-free default. Everything else has a working default.DefaultEmbeddingFunction; we deliberately did not re-implement the ONNX download/inference path so the two systems stay in lock-step on the default model.🤖 Generated with Claude Code