Skip to content

feat(dream): add bd dream subcommand for memory consolidation (#3263)#3264

Open
boaz-hwang wants to merge 1 commit intogastownhall:mainfrom
boaz-hwang:feat/dream-mvp
Open

feat(dream): add bd dream subcommand for memory consolidation (#3263)#3264
boaz-hwang wants to merge 1 commit intogastownhall:mainfrom
boaz-hwang:feat/dream-mvp

Conversation

@boaz-hwang
Copy link
Copy Markdown

Closes #3263.

Summary

Adds bd dream — a subcommand that consolidates the memory store (bd remember / bd memories) by asking an LLM to identify duplicates, stale references, and low-signal entries, then applies a structured forget / merge / update plan via the existing storage layer. See docs/DREAM.md (added in this PR) for the full design.

CLI surface:

bd dream [run]                   # default = run
bd dream run --dry-run           # show plan only
bd dream run --force             # bypass the time/churn threshold
bd dream run --check             # exit 0 if eligible, 1 if not (cron / hooks)
bd dream status                  # last run + threshold state

--json is supported on every subcommand.

What's included

  • internal/dream/ (consolidator + prompt + safety guard + types) — all unit-tested with a messagesNewer fake (no network in tests).
  • cmd/bd/dream.go (cobra commands + threshold logic + state read/write + apply pipeline) — integration-tested against embedded Dolt.
  • cmd/bd/kv.go — adds \"dream.\" to the reserved-prefix list so users can't shadow dream's state via bd kv set.
  • docs/DREAM.md + CHANGELOG.md (Unreleased entry).

Conventions followed

  • Cobra parent-child pattern (hooksCmd / kvCmd style) with GroupID: \"setup\".
  • CheckReadonly(\"dream\") on mutating ops, ensureDirectMode on every DB-touching op.
  • store.GetAllConfig filtered by kvPrefix + memoryPrefix (inlined as in cmd/bd/memory.go:134-141).
  • store.SetConfig / DeleteConfig followed by a single CommitPending(ctx, getActor()) per command run.
  • --json flag respected in every subcommand (matches bd memories / bd recall).
  • internal/compact/haiku.go retry / API-key / model resolution pattern reused for the SDK call.
  • //go:build cgo on the embedded-Dolt integration test file.

Threshold semantics

  • dream.min-interval-hours (default 24) — minimum hours between runs.
  • dream.min-churn (default 5) — minimum memory-count delta since last run.
  • 2-memory floor applies even with --force (nothing to consolidate otherwise).
  • First run is always eligible.

State (dream.last-run-at, dream.last-run-status, dream.last-run-summary, dream.memory-count-at-last-run, dream.min-interval-hours, dream.min-churn) lives in the Dolt config table and syncs via bd dolt push like every other dream / sync-related key.

Test plan

  • go test ./internal/dream/... — 8 unit tests pass (safety guard, malformed responses, eligibility logic).
  • go test ./cmd/bd/ -run TestDream|TestKVReservedDream — 6 integration tests pass against embedded Dolt (status / status JSON / --check exit codes / churn gate / missing-API-key path / reserved-prefix guard).
  • gofmt -l clean, golangci-lint run --build-tags=gms_pure_go 0 issues on changed files.
  • make build clean.
  • End-to-end manual: applied a real consolidation pass on a project with 12 memories — merged 2 newsletter-related entries into one richer entry. Safety guard correctly rejected an earlier run that would have deleted >50%.

Out of scope

  • A pluggable internal/ai.Executor (sdk + claude-cli) so users on a Claude Code subscription can use bd dream (and eventually bd compact) without a separate API key. Follow-up PR is already prepared on a stacked branch and will be opened once this lands. Keeping it separate per CONTRIBUTING.md's "one issue per PR" rule.
  • A bd dream config interactive wizard.
  • A native scheduler / bd dream schedule helper.
  • OpenAI / Ollama providers.
  • Refactoring bd compact and bd find-duplicates --method ai onto the executor abstraction (lands with the executor PR).

Local verification recipe

export ANTHROPIC_API_KEY=sk-ant-...
cd <a project with several bd memories>
bd dream run --dry-run --force
bd dream run --force
bd dream status

🤖 Generated with Claude Code

`bd dream` is a periodic LLM-driven pass over the memory store created by
`bd remember` / `bd memories`. It identifies duplicates, stale references,
and low-signal entries and applies merge / forget / update operations.
Inspired by Claude Code's AutoDream, but built into bd so it's not coupled
to any specific editor.

- New package internal/dream with a Consolidator that wraps the existing
  anthropic-sdk-go (already used by internal/compact) — same retry / model
  resolution / API key resolution patterns. Tool-use is used for structured
  output. A safety guard rejects plans that would delete > 50% of memories.
- New cmd/bd/dream.go subcommand with `run`, `status`, and `--check`.
  Threshold gate: at least N hours since last run AND M memories of churn,
  defaults 24h / 5 (matching AutoDream's "5 sessions over 24 hours" adapted
  to bd's stateless model). All state lives in the Dolt config table under
  the `dream.*` prefix so it syncs via `bd dolt push`.
- Added `dream.` to the kv reserved-prefix list so users cannot shadow
  dream's state through `bd kv set`.
- Unit tests in internal/dream cover the safety guard, empty/malformed
  responses, and the eligibility gate via a fake messagesNewer.
- Integration tests in cmd/bd/dream_embedded_test.go exercise status, the
  --check exit codes, churn gating, and the missing-API-key path against
  embedded Dolt.
- docs/DREAM.md documents the feature, threshold semantics, scheduler
  recipes (cron / launchd / editor stop hooks), and configuration keys.

Out of scope (follow-up PRs): provider abstraction beyond Anthropic,
`bd dream config` interactive setup wizard, native scheduler / `bd dream
schedule` helper, and bd-hook-based triggering.

Reuses: `config.GetString("ai.api_key")`, `config.DefaultAIModel()`,
`store.GetAllConfig` / `SetConfig` / `DeleteConfig` / `CommitPending`,
`getActor`, `kvPrefix + memoryPrefix` from cmd/bd/memory.go.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(dream): bd dream subcommand for memory consolidation

1 participant