feat(phase5): hierarchical abstraction engine (v3.11)#164
Merged
Conversation
Feature 4 of the Phase 5 split (umbrella #129): Sleep Phase 5.11 distills cross-cutting principles from accumulated meta-reflections into a queryable abstractions store. - abstractions table (migration-v3.11.sql + canonical schema): level CHECK (principle|strategy|mental_model), stored md5 content_hash with UNIQUE (agent_id, level, namespace, content_hash) — the umbrella's ON CONFLICT DO NOTHING had no conflict target and would have duplicated every principle every cycle. RLS + grants. - Phase 5.11: gated on LLM presence, token budget, and >= 3 meta-reflections; prompt carries the codebase-standard injection guard, wraps reflection content via wrapUserContent, and states the schema's own limits (max 10, under 2000 chars, empty-array escape) so one overlong item does not sink the batch silently. Re-derivation is corroboration: ON CONFLICT keeps GREATEST confidence and revives a deactivated principle only when the fresh derivation is confident (>= 0.5); the conditional WHERE keeps change counts honest. Age-based deactivation of stale sub-0.2 principles. Not skip-gated (input accumulates independently). Cancellation checkpoint before the LLM call so canceled dream runs stop paying. runPhase now records real per-phase token spend in sleep_phase_analytics; count surfaces as SleepCycleResult.principles_extracted. - getAbstractions()/getPrinciples(), GET /principles + /abstractions routes, memforge_principles + memforge_mental_models MCP tools wired to real client methods (honest description: only 'principle' is auto-extracted today), TS + Python SDKs incl. resilient wrappers, tool-definitions, OpenAPI with shared Abstraction component. - tests/abstractions.test.ts: 39 tests — read paths, extraction incl. dedup/revival/confidence-raise/skip-gate immunity/token attribution/ budget + window gates/malformed + schema-invalid + missing-key LLM output, deactivation age and scope boundaries, ordering tiebreak, HTTP status paths, migration shape + RLS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru
This was referenced Jul 26, 2026
salishforge
added a commit
that referenced
this pull request
Jul 26, 2026
* feat(phase5): causal memory graph (v3.10) Feature 3 of the Phase 5 split (umbrella #129): MemForge learns which events tend to follow which, and can answer "what led here" and "what happens next". - causal_edges table (migration-v3.10.sql + canonical schema, RLS, agent-scoped FKs to warm_tier with CASCADE) - Sleep Phase 6.1 mines memory_sequences for repeated content-level A→B patterns (md5 of 50-char prefixes, the Phase 5.5 technique) — the umbrella's row-pair grouping was unsatisfiable under the table's UNIQUE constraint and could never mine an edge. One edge per pattern, anchored to the earliest surviving pair so re-mining updates in place and FK cascade self-heals evicted anchors. Not skip-gated: the phase's input accumulates independently of its own change history, so the zero-change gate would permanently disable mining for young agents. Count surfaces as SleepCycleResult.causal_edges_updated. - getCausalChain(): recursive CTE traversal (causes|effects, depth clamped 1-10, 50-row cap, ordered depth then strength) - predict(): context match → outgoing edges → ranked events. probability = confidence × (1 − e^(−strength/30)) — monotonic, never saturated (raw strength × confidence clamps to 1.0 for virtually every mined edge). Shared-pool matches are excluded from the cause-id list (different id space than warm_tier), and effects are confined to the requested namespace. - GET /memory/:id/causal + POST /memory/:id/predict, memforge_causal_chain + memforge_predict MCP tools wired to real client methods, TS + Python SDKs incl. resilient wrappers, tool-definitions, OpenAPI - tests/causal-graph.test.ts: 39 tests (traversal incl. cycles/clamps/ cap/ordering, predict incl. namespace + pool-collision, Phase 6.1 mining incl. cv weighting + anchor stability + true re-confirmation step, pruning, HTTP status paths, migration shape + RLS) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru * feat(phase5): hierarchical abstraction engine (v3.11) (#164) Feature 4 of the Phase 5 split (umbrella #129): Sleep Phase 5.11 distills cross-cutting principles from accumulated meta-reflections into a queryable abstractions store. - abstractions table (migration-v3.11.sql + canonical schema): level CHECK (principle|strategy|mental_model), stored md5 content_hash with UNIQUE (agent_id, level, namespace, content_hash) — the umbrella's ON CONFLICT DO NOTHING had no conflict target and would have duplicated every principle every cycle. RLS + grants. - Phase 5.11: gated on LLM presence, token budget, and >= 3 meta-reflections; prompt carries the codebase-standard injection guard, wraps reflection content via wrapUserContent, and states the schema's own limits (max 10, under 2000 chars, empty-array escape) so one overlong item does not sink the batch silently. Re-derivation is corroboration: ON CONFLICT keeps GREATEST confidence and revives a deactivated principle only when the fresh derivation is confident (>= 0.5); the conditional WHERE keeps change counts honest. Age-based deactivation of stale sub-0.2 principles. Not skip-gated (input accumulates independently). Cancellation checkpoint before the LLM call so canceled dream runs stop paying. runPhase now records real per-phase token spend in sleep_phase_analytics; count surfaces as SleepCycleResult.principles_extracted. - getAbstractions()/getPrinciples(), GET /principles + /abstractions routes, memforge_principles + memforge_mental_models MCP tools wired to real client methods (honest description: only 'principle' is auto-extracted today), TS + Python SDKs incl. resilient wrappers, tool-definitions, OpenAPI with shared Abstraction component. - tests/abstractions.test.ts: 39 tests — read paths, extraction incl. dedup/revival/confidence-raise/skip-gate immunity/token attribution/ budget + window gates/malformed + schema-invalid + missing-key LLM output, deactivation age and scope boundaries, ordering tiebreak, HTTP status paths, migration shape + RLS. Claude-Session: https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
salishforge
added a commit
that referenced
this pull request
Jul 26, 2026
* feat(phase5): explainable memory operations (v3.10) Feature 2 of the Phase 5 split (umbrella #129): every retrieval can say why, and any warm memory can be asked where it stands. - query(): opt-in explain=true attaches ExplanationFactor[] per result (rank_score, epistemic_status weight, search_mode, temporal_decay) - explainMemory() + GET /memory/:id/explain?warm_id= reports scores, access patterns, and standing against the sleep-cycle score thresholds. Flags are scoped honestly (would_evict_by_threshold, would_flag_low_confidence): capacity eviction and the outcome / contradiction revision channels depend on live cross-table state and are deliberately not predicted here. - memforge_explain MCP tool wired to the real client method; TS + Python SDKs including both resilient wrappers; OpenAPI entries - query cache key extracted to queryKey() in cache.ts so the explain flag's participation in the key is unit-testable without Redis - warm_id validated to int8 range at REST and MCP boundaries; invalid agent ids on /explain return 400 like sibling routes No schema migration — pure code feature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru * feat(phase5): causal memory graph (v3.10) (#163) * feat(phase5): causal memory graph (v3.10) Feature 3 of the Phase 5 split (umbrella #129): MemForge learns which events tend to follow which, and can answer "what led here" and "what happens next". - causal_edges table (migration-v3.10.sql + canonical schema, RLS, agent-scoped FKs to warm_tier with CASCADE) - Sleep Phase 6.1 mines memory_sequences for repeated content-level A→B patterns (md5 of 50-char prefixes, the Phase 5.5 technique) — the umbrella's row-pair grouping was unsatisfiable under the table's UNIQUE constraint and could never mine an edge. One edge per pattern, anchored to the earliest surviving pair so re-mining updates in place and FK cascade self-heals evicted anchors. Not skip-gated: the phase's input accumulates independently of its own change history, so the zero-change gate would permanently disable mining for young agents. Count surfaces as SleepCycleResult.causal_edges_updated. - getCausalChain(): recursive CTE traversal (causes|effects, depth clamped 1-10, 50-row cap, ordered depth then strength) - predict(): context match → outgoing edges → ranked events. probability = confidence × (1 − e^(−strength/30)) — monotonic, never saturated (raw strength × confidence clamps to 1.0 for virtually every mined edge). Shared-pool matches are excluded from the cause-id list (different id space than warm_tier), and effects are confined to the requested namespace. - GET /memory/:id/causal + POST /memory/:id/predict, memforge_causal_chain + memforge_predict MCP tools wired to real client methods, TS + Python SDKs incl. resilient wrappers, tool-definitions, OpenAPI - tests/causal-graph.test.ts: 39 tests (traversal incl. cycles/clamps/ cap/ordering, predict incl. namespace + pool-collision, Phase 6.1 mining incl. cv weighting + anchor stability + true re-confirmation step, pruning, HTTP status paths, migration shape + RLS) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru * feat(phase5): hierarchical abstraction engine (v3.11) (#164) Feature 4 of the Phase 5 split (umbrella #129): Sleep Phase 5.11 distills cross-cutting principles from accumulated meta-reflections into a queryable abstractions store. - abstractions table (migration-v3.11.sql + canonical schema): level CHECK (principle|strategy|mental_model), stored md5 content_hash with UNIQUE (agent_id, level, namespace, content_hash) — the umbrella's ON CONFLICT DO NOTHING had no conflict target and would have duplicated every principle every cycle. RLS + grants. - Phase 5.11: gated on LLM presence, token budget, and >= 3 meta-reflections; prompt carries the codebase-standard injection guard, wraps reflection content via wrapUserContent, and states the schema's own limits (max 10, under 2000 chars, empty-array escape) so one overlong item does not sink the batch silently. Re-derivation is corroboration: ON CONFLICT keeps GREATEST confidence and revives a deactivated principle only when the fresh derivation is confident (>= 0.5); the conditional WHERE keeps change counts honest. Age-based deactivation of stale sub-0.2 principles. Not skip-gated (input accumulates independently). Cancellation checkpoint before the LLM call so canceled dream runs stop paying. runPhase now records real per-phase token spend in sleep_phase_analytics; count surfaces as SleepCycleResult.principles_extracted. - getAbstractions()/getPrinciples(), GET /principles + /abstractions routes, memforge_principles + memforge_mental_models MCP tools wired to real client methods (honest description: only 'principle' is auto-extracted today), TS + Python SDKs incl. resilient wrappers, tool-definitions, OpenAPI with shared Abstraction component. - tests/abstractions.test.ts: 39 tests — read paths, extraction incl. dedup/revival/confidence-raise/skip-gate immunity/token attribution/ budget + window gates/malformed + schema-invalid + missing-key LLM output, deactivation age and scope boundaries, ordering tiebreak, HTTP status paths, migration shape + RLS. Claude-Session: https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
salishforge
added a commit
that referenced
this pull request
Jul 26, 2026
main's only unique commit is 2970a38 — the squash-merge of the collapsed F2+F3+F4 stack, whose content this branch already carries as individual commits (verified: the branch-vs-main diff is exactly the F7 + contested feature delta, purely additive except for lines those features replace). Recording the merge with our tree resolves the squash-vs-branch conflicts without content changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Feature 4 of the Phase 5 split from umbrella #129. Stacked on #163 (causal graph), which stacks on #160.
abstractionstable with a storedmd5(content)hash andUNIQUE (agent_id, level, namespace, content_hash)— the umbrella'sON CONFLICT DO NOTHINGhad no conflict target, so every cycle would have re-inserted the same principles forever.sleep_phase_analytics(therunPhasewrapper previously hardcoded 0 — fine while all wrapped phases were SQL-only, wrong for the first LLM phase).GET /principles,GET /abstractions,memforge_principles+memforge_mental_modelsMCP tools (real executors; descriptions state honestly that only theprinciplelevel is auto-extracted today), TS + Python SDKs incl. resilient wrappers, OpenAPI with a sharedAbstractioncomponent.Review process
3-stage implementation workflow, then a 20-agent adversarial review: 16 raw findings → 12 confirmed, all fixed. Three verifier agents died on a session limit mid-run; I adjudicated their findings manually — two were real (missing cancellation checkpoint before the new LLM call; frozen-confidence/no-revival upsert semantics) and are fixed, one duplicated an already-confirmed finding.
Test evidence
npm run type-check/npm run linttests/abstractions.test.ts(new)npm run test:causal-graphnpm run test:dream-runsnpm run test:integrationDev-DB migration applied and verified idempotent.
🤖 Generated with Claude Code
https://claude.ai/code/session_011xCqQo49d3CEbn6oEvb3Ru