Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 145 additions & 86 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,65 +75,15 @@ High-level system design and performance optimization layers introduced in v3.0.
- Hit/miss counters for diagnostics
- Lazy cleanup of expired entries

**Disk Persistence (Theme C):**
- `loadFromDisk(diskPath, vaultVersion)` - Restore cache on startup
- `saveToDisk(diskPath, vaultVersion)` - Non-blocking write-through
- Location: `<vault>/.clausidian/cache.json`
- Atomic writes via temp file + rename

**Invalidation:**
- Triggered by `vault.write()` (triggers all cache clears)
- Per-query invalidation via `SelectiveInvalidation` hook

### 2. ClusterCache - Union-Find Results

**Location:** `src/cluster-cache.mjs`

**Purpose:** Cache graph clustering results (union-find algorithm output)

**Key Features:**
- Vault-version aware (auto-invalidate on version mismatch)
- Bulk load support for multiple queries
- Expiry checking integrated with vault versioning

**Invalidation:**
- Triggers when `vault.version` changes
- Fallback for schema migrations
- Triggered by `vault.write()` (triggers full clear)
- Per-query invalidation via `invalidate()` method

### 3. SelectiveInvalidation - Per-Note Dirty Tracking
**Notes:**
- Disk persistence via separate `cache` command (see `src/commands/cache.mjs`)
- In-memory only during process lifetime; use `cache save` to persist state

**Location:** `src/vault-selective-invalidation.mjs`

**Purpose:** Track which notes were modified, avoiding full vault re-indexing

**Key Features:**
- Per-note dirty marking (not boolean flag)
- Separate tracking for tags index and graph index
- `getDirty(indexType)` returns only modified notes
- `clearDirty(partial)` allows selective clearing

**Integration:**
- Called by `vault.write()` when notes are modified
- Feeds invalidation signals to SelectiveInvalidation hook

### 4. FileHasher - Change Detection

**Location:** `src/file-hasher.mjs`

**Purpose:** Detect file changes with mtime + size hashing (fast, reliable)

**Key Features:**
- Single file hashing: `O(1)` time
- Directory traversal: recursive hashing of note tree
- Diff detection: created, modified, deleted files
- Size + mtime both checked (prevents false negatives)

**Usage:**
- Incremental sync foundation
- Backup/sync tools can query change sets
- No full vault hash needed

### 5. VaultValidator - Root Directory Validation
### 2. VaultValidator - Root Directory Validation

**Location:** `src/vault-validator.mjs`

Expand Down Expand Up @@ -180,37 +130,16 @@ High-level system design and performance optimization layers introduced in v3.0.
- `review`, `review monthly` - Report generation
- `cache stats`, `cache clear` - Persistent cache management

## Data Persistence (Theme C)

### Disk Cache Structure
## Planned Features (Future Work)

**File:** `<vault>/.clausidian/cache.json`
The following performance optimization modules are designed but not yet implemented:

**Format:**
```json
{
"vaultVersion": "3.1.0",
"timestamp": 1711827600000,
"entries": [
[
"keyword|type|tag|status|regex",
{
"results": [...],
"timestamp": 1711827500000
}
]
]
}
```
- **ClusterCache** - Union-find result caching (for graph clustering)
- **SelectiveInvalidation** - Per-note dirty tracking (incremental re-indexing)
- **FileHasher** - Change detection via mtime + size hashing
- **SearchCache disk persistence** - Write-through caching to `<vault>/.clausidian/cache.json`

**Lifecycle:**
1. Process startup → `SearchCache.loadFromDisk()` restores valid entries
2. Query → `SearchCache.set()` triggers `setImmediate()` write
3. `vault.write()` → invalidates cache, clears disk file
4. Process lifecycle → periodic cleanup of expired entries

**Performance:**
- Cold-start search: 500ms+ → 50ms (10x improvement)
These are architectural designs for v3.2.0+. Current implementation focuses on in-memory SearchCache.
- No blocking I/O during query execution (setImmediate)
- Graceful degradation on disk errors

Expand Down Expand Up @@ -273,7 +202,137 @@ High-level system design and performance optimization layers introduced in v3.0.
| Plugin ecosystem | API stability first | Defer |
| Persistent TTL index | TTL-aware on-disk cache | v3.2.0 |
| Batch parallelization | Low ROI (batches < 100 items) | v3.3.0+ |
| AI capabilities | LLM integration (breaks zero-dep) | v3.2.0 |
| Vector embedding search | Semantic similarity for memory graph | v3.7.0 |

## Dynamic Memory System (v3.6.0)

### Architecture

```
┌─────────────────────────────────────────────────────────────┐
│ MemoryBridge (Coordinator) │
│ Full sync, auto-wiring, unified context, lifecycle │
└───────────┬──────────────────┬──────────────────┬───────────┘
│ │ │
┌────────▼────────┐ ┌─────▼──────┐ ┌───────▼────────┐
│ MemoryGraph │ │ SessionMemory│ │ Claude Memory │
│ (Graph DB) │ │ (Sessions) │ │ (~/.claude/) │
└────────┬────────┘ └─────┬──────┘ └───────┬────────┘
│ │ │
┌────────▼──────────────────▼──────────────────▼───────────┐
│ EventBus (Events) │
│ memory:*, session:*, note:* → auto-trigger sync/bridge │
└──────────────────────────────────────────────────────────┘
```

### MemoryGraph

**Location:** `src/memory-graph.mjs`

**Purpose:** Track weighted relationships between notes, sessions, and topics as a graph

**Key Features:**
- Node types: `project`, `area`, `resource`, `idea`, `journal`, `session`, `topic`
- Edge types: `related`, `tag-similar`, `session-active`, `session-note:created`
- Weighted edges with reinforcement (cap at 10) and automatic decay
- Context-aware retrieval: graph traversal + relevance scoring
- Persistent storage: `.clausidian/memory-graph.json`

**Lifecycle:**
- Decay: `weight *= 0.95^(days since last access)` — natural forgetting
- Promotion: ephemeral nodes become persistent after 3+ accesses
- Pruning: edges below 0.1 weight are removed; max 20 edges per node

**Storage Format:**
```json
{
"version": "1.0",
"nodes": { "api-project": { "type": "project", "weight": 1.5, ... } },
"edges": { "api-project::backend-dev": { "weight": 2.0, "type": "related" } }
}
```

### SessionMemory

**Location:** `src/session-memory.mjs`

**Purpose:** Persist session context (decisions, learnings, next steps) across agent restarts

**Key Features:**
- Session lifecycle: `startSession()` → record events → `endSession()` / `abandonSession()`
- Auto-extraction: decisions from note creation patterns, learnings from search frequency
- Context window: combines current session + recent sessions + graph results
- Pending step tracking: incomplete next steps surface across sessions
- Storage: `.clausidian/sessions/{sessionId}.json`

**Session Structure:**
```json
{
"id": "20260402120000-ab12",
"state": "completed",
"context": { "topic": "api-design", "activeNotes": ["api-project"] },
"events": [ { "type": "note:created", "note": "new-endpoint" } ],
"decisions": [ { "text": "Use Fastify", "timestamp": "..." } ],
"learnings": [ { "text": "Always validate input", "timestamp": "..." } ],
"nextSteps": [ { "text": "Add auth middleware", "completed": false } ]
}
```

### MemoryBridge

**Location:** `src/memory-bridge.mjs`

**Purpose:** Unified coordinator — one API for all memory operations

**Key Features:**
- Bidirectional sync: vault → graph, vault ↔ Claude memory
- Auto-pull: detects external changes in Claude memory, auto-merges
- Event-driven: subscribes to `note:created/updated/deleted`, `session:stop`
- Unified query: `queryContext("topic")` → graph + sessions + vault results
- Lifecycle maintenance: `maintenance()` runs decay + promote + cleanup

### CLI Commands

```bash
# Full bidirectional sync
clausidian memory full-sync

# Graph operations
clausidian memory graph stats
clausidian memory graph neighbors --node api-project --depth 2
clausidian memory graph query --query "backend"
clausidian memory graph connections --node api-project
clausidian memory graph hubs
clausidian memory graph decay

# Session operations
clausidian memory session start --topic "api-design"
clausidian memory session end --decisions "Use Fastify" --learnings "Validate input"
clausidian memory session stats
clausidian memory session recent --days 7
clausidian memory session pending
clausidian memory session learnings
clausidian memory session context --topic "api-design"
clausidian memory session cleanup

# Lifecycle
clausidian memory lifecycle promote
clausidian memory lifecycle stale --days 30
clausidian memory lifecycle maintenance
clausidian memory lifecycle diagnostics

# Unified context
clausidian memory context "api design"
```

### Integration with EventBus

| Event | Action |
|-------|--------|
| `note:created` | Add node to graph, record in session, push if memory:true |
| `note:updated` | Update node metadata, re-push if memory:true |
| `note:deleted` | Remove node from graph, remove from Claude memory |
| `session:stop` | End session with decisions/learnings/nextSteps |

---

Expand Down Expand Up @@ -331,4 +390,4 @@ invalidateCache() {

---

Last updated: 2026-03-30 (v3.1.0)
Last updated: 2026-04-02 (v3.6.0)
56 changes: 56 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,62 @@

All notable changes to this project will be documented in this file.

## [3.6.0] - 2026-04-02

### Added — Dynamic Vault-Memory Management System
- **MemoryGraph** (`src/memory-graph.mjs`, 320 LOC)
- Graph-based memory relationship tracking with weighted edges
- Context-aware retrieval via graph traversal + relevance scoring
- Automatic decay (0.95/day) and promotion (access count threshold)
- Vault sync: auto-creates nodes + edges from notes, related links, shared tags
- Persistent storage in `.clausidian/memory-graph.json`
- Edge pruning (max 20/node, min weight 0.1)

- **SessionMemory** (`src/session-memory.mjs`, 280 LOC)
- Session lifecycle: start → record events → end/abandon
- Tracks decisions, learnings, next steps per session
- Context window builder (combines current + recent sessions + graph)
- Pending step tracking across sessions
- Aggregated learnings with frequency counting
- Auto-extraction of decisions from note creation patterns
- Session storage in `.clausidian/sessions/*.json`

- **MemoryBridge** (`src/memory-bridge.mjs`, 250 LOC)
- Unified coordinator for MemoryGraph + SessionMemory + Claude memory
- Full bidirectional sync: vault ↔ graph, vault ↔ Claude memory
- Auto-pull from Claude memory (detects external changes, auto-merges)
- Event-driven: auto-sync on note:created/updated/deleted
- Unified context query (graph + sessions + vault search combined)
- Lifecycle maintenance: decay + promote + stale detection + cleanup

- **Enhanced CLI Commands** (expanded `memory` subcommands)
- `memory full-sync` — full bidirectional sync with graph + lifecycle
- `memory graph <action>` — stats|sync|neighbors|query|connections|hubs|decay
- `memory session <action>` — start|end|stats|recent|pending|learnings|context|cleanup
- `memory lifecycle <action>` — promote|stale|maintenance|diagnostics
- `memory context <topic>` — unified context (graph + sessions + vault)
- All commands available as MCP tools (memory_graph, memory_session, memory_lifecycle, memory_context)

- **New Event Types** (10 new events)
- `memory:node_added`, `memory:edge_added`, `memory:decay_applied`, `memory:promoted`
- `session:start`, `session:stop`, `session:abandoned`
- `memory:full_sync`, `memory:pushed`, `memory:pulled`

- **Tests** (`test/memory-system.test.mjs`, 26 tests)
- MemoryGraph: 11 tests (nodes, edges, traversal, context, decay, promotion, stats)
- SessionMemory: 10 tests (lifecycle, events, persistence, cleanup, stats)
- MemoryBridge: 5 tests (sync, context, diagnostics, maintenance)

### Changed
- Refactored `commands/memory.mjs` to integrate MemoryBridge (backward compatible)
- Updated `registry/integration.mjs` with new subcommands
- Updated `events/event-types.mjs` with memory/session event patterns

### Infrastructure
- 406 tests passing (26 new), 1 pre-existing failure (unrelated)
- Zero new dependencies (uses only Node.js stdlib)
- All new modules follow existing ESM + zero-dep patterns

## [3.5.0] - 2026-03-31

### Added
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "clausidian",
"version": "3.5.0",
"version": "3.6.0",
"description": "Claude Code's Obsidian integration — AI agent toolkit for vault management, journal, notes, search, index sync, and more",
"type": "module",
"bin": {
Expand Down
12 changes: 12 additions & 0 deletions scaffold/.claude/commands/capture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Quick capture an idea.

Run: `clausidian capture "<idea text>"`

If the CLI is not available:
1. Extract a title from the idea text
2. Read `templates/idea.md` and replace placeholders
3. Search for related notes, fill `related` field
4. Write to `ideas/` with lowercase-hyphen filename
5. Update indices

$ARGUMENTS
12 changes: 12 additions & 0 deletions scaffold/.claude/commands/journal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Create or open today's journal entry.

Run: `clausidian journal`

If the CLI is not available, follow these manual steps:
1. Calculate today's date (YYYY-MM-DD) and weekday
2. Check if `journal/YYYY-MM-DD.md` exists
3. If exists: read and display it
4. If not: read `templates/journal.md`, replace all `{{}}` placeholders, write to `journal/YYYY-MM-DD.md`
5. Update `journal/_index.md`, `_tags.md`, `_graph.md`

$ARGUMENTS
15 changes: 15 additions & 0 deletions scaffold/.claude/commands/list.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
List notes in the knowledge base.

Usage: `[type] [--status STATUS] [--tag TAG] [--recent N]`

Run: `clausidian list [type] [--status STATUS] [--tag TAG] [--recent N]`

If the CLI is not available:
1. Parse filter parameters
2. Scan frontmatter across all note directories
3. Apply filters (type, status, tag, recent days)
4. Display as table: file, title, type, status, summary, updated
5. Sort by updated date descending
6. Show stats at the end

$ARGUMENTS
16 changes: 16 additions & 0 deletions scaffold/.claude/commands/note.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Create a new note.

Usage: `<title> <type>` where type is area/project/resource/idea

Run: `clausidian note "<title>" <type>`

If the CLI is not available, follow these manual steps:
1. Read `CONVENTIONS.md`
2. Read the template for the given type from `templates/`
3. Replace all `{{}}` placeholders with actual values
4. Search for related notes and fill `related` field
5. Write to the correct directory (area→areas/, project→projects/, etc.)
6. Update `_index.md`, `_tags.md`, `_graph.md`
7. Update reverse links on related notes (bidirectional linking)

$ARGUMENTS
Loading
Loading