🧠 Memory System For AI

A human-inspired memory system for AI agents, connected via MCP (Model Context Protocol).
Modeled after how human memory works — episodic, semantic, and procedural — with dual-index Q&A search for high recall.

Works with: Cursor · Claude Desktop · Kiro · Antigravity · any MCP-compatible client

System Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         MCP Client                                  │
│        (Cursor / Claude Desktop / Kiro / Antigravity / …)           │
└──────────────────────────────┬──────────────────────────────────────┘
                               │  MCP Protocol (stdio)
                               ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     mcp_memory_server.py                            │
│                        20 MCP Tools                                 │
│                                                                     │
│  save_experience  search_experiences  save_fact   get_all_facts     │
│  create_summary   smart_recall        append_fact get_fact          │
│  write_pattern    read_pattern        get_facts_by_category         │
│  delete_pattern   list_patterns       write_skill  read_skill      │
│  delete_skill     list_skills         get_recent_history            │
│  delete_memory    delete_fact                                       │
└──────────────────────────────┬──────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│                      MemoryManager                                  │
│                   (core/memory_manager.py)                          │
│           Orchestrates storage and retrieval                        │
└──────┬──────────────────────┬──────────────────────┬────────────────┘
       │                      │                      │
       ▼                      ▼                      ▼
┌──────────────────┐  ┌──────────────────┐  ┌────────────────────────┐
│   VectorEngine   │  │  StructuredStore │  │   FileStore — NEW      │
│ (vector_engine)  │  │ (structured_store│  │   (file_store.py)      │
│                  │  │                  │  │                        │
│ Dual-index Q&A   │  │ Key-value facts  │  │ Patterns & Skills      │
│ Episodic memory  │  │ Memory index     │  │ as .md files on disk   │
│ Similarity+dedup │  │ Fact indexing    │  │ Read/Write/Delete/List │
└────────┬─────────┘  └────────┬─────────┘  └───────────┬────────────┘
         │                     │                         │
         ▼                     ▼                         ▼
┌──────────────────┐  ┌──────────────────┐  ┌────────────────────────┐
│  EmbeddingEngine │  │     SQLite       │  │   ~/.memory_system/    │
│ Sentence-Transf. │  │ ~/.memory_system/│  │     patterns/*.md      │
│ all-mpnet (768d) │  │    facts.db      │  │     skills/*.md        │
└────────┬─────────┘  └──────────────────┘  └────────────────────────┘
         │
         ▼
┌──────────────────┐
│    ChromaDB      │
│ ~/.memory_system/│
│   chroma_db/     │
└──────────────────┘
│       chroma_db/            │
└─────────────────────────────┘

Memory Types

Type	Inspired By	Storage	Purpose
Episodic	Human episodic memory	ChromaDB (vectors)	Experiences, lessons learned, mistakes, summaries
Semantic	Human semantic memory	SQLite (key-value)	Facts, user info, preferences, project details
Procedural	Human procedural memory	`.md` files + fact indexing	Patterns (small rules) and Skills (multi-step workflows)

Dual-Index Search

Every memory is stored as a Q&A pair with two vector embeddings:

                    ┌──────────────┐
                    │   Memory     │
                    │  (Q&A Pair)  │
                    └──────┬───────┘
                           │
              ┌────────────┴────────────┐
              ▼                         ▼
     ┌────────────────┐       ┌────────────────┐
     │  Q Embedding   │       │  A Embedding   │
     │  (question)    │       │  (answer)      │
     └────────────────┘       └────────────────┘
              │                         │
              ▼                         ▼
     Matches queries like:     Matches queries like:
     "how did we fix the       "Redis TTL
      caching bug?"             configuration"

A memory is found whether your query matches the question or the answer — significantly improving recall.

Features

3 memory types — Episodic (experiences), Semantic (facts), Procedural (file-based patterns & skills)
Dual-index Q&A search — every memory stored as question + answer with two embeddings for high recall
Token-efficient — semantic search returns only relevant memories (~150 tokens vs loading everything)
File-based patterns & skills — human-readable .md files with automatic fact indexing for discovery
Pattern indexing — patterns and skills auto-indexed as facts for cheap discovery via get_all_facts()
20 stateless MCP tools — save, search, recall, delete, read/write patterns & skills — no session management required
Project tagging — tag memories by project, search within or across projects
Similarity threshold — filters out noise with configurable minimum similarity score
Fully stateless — no sessions, no global state, each tool call is self-contained

Quick Start

1. Clone & Install

git clone https://github.com/AhmeedGamil/memory_system_for_ai.git
cd memory_system_for_ai

# Create virtual environment
python -m venv .venv

# Activate it
# Windows:
.venv\Scripts\activate
# macOS/Linux:
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

Note: First run downloads the embedding model (~420 MB). This only happens once.

Or use the one-command setup script

# Windows:
setup.bat

# macOS/Linux:
chmod +x setup.sh && ./setup.sh

This does everything above automatically and prints the exact paths you need for step 2.

2. Configure Your AI Tool

Add the memory server to your AI tool's MCP configuration:

Cursor

Go to Settings → MCP Servers → Add Server, then use:

Command: /full/path/to/memory_system_for_ai/.venv/bin/python
Args: /full/path/to/memory_system_for_ai/mcp_memory_server.py

Or edit .cursor/mcp.json in your project:

{
  "mcpServers": {
    "memory": {
      "command": "/full/path/to/memory_system_for_ai/.venv/bin/python",
      "args": ["/full/path/to/memory_system_for_ai/mcp_memory_server.py"]
    }
  }
}

Claude Desktop

Edit claude_desktop_config.json:

Windows: %APPDATA%\Claude\claude_desktop_config.json
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "memory": {
      "command": "/full/path/to/memory_system_for_ai/.venv/bin/python",
      "args": ["/full/path/to/memory_system_for_ai/mcp_memory_server.py"]
    }
  }
}

Kiro

Edit .kiro/settings/mcp.json in your project (or ~/.kiro/settings/mcp.json for global):

{
  "mcpServers": {
    "memory": {
      "command": "/full/path/to/memory_system_for_ai/.venv/bin/python",
      "args": ["/full/path/to/memory_system_for_ai/mcp_memory_server.py"]
    }
  }
}

On Windows, use \\ paths:

{
  "mcpServers": {
    "memory": {
      "command": "C:\\path\\to\\memory_system_for_ai\\.venv\\Scripts\\python.exe",
      "args": ["C:\\path\\to\\memory_system_for_ai\\mcp_memory_server.py"]
    }
  }
}

Antigravity

Edit mcp_config.json in your Antigravity config directory:

Windows: C:\Users\<username>\.gemini\antigravity\mcp_config.json
macOS: ~/.gemini/antigravity/mcp_config.json

{
  "mcpServers": {
    "memory": {
      "command": "/full/path/to/memory_system_for_ai/.venv/bin/python",
      "args": ["/full/path/to/memory_system_for_ai/mcp_memory_server.py"]
    }
  }
}

On Windows, use \\ paths:

{
  "mcpServers": {
    "memory": {
      "command": "C:\\path\\to\\memory_system_for_ai\\.venv\\Scripts\\python.exe",
      "args": ["C:\\path\\to\\memory_system_for_ai\\mcp_memory_server.py"]
    }
  }
}

3. Tell Your AI How to Use It

Copy the contents of MEMORY_SYSTEM_GUIDE.md into your AI's system prompt or custom instructions. This tells the AI when and how to use the memory tools.

That's it — your AI now has persistent memory.

MCP Tools Reference

Episodic Memory

Tool	Description
`save_experience`	Save an experience as a Q&A pair (tagged with project)
`create_summary`	Summarize progress at milestones or before switching context
`search_experiences`	Search all episodic memories (optional project filter)

Semantic Memory

Tool	Description
`save_fact`	Store a key-value fact
`append_fact`	Append to a list-type fact
`get_fact`	Get a specific fact by key
`get_all_facts`	Get all stored facts

Procedural Memory (Patterns)

Tool	Description
`write_pattern`	Write a pattern as a `.md` file with automatic fact indexing
`read_pattern`	Read a pattern's full content by name
`delete_pattern`	Delete a pattern file and its fact index entry
`list_patterns`	List all pattern names from disk

Procedural Memory (Skills)

Tool	Description
`write_skill`	Write a skill as a `.md` file with automatic fact indexing
`read_skill`	Read a skill's full content by name
`delete_skill`	Delete a skill file and its fact index entry
`list_skills`	List all skill names from disk
`smart_recall`	Search episodic memories (patterns/skills accessed directly by name)

Memory Management

~/.memory_system/
├── facts.db          ← SQLite (semantic facts + memory index + pattern/skill indexes)
├── chroma_db/        ← ChromaDB (vector embeddings for episodic memory)
├── patterns/         ← Pattern .md files (procedural memory — small rules)
└── skills/           ← Skill .md files (procedural memory — multi-step workflows)

Recall & History

Tool	Description
`smart_recall`	Search all memory types at once
`get_recent_history`	Get recent experiences and summaries chronologically

mcp_memory_server.py       ← MCP entry point (21 tools)
MEMORY_SYSTEM_GUIDE.md     ← AI agent instructions for using the memory system
setup.bat / setup.sh       ← One-command setup scripts
core/
├── config.py              ← User-configurable settings
├── memory_manager.py      ← Orchestration layer
├── vector_engine.py       ← ChromaDB dual-index Q&A search
├── structured_store.py    ← SQLite facts + memory index
├── file_store.py          ← File-based pattern/skill storage (.md files)
└── embedding_engine.py    ← Sentence-Transformers embeddings
tests/
├── test_file_store.py              ← FileStore unit tests
├── test_file_store_properties.py   ← FileStore property-based tests
├── test_memory_manager.py          ← MemoryManager unit tests
├── test_memory_manager_properties.py ← MemoryManager property-based tests
├── test_delete.py
└── test_speed.py

Project Structure

mcp_memory_server.py       ← MCP entry point (14 tools)
view_database.py           ← CLI tool to inspect stored data
setup.bat / setup.sh       ← One-command setup scripts
core/
├── config.py              ← User-configurable settings
├── memory_manager.py      ← Orchestration layer
├── vector_engine.py       ← ChromaDB dual-index Q&A search
├── structured_store.py    ← SQLite facts + memory index
└── embedding_engine.py    ← Sentence-Transformers embeddings
tests/
├── test_schema_refactor.py
├── test_delete.py
└── test_speed.py

Configuration

All settings are in core/config.py:

Setting	Default	Description
`DATA_DIR`	`~/.memory_system/`	Where all data is stored
`FACTS_LOAD_MODE`	`"all"`	`"all"` or `"categories"`
`FACTS_CATEGORIES`	`[]`	Categories to load (when mode is `"categories"`)
`FACTS_MAX_COUNT`	`None`	Max facts returned (`None` = unlimited)
`FACTS_MAX_VALUE_LENGTH`	`None`	Truncate long values (`None` = full text)
`FACTS_EXCLUDE_KEYS`	`[]`	Keys to hide from results
`SEARCH_TOP_K`	`5`	Default search results count
`SIMILARITY_THRESHOLD`	`0.2`	Minimum similarity score for results
`EMBEDDING_MODEL`	`"all-mpnet-base-v2"`	Sentence-Transformers model (768 dimensions)

Running Tests

# Integration tests
python tests/test_schema_refactor.py

# Delete functionality tests
python tests/test_delete.py

# Speed benchmark
python tests/test_speed.py

# View stored data
python view_database.py                           # Everything
python view_database.py facts                     # Facts only
python view_database.py experiences_and_patterns  # Experiences & patterns

License

MIT — Created by Ahmed Gamil, AI Systems Engineer

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

🧠 Memory System For AI

System Architecture

Memory Types

Dual-Index Search

Features

Quick Start

1. Clone & Install

2. Configure Your AI Tool

3. Tell Your AI How to Use It

MCP Tools Reference

Episodic Memory

Semantic Memory

Procedural Memory (Patterns)

Procedural Memory (Skills)

Memory Management

Recall & History

Project Structure

Configuration

Running Tests

License

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
.kiro/specs/pattern-skill-file-storage		.kiro/specs/pattern-skill-file-storage
core		core
tests		tests
.gitignore		.gitignore
MEMORY_SYSTEM_GUIDE.md		MEMORY_SYSTEM_GUIDE.md
README.md		README.md
mcp_config.json		mcp_config.json
mcp_memory_server.py		mcp_memory_server.py
requirements.txt		requirements.txt
setup.bat		setup.bat
setup.sh		setup.sh

Folders and files

Latest commit

History

Repository files navigation

🧠 Memory System For AI

System Architecture

Memory Types

Dual-Index Search

Features

Quick Start

1. Clone & Install

2. Configure Your AI Tool

3. Tell Your AI How to Use It

MCP Tools Reference

Episodic Memory

Semantic Memory

Procedural Memory (Patterns)

Procedural Memory (Skills)

Memory Management

Recall & History

Project Structure

Configuration

Running Tests

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages