NotesAI is an aesthetic note generation platform that transforms lecture PDFs into beautifully formatted, comprehensive study notes using RAG (Retrieval Augmented Generation) with Claude AI.
Core Value Proposition:
- Aesthetics: EB Garamond for headings (H1, H2, H3), Inter for body text
- Completeness: Comprehensive, detailed notes (not summaries) that preserve all lecture content
- Intelligence: RAG-powered generation using Claude API and Pinecone vector database
- Framework: Next.js 14 (App Router)
- Language: TypeScript
- Styling: Tailwind CSS
- UI Components: Custom components (Button, Card, Badge, etc.)
- Fonts: EB Garamond (headings), Inter (body)
- Framework: FastAPI (Python)
- Task Queue: Celery + Redis
- Database: PostgreSQL (metadata, job status, generated notes)
- Vector Database: Pinecone (document embeddings)
- LLM: Claude API (Anthropic)
- Embeddings: OpenAI text-embedding-3-small
- File Storage: Local/S3 for PDF uploads
- Caching: Redis
- Background Jobs: Celery workers
┌─────────────────────────────────────────────────────────────┐
│ Frontend (Next.js) │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Landing Page │ │ Dashboard │ │ Note Viewer │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└────────────────────────┬────────────────────────────────────┘
│ HTTP/REST
▼
┌─────────────────────────────────────────────────────────────┐
│ Backend (FastAPI) │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ API Layer │ │
│ │ /upload /status/{id} /notes/generate/{id} │ │
│ └────────────────────┬─────────────────────────────────┘ │
│ │ │
│ ┌────────────────────┴─────────────────────────────────┐ │
│ │ Business Logic Layer │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌────────────┐ │ │
│ │ │ PDF Processor│ │ RAG Engine │ │ Claude │ │ │
│ │ └──────────────┘ └──────────────┘ │ Client │ │ │
│ │ └────────────┘ │ │
│ └────────────────────┬─────────────────────────────────┘ │
└───────────────────────┼─────────────────────────────────────┘
│
┌───────────────┼───────────────┐
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌──────────┐ ┌────────────────┐
│ PostgreSQL │ │ Pinecone │ │ Celery Workers │
│ (Metadata) │ │ (Vectors)│ │ (Redis) │
└──────────────┘ └──────────┘ └────────────────┘
User uploads PDF
│
▼
┌─────────────────────────────────────────┐
│ 1. PDF Upload & Validation │
│ - Check file type (.pdf) │
│ - Check file size (< 50MB) │
│ - Generate unique doc_id (UUID) │
│ - Save to storage │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 2. Trigger Background Job (Celery) │
│ - Return doc_id immediately │
│ - Process asynchronously │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 3. Text Extraction (PyMuPDF) │
│ - Extract text from each page │
│ - Detect structure (headings, etc.) │
│ - Identify formulas, diagrams │
│ - OCR if needed │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 4. Intelligent Chunking │
│ Strategy: │
│ - Semantic boundaries (topics) │
│ - 400-1000 tokens per chunk │
│ - 200 token overlap │
│ - Preserve slide/section integrity │
│ │
│ Output: Array of chunks with: │
│ { │
│ chunk_id: "page1_0", │
│ text: "...", │
│ page: 1, │
│ heading: "Introduction", │
│ chunk_type: "definition", │
│ has_formula: true, │
│ chunk_index: 0 │
│ } │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 5. Generate Embeddings │
│ - Use OpenAI text-embedding-3-small │
│ - 1536-dimensional vectors │
│ - Batch process for efficiency │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 6. Store in Pinecone │
│ - Namespace: doc_id │
│ - Vector + metadata │
│ - Enable filtering by page, type │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 7. Update Job Status │
│ - Mark as "ready_for_generation" │
│ - Store metadata in PostgreSQL │
└─────────────────────────────────────────┘
For comprehensive notes that preserve all content:
- Challenge: PDFs can exceed Claude's context window
- Solution: Hierarchical generation
- Map: Generate detailed notes per section
- Reduce: Combine sections into final structured output
User clicks "Generate Notes"
│
▼
┌─────────────────────────────────────────┐
│ 1. Retrieve ALL Chunks from Pinecone │
│ - Fetch entire doc_id namespace │
│ - Sort chronologically (page order) │
│ - ~50-200 chunks depending on PDF │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 2. Group Chunks by Section/Topic │
│ Strategy: │
│ - Use heading metadata │
│ - Detect topic boundaries │
│ - Group related chunks │
│ │
│ Output: │
│ sections = { │
│ "Introduction": [chunk1, chunk2], │
│ "Topic 1": [chunk5, chunk6, ...], │
│ "Topic 2": [chunk10, ...] │
│ } │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 3. MAP PHASE: Process Each Section │
│ │
│ For each section: │
│ a) Combine chunk texts │
│ b) Call Claude API with prompt: │
│ │
│ "Generate comprehensive notes │
│ for this section. Include │
│ ALL details, definitions, │
│ examples, formulas..." │
│ │
│ c) Get structured JSON response: │
│ { │
│ "heading": "...", │
│ "subsections": [ │
│ { │
│ "subheading": "...", │
│ "points": [...], │
│ "examples": [...], │
│ "formulas": [...] │
│ } │
│ ], │
│ "keyTerms": [...] │
│ } │
│ │
│ Execute in parallel for speed │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 4. REDUCE PHASE: Combine Sections │
│ │
│ a) Merge all section_notes │
│ b) Generate executive summary │
│ (brief overview of full lecture) │
│ c) Consolidate all key terms │
│ (deduplicate, alphabetize) │
│ d) Create quiz questions │
│ (from all sections) │
│ │
│ Final structure: │
│ { │
│ "title": "Lecture X: ...", │
│ "summary": "Brief overview", │
│ "keyTerms": [...], │
│ "sections": [ │
│ {section1_notes}, │
│ {section2_notes}, │
│ ... │
│ ], │
│ "quiz": [...] │
│ } │
└─────────────────┬───────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ 5. Save to Database │
│ - Store in PostgreSQL │
│ - Cache in Redis │
│ - Return to frontend │
└─────────────────────────────────────────┘
POST /api/upload
Content-Type: multipart/form-data
Request:
- file: PDF file (< 50MB)
Response:
{
"doc_id": "uuid",
"status": "processing",
"message": "PDF uploaded. Processing started."
}Flow:
- Validate PDF
- Generate doc_id
- Save file
- Trigger Celery task
- Return immediately
GET /api/status/{doc_id}
Response:
{
"doc_id": "uuid",
"status": "processing" | "ready" | "generating" | "completed" | "failed",
"progress": 65,
"stage": "Generating embeddings",
"chunks_processed": 45,
"total_chunks": 67
}Statuses:
processing: PDF being chunked/embeddedready: Ready for note generationgenerating: Notes being generatedcompleted: Notes readyfailed: Error occurred
POST /api/notes/generate/{doc_id}
Response:
{
"doc_id": "uuid",
"status": "generating",
"message": "Note generation started"
}
// Then poll GET /api/notes/{doc_id} for resultGET /api/notes/{doc_id}
Response:
{
"doc_id": "uuid",
"status": "completed",
"notes": {
"title": "Lecture 4: Renaissance",
"summary": "...",
"keyTerms": [
{"term": "Humanism", "definition": "..."}
],
"sections": [
{
"heading": "The Medici Influence",
"introduction": "...",
"subsections": [
{
"subheading": "Rise to Power",
"points": [...],
"examples": [...],
"formulas": []
}
]
}
],
"quiz": [...]
},
"generated_at": "2024-01-15T10:30:00Z"
}File: backend/app/services/pdf_processor.py
Responsibilities:
- Extract text from PDF (PyMuPDF)
- Detect document structure (headings, bullets, formulas)
- Chunk intelligently by semantic boundaries
- Preserve metadata (page numbers, headings, types)
Key Methods:
class PDFProcessor:
def extract_and_chunk(pdf_path: str) -> List[Chunk]
def parse_structure(blocks: List) -> StructuredContent
def chunk_by_topics(content, min_tokens, max_tokens, overlap) -> List[Chunk]
def contains_formula(text: str) -> boolFile: backend/app/services/embeddings.py
Responsibilities:
- Generate vector embeddings for text chunks
- Batch processing for efficiency
- Handle API rate limits
Key Methods:
class EmbeddingService:
def embed(text: str) -> List[float]
def embed_batch(texts: List[str]) -> List[List[float]]Model: OpenAI text-embedding-3-small (1536 dimensions)
File: backend/app/services/pinecone_client.py
Responsibilities:
- Store document embeddings with metadata
- Retrieve chunks (all or by query)
- Manage namespaces (one per document)
Key Methods:
class PineconeClient:
def upsert_chunks(namespace: str, chunks: List[Chunk])
def fetch_all(namespace: str) -> List[Chunk]
def query(namespace: str, query_embedding: List[float], top_k: int)
def delete_namespace(namespace: str)Index Configuration:
- Dimension: 1536
- Metric: Cosine similarity
- Namespaces: One per doc_id
File: backend/app/core/rag.py
Responsibilities:
- Orchestrate retrieval and generation
- Implement Map-Reduce pattern
- Assemble context for Claude
Key Methods:
class RAGEngine:
def retrieve_all_content(doc_id: str) -> List[Chunk]
def group_by_section(chunks: List[Chunk]) -> Dict[str, List[Chunk]]
def generate_section_notes(section_name: str, chunks: List[Chunk]) -> Dict
def combine_sections(section_notes: List[Dict]) -> Dict
def assemble_context(chunks: List[Chunk]) -> strFile: backend/app/services/claude_client.py
Responsibilities:
- Interface with Anthropic Claude API
- Manage prompts and responses
- Handle token limits and errors
Key Methods:
class ClaudeClient:
def generate(system: str, user: str, max_tokens: int) -> str
def generate_structured(prompt: str, schema: Dict) -> DictModel: Claude 3.5 Sonnet (200k context window)
User → Frontend → FastAPI → Celery → Worker
↓
PostgreSQL (job created)
Worker → PyMuPDF → Chunks → OpenAI → Embeddings → Pinecone
↓
PostgreSQL (status: ready)
User → Frontend → FastAPI → RAG Engine
↓
Pinecone (fetch chunks)
↓
Group by sections
↓
┌──────────┴──────────┐
▼ ▼
Claude API Claude API
(Section 1) (Section 2) ... (Parallel)
▼ ▼
└──────────┬──────────┘
▼
Combine & Structure
▼
PostgreSQL + Redis
▼
Frontend
CREATE TABLE documents (
id UUID PRIMARY KEY,
filename VARCHAR(255),
file_size INTEGER,
upload_timestamp TIMESTAMP,
status VARCHAR(50), -- processing, ready, completed, failed
total_pages INTEGER,
total_chunks INTEGER,
error_message TEXT
);CREATE TABLE notes (
id UUID PRIMARY KEY,
doc_id UUID REFERENCES documents(id),
title VARCHAR(500),
content JSONB, -- Full note structure
generated_at TIMESTAMP,
generation_time_seconds INTEGER
);CREATE TABLE processing_jobs (
id UUID PRIMARY KEY,
doc_id UUID REFERENCES documents(id),
job_type VARCHAR(50), -- embedding, generation
status VARCHAR(50),
progress INTEGER,
current_stage VARCHAR(100),
created_at TIMESTAMP,
updated_at TIMESTAMP
);You are an expert educational content creator specializing in comprehensive,
aesthetic study notes.
YOUR TASK: Transform lecture content into detailed, beautifully structured
notes that preserve ALL information while enhancing clarity and organization.
REQUIREMENTS:
1. COMPLETENESS: Include every concept, definition, example, and formula
2. STRUCTURE: Use clear hierarchical organization
3. CLARITY: Explain complex concepts thoroughly
4. EXAM-READY: Format for effective studying
OUTPUT: JSON with this exact structure:
{
"heading": "Section title",
"introduction": "Brief context for this section",
"subsections": [
{
"subheading": "Subtopic name",
"points": ["Detailed point 1", "Detailed point 2", ...],
"examples": ["Complete example with explanation"],
"formulas": [
{
"formula": "Mathematical expression",
"explanation": "What it means and when to use it",
"variables": {"x": "description", ...}
}
]
}
],
"keyTerms": [
{"term": "Exact term", "definition": "Complete definition"}
]
}
Generate comprehensive, detailed notes for the following section.
SECTION: {section_name}
CONTENT:
{chunk_texts}
Remember:
- Include ALL information (don't summarize or skip)
- Preserve all definitions, formulas, examples
- Explain technical terms
- Maintain logical flow
- Create study-ready reference material
notes_ai/
├── frontend/
│ ├── app/
│ │ ├── page.tsx # Landing page
│ │ ├── chat/
│ │ │ └── page.tsx # Dashboard route
│ │ ├── layout.tsx
│ │ └── globals.css
│ ├── components/
│ │ ├── ui.tsx # UI components
│ │ ├── dashboard.tsx # Main dashboard
│ │ └── Sidebar.tsx # Sidebar component
│ └── package.json
│
├── backend/
│ ├── app/
│ │ ├── main.py # FastAPI entry point
│ │ ├── api/
│ │ │ └── routes/
│ │ │ ├── upload.py # PDF upload endpoint
│ │ │ ├── notes.py # Note generation
│ │ │ └── status.py # Status polling
│ │ ├── core/
│ │ │ ├── config.py # Environment config
│ │ │ ├── rag.py # RAG engine
│ │ │ └── chunking.py # Chunking logic
│ │ ├── services/
│ │ │ ├── pdf_processor.py # PDF extraction
│ │ │ ├── embeddings.py # Embedding service
│ │ │ ├── pinecone_client.py
│ │ │ └── claude_client.py
│ │ ├── models/
│ │ │ └── schemas.py # Pydantic models
│ │ └── workers/
│ │ └── celery_worker.py # Background tasks
│ ├── requirements.txt
│ └── .env
│
├── Architecture.md # This file
└── README.md
- Single PDF processing (no multi-document workspace)
- Sequential section processing in map phase
- Local file storage
- Parallel Map Processing: Process sections concurrently
- Caching: Cache embeddings for frequently uploaded PDFs
- Streaming: Stream Claude responses for real-time updates
- Multi-document: Support multiple PDFs in one workspace
- Incremental Updates: Add new slides without re-processing entire PDF
- Invalid file type → 400 error
- File too large → 413 error
- Storage failure → 500 error, retry logic
- PDF extraction failure → Mark job as failed, notify user
- Embedding API failure → Retry with exponential backoff
- Pinecone timeout → Retry, fallback to partial processing
- Claude API failure → Retry with different prompts
- Token limit exceeded → Fall back to smaller chunks
- Invalid JSON response → Parse with error recovery
- Upload Response: < 500ms
- PDF Processing: 30-60 seconds for 50-page PDF
- Note Generation: 2-4 minutes for comprehensive notes
- API Latency: < 200ms for status checks
- Track processing times per stage
- Monitor API call success rates
- Log token usage for cost optimization
- Rate limiting on upload endpoint
- File type validation (magic bytes check)
- Max file size enforcement
- CORS configuration for frontend domain
- Document IDs are UUIDs (not sequential)
- PDFs deleted after processing (configurable retention)
- Notes stored encrypted at rest
- No user authentication in MVP (add later)
# Backend (.env)
CLAUDE_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...
PINECONE_API_KEY=...
PINECONE_ENVIRONMENT=us-east-1
PINECONE_INDEX_NAME=notes-ai
DATABASE_URL=postgresql://user:pass@localhost/notesai
REDIS_URL=redis://localhost:6379/0
MAX_FILE_SIZE_MB=50
PDF_RETENTION_DAYS=7- Start PostgreSQL and Redis
- Start Celery worker:
celery -A app.workers.celery_worker worker - Start FastAPI:
uvicorn app.main:app --reload - Start Next.js:
npm run dev
- Unit tests for chunking logic
- Integration tests for RAG pipeline
- E2E tests for upload → generation flow
Users → Vercel (Next.js) → API Gateway
↓
AWS ECS (FastAPI)
↓
┌─────────┴─────────┐
▼ ▼
RDS PostgreSQL ElastiCache Redis
▼ ▼
Pinecone Celery Workers (ECS)
▼
S3 (PDFs)
This architecture provides:
- ✅ Scalability: Map-Reduce handles large PDFs
- ✅ Quality: Comprehensive notes with all content preserved
- ✅ Performance: Async processing, parallel generation
- ✅ Aesthetics: Structured output ready for beautiful rendering
- ✅ Maintainability: Clean separation of concerns
The hierarchical Map-Reduce approach ensures we generate complete, detailed study notes while staying within API limits and maintaining high quality.