diff --git a/.gitignore b/.gitignore index f9ba7f8b..c3b209b7 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,8 @@ node_modules dist +data/ .DS_Store server/public vite.config.ts.* -*.tar.gz \ No newline at end of file +*.tar.gz +.env \ No newline at end of file diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 00000000..8acea461 --- /dev/null +++ b/.mcp.json @@ -0,0 +1,15 @@ +{ + "mcpServers": { + "aiqa": { + "command": "npx", + "args": [ + "aiqastudio-mcp" + ], + "env": { + "AIQA_API_KEY": "aiq_425865884d0ef800ba96bab750fc43f6", + "AIQA_PROJECT_ID": "eacc71c9-f5f5-4d1b-a798-3f47b99a04da", + "AIQA_URL": "https://aiqastudio.com" + } + } + } +} diff --git a/CLAUDE.md b/CLAUDE.md index 04bcac11..3cc40c93 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,18 +5,48 @@ **Provocations** is an AI-augmented document workspace where users iteratively shape ideas into polished documents through voice, text, and thought-provoking AI interactions. Think of it as a **smarter Google Docs** — the document is the output, and the AI helps you create and refine it through: -- **Multiple perspectives** (lenses) that challenge your thinking -- **Provocations** that push you to address gaps, fallacies, and alternatives +- **Multiple personas** (14 expert perspectives) that challenge your thinking +- **Provocations** that push you to address gaps, assumptions, and alternatives - **Voice input** for natural ideation and feedback - **Iterative shaping** where each interaction evolves the document +- **Screen capture** mode for requirement discovery through annotations **Core Philosophy**: "Would you rather have a tool that thinks for you, or a tool that makes you think?" The AI doesn't write for you — it provokes deeper thinking so *you* write better. +## AIQA — QA Bug Tracking + +The AIQA QA overlay is embedded in `client/index.html`. Environment variables are in `.env` (`AIQA_URL`, `AIQA_API_KEY`, `AIQA_PROJECT_ID`). MCP integration is in `.mcp.json` (uses `aiqastudio-mcp` package). + +### Fetch bugs and feature requests + +```bash +curl -H "X-API-Key: 3b03bd31-93b9-4efe-9c3d-4d6a54db97f0" \ + "https://aiqastudio.replit.app/api/bugs?project_id=0681dd20-4cec-4a1a-8280-7c7b33fdaee7" +``` + +Filter options: `&submission_type=bug_report`, `&submission_type=feature_request`, `&status=open`, `&urgency=high`. + +### Other AIQA API endpoints + +```bash +# Check test results +GET https://aiqastudio.replit.app/api/projects/0681dd20-4cec-4a1a-8280-7c7b33fdaee7/results + +# List test cases +GET https://aiqastudio.replit.app/api/test-cases?project_id=0681dd20-4cec-4a1a-8280-7c7b33fdaee7 + +# Stats overview +GET https://aiqastudio.replit.app/api/stats/overview?project_id=0681dd20-4cec-4a1a-8280-7c7b33fdaee7 +``` + +All endpoints require `X-API-Key: {AIQA_API_KEY}` header. + ## Quick Commands ```bash +npm install # Install dependencies (REQUIRED before any other command) npm run dev # Start development server (Express + Vite HMR on port 5000) npm run build # Build for production (outputs to dist/) npm run start # Run production build @@ -24,51 +54,145 @@ npm run check # TypeScript type checking npm run db:push # Push Drizzle schema to database ``` +> **Important:** You must run `npm install` before `npm run check` or `npm run build`. Without full dependencies installed, TypeScript will report false errors like `Cannot find type definition file for 'node'` and `Cannot find type definition file for 'vite/client'`. These are **not real code errors** — they indicate missing `node_modules` (specifically `@types/node` and `vite`). Always ensure dependencies are installed first. + ## Tech Stack | Layer | Technologies | |-------|-------------| -| **Frontend** | React 18, TypeScript, Vite 7, Tailwind CSS 3.4, shadcn/ui | -| **Backend** | Express 5.0, OpenAI GPT-5.2 | -| **Database** | PostgreSQL via Drizzle ORM (documents are in-memory only) | +| **Frontend** | React 18, TypeScript 5.6, Vite 7, Tailwind CSS 3.4, shadcn/ui (47 components) | +| **Backend** | Express 5.0, OpenAI GPT-4o (default) / Google Gemini 2.0 Flash / Anthropic Claude (configurable via `LLM_PROVIDER`) | +| **Database** | PostgreSQL via Drizzle ORM, zero-knowledge AES-256-GCM encryption (all user text encrypted at rest) | +| **Auth** | Clerk (authentication & user ownership) | | **Validation** | Zod schemas shared between frontend/backend | | **State** | React Query (TanStack), React hooks | -| **Routing** | Wouter (lightweight) | +| **Routing** | Wouter (lightweight client-side router) | +| **Voice** | Web Speech API + custom audio worklets | ## Directory Structure ``` provocations/ -├── client/src/ # React frontend +├── apps/ # Per-app CLAUDE.md documentation +│ ├── write-a-prompt/CLAUDE.md +│ ├── product-requirement/CLAUDE.md +│ ├── new-application/CLAUDE.md +│ ├── streaming/CLAUDE.md # Screen Capture app +│ ├── research-paper/CLAUDE.md +│ ├── persona-definition/CLAUDE.md +│ ├── research-context/CLAUDE.md +│ ├── voice-capture/CLAUDE.md +│ ├── youtube-to-infographic/CLAUDE.md +│ ├── text-to-infographic/CLAUDE.md +│ ├── email-composer/CLAUDE.md +│ └── agent-editor/CLAUDE.md +├── client/src/ │ ├── pages/ -│ │ └── Workspace.tsx # Main app interface (primary orchestrator) +│ │ ├── NotebookWorkspace.tsx # Primary 3-panel workspace +│ │ ├── ContextStore.tsx # Standalone /store page for context management +│ │ ├── Admin.tsx # Admin analytics dashboard +│ │ ├── Pricing.tsx # Pricing page +│ │ └── not-found.tsx # 404 page │ ├── components/ -│ │ ├── ui/ # 47 shadcn/ui components -│ │ ├── TextInputForm.tsx -│ │ ├── LensesPanel.tsx -│ │ ├── ProvocationsDisplay.tsx -│ │ ├── OutlineBuilder.tsx -│ │ ├── ReadingPane.tsx -│ │ ├── DimensionsToolbar.tsx -│ │ ├── DiffView.tsx -│ │ ├── VoiceRecorder.tsx -│ │ └── TranscriptOverlay.tsx -│ ├── hooks/ # Custom React hooks +│ │ ├── ui/ # 47 shadcn/ui primitives (Radix-based) +│ │ ├── notebook/ # Notebook layout components (see below) +│ │ │ ├── NotebookTopBar.tsx # Session header with name, version count, admin controls +│ │ │ ├── NotebookLeftPanel.tsx # Left panel: Context | Chat | Video tabs +│ │ │ ├── NotebookCenterPanel.tsx # Center: document editor + objective +│ │ │ ├── NotebookRightPanel.tsx # Right panel: Research | Notes | Provo | Generate tabs +│ │ │ ├── ContextSidebar.tsx # Document/folder tree with pin, search, inline edit +│ │ │ ├── NotebookResearchChat.tsx # Streaming research chat (SSE via /api/chat/stream) +│ │ │ ├── ProvoThread.tsx # Persona discussion thread with accept/respond +│ │ │ ├── TranscriptPanel.tsx # Notes panel: add/remove notes, save to context, evolve doc +│ │ │ ├── SplitDocumentEditor.tsx # Multi-tab editor with smart buttons (Expand/Condense/...) +│ │ │ ├── PersonaAvatarRow.tsx # Persona toggle row with avatars +│ │ │ └── ChatThread.tsx # User-to-user chat thread rendering +│ │ ├── bschart/ # BS Chart visual diagramming tool +│ │ │ ├── BSChartWorkspace.tsx # Main chart workspace with canvas, toolbar, properties +│ │ │ ├── BSChartCanvas.tsx # Canvas rendering layer +│ │ │ ├── BSChartToolbar.tsx # Node creation and format controls +│ │ │ ├── BSChartProperties.tsx # Property editor for nodes/connectors +│ │ │ ├── BSConnectorLayer.tsx # Edge/connector rendering +│ │ │ ├── types.ts # Chart type definitions +│ │ │ ├── nodes/BSNodeRenderer.tsx # Renders node types (table, diamond, rect, text, badge) +│ │ │ └── hooks/ # useChartState, useCanvasInteraction, useVoiceChartCommands +│ │ ├── ProvokeText.tsx # Smart text area with voice + processing (ADR: all text must use this) +│ │ ├── StoragePanel.tsx # Context Store browser (embeddable or standalone) +│ │ ├── ChatDrawer.tsx # User-to-user messaging drawer +│ │ ├── GeneratePanel.tsx # Document generation panel +│ │ ├── ArtifyPanel.tsx # Image generation from text +│ │ └── ... # See apps/*/CLAUDE.md for app-specific components +│ ├── hooks/ +│ │ ├── use-whisper.ts # Whisper-based voice recording +│ │ ├── use-role.ts # Admin role detection +│ │ ├── use-app-favorites.ts # App favorites persistence +│ │ └── ... │ ├── lib/ -│ │ ├── queryClient.ts # React Query config -│ │ └── utils.ts # Tailwind merge utilities -│ ├── App.tsx # Router setup -│ └── main.tsx # Entry point +│ │ ├── queryClient.ts # React Query config + apiRequest helper +│ │ ├── prebuiltTemplates.ts # Template definitions (15 app types) +│ │ ├── appWorkspaceConfig.ts # App workspace behavior configs +│ │ ├── workspace-context.tsx # Shared workspace context provider +│ │ ├── tracking.ts # Client-side usage tracking +│ │ ├── errorLog.ts # Error log store +│ │ ├── featureFlags.ts # Feature flag management +│ │ └── utils.ts # Tailwind merge utilities, generateId +│ ├── App.tsx # Router setup (NotebookWorkspace is default) +│ └── main.tsx # Entry point ├── server/ -│ ├── index.ts # Express app setup -│ ├── routes.ts # Core API endpoints -│ └── storage.ts # In-memory document storage +│ ├── index.ts # Express app setup +│ ├── routes.ts # All API endpoints +│ ├── llm.ts # Configurable LLM provider (OpenAI/Gemini/Anthropic) +│ ├── llm-gateway.ts # LLM call logging and cost tracking +│ ├── context-builder.ts # Per-app LLM system prompts & output config +│ ├── storage.ts # Database operations (Drizzle ORM) +│ ├── crypto.ts # AES-256-GCM encryption/decryption (zero-knowledge) +│ ├── agent-executor.ts # Agent workflow execution engine +│ ├── invoke.ts # Task type routing for LLM calls +│ ├── static.ts # Static file serving +│ └── db.ts # Database connection ├── shared/ -│ ├── schema.ts # Zod schemas & types -│ └── models/chat.ts # Drizzle ORM models -└── script/build.ts # Build configuration +│ ├── schema.ts # Zod schemas & TypeScript types (templateIds source of truth) +│ ├── personas.ts # 14 built-in persona definitions +│ └── models/chat.ts # Drizzle ORM table definitions (connections, conversations, messages) +└── script/build.ts # Production build configuration ``` +## Per-App Documentation + +Each of the 12 applications has its own `apps//CLAUDE.md` containing: +- **Identity**: Purpose, category, philosophy, user workflow +- **Three-layer definition**: Exact config from prebuiltTemplates, appWorkspaceConfig, context-builder +- **App-specific components**: Components unique to that app +- **App-specific API endpoints**: Endpoints only that app uses +- **Key behaviors**: What makes this app different from others + +**See `apps//CLAUDE.md` for app-specific guidance.** + +| Template ID | Title | Category | Layout | Writer Mode | +|-------------|-------|----------|--------|-------------| +| `write-a-prompt` | Write a Prompt | write | standard | edit | +| `product-requirement` | Product Requirement | build | standard | edit | +| `new-application` | New Application | build | standard | edit | +| `streaming` | Screen Capture | analyze | standard | edit | +| `research-paper` | Research Paper | write | standard | edit | +| `persona-definition` | Persona / Agent | write | standard | edit | +| `research-context` | Research into Context | capture | standard | aggregate | +| `voice-capture` | Voice Capture | capture | voice-capture | aggregate | +| `youtube-to-infographic` | YouTube to Infographic | capture | standard | edit | +| `text-to-infographic` | Text to Infographic | capture | infographic-studio | edit | +| `email-composer` | Email Composer | write | standard | edit | +| `agent-editor` | Agent Editor | build | standard | edit | +| `gpt-to-context` | GPT to Context | capture | research-chat | aggregate | +| `bs-chart` | BS Chart | build | bs-chart | edit | +| `query-editor` | Query Editor | analyze | external | — | + +### Context Store Sync + +Admin users can sync all per-app CLAUDE.md files into the document store: +- `POST /api/admin/sync-app-docs` — Reads `apps/*/CLAUDE.md` from disk, creates/updates encrypted documents in the admin's `Applications/Provocations/` folder hierarchy +- Auto-syncs on server startup for the admin user +- Each app's documentation becomes available as context when using any app via the Context Manager + ## Path Aliases Configured in `tsconfig.json`: @@ -91,106 +215,185 @@ Configured in `tsconfig.json`: ``` 1. **Input** — Start with rough ideas, notes, or existing material -2. **Analyze** — AI generates lenses (perspectives) and provocations (challenges) -3. **Respond** — Use voice or text to address provocations +2. **Analyze** — AI generates challenges from multiple persona perspectives +3. **Respond** — Use voice or text to address challenges 4. **Merge** — AI intelligently weaves your responses into the document 5. **Iterate** — Repeat until the document fully captures your thinking -### Key Insight +## API Endpoints -The document is not static input to be critiqued — it's a **living artifact** that grows through dialogue between you and the AI. Each provocation is an invitation to think deeper; each response shapes the final output. +All endpoints use Zod validation. Document endpoints require Clerk authentication. App-specific endpoints are documented in each app's `apps//CLAUDE.md`. -## API Endpoints +### Shared AI Endpoints (used by all apps) | Endpoint | Method | Purpose | |----------|--------|---------| -| `/api/analyze` | POST | Generate lenses & provocations from current document | -| `/api/expand` | POST | AI-expand outline headings into paragraphs | -| `/api/refine` | POST | Adjust tone & length of text | -| `/api/merge` | POST | Integrate voice/text feedback into document | -| `/api/edit-text` | POST | Edit selected text with instructions | - -### Request Schemas - -All schemas defined in `shared/schema.ts` with Zod validation. +| `/api/generate-challenges` | POST | Generate challenges from selected personas | +| `/api/generate-advice` | POST | Generate advice for a specific challenge | +| `/api/write` | POST | Unified document editor (edit, expand, refine) | +| `/api/write/stream` | POST | Streaming write for large documents (SSE) | +| `/api/summarize-intent` | POST | Clean voice transcripts into clear intent | +| `/api/interview/question` | POST | Generate next interview question | +| `/api/interview/summary` | POST | Synthesize interview entries into instructions | +| `/api/discussion/ask` | POST | Multi-persona response to user questions | -```typescript -// Analyze - generate perspectives and challenges -{ text: string, selectedLenses?: LensType[] } +### Research Chat Endpoints (NotebookWorkspace) -// Expand - develop outline into content -{ heading: string, context?: string, tone?: ToneOption } +| Endpoint | Method | Purpose | +|----------|--------|---------| +| `/api/chat/stream` | POST | Streaming research chat (SSE) | +| `/api/chat/summarize` | POST | Summarize a research chat session | +| `/api/chat/save-session` | POST | Save research chat session to context | +| `/api/chat/models` | GET | List available chat models | -// Refine - adjust style -{ text: string, tone: ToneOption, targetLength: "shorter" | "same" | "longer" } +### User-to-User Messaging Endpoints -// Merge - integrate feedback (supports voice context) -{ originalText: string, userFeedback: string, provocationContext?: string, selectedText?: string, activeLens?: LensType } +| Endpoint | Method | Purpose | +|----------|--------|---------| +| `/api/chat/connections/invite` | POST | Send connection invitation | +| `/api/chat/connections` | GET | List user's connections | +| `/api/chat/connections/respond` | POST | Accept/reject connection | +| `/api/chat/conversations` | GET | List conversations with users | +| `/api/chat/messages` | POST | Send message in conversation | +| `/api/chat/messages/:id` | GET | Fetch conversation history | +| `/api/chat/messages/:id/read` | POST | Mark messages as read | -// Edit - modify selection -{ instruction: string, selectedText: string, fullDocument: string } -``` +### Shared Data Endpoints -## Domain Concepts - -### Lenses (6 Perspectives) -Different viewpoints to examine your document: -- `consumer` — End-user/customer viewpoint -- `executive` — Strategic leadership view -- `technical` — Implementation constraints -- `financial` — Cost/ROI analysis -- `strategic` — Competitive positioning -- `skeptic` — Critical assumptions challenge - -### Provocations (3 Categories) -Challenges that push your thinking: -- `opportunity` — Growth/innovation gaps you might be missing -- `fallacy` — Logical errors, weak arguments, unsupported claims -- `alternative` — Different approaches or perspectives to consider +| Endpoint | Method | Purpose | +|----------|--------|---------| +| `/api/personas` | GET | List all 14 built-in personas | +| `/api/documents` | POST/GET | Save new document (encrypted) / List user's documents | +| `/api/documents/:id` | GET/PUT/PATCH/DELETE | Load, update, rename, delete document | +| `/api/folders` | POST/GET | Create folder / List user's folders | +| `/api/preferences` | GET/PUT | User preferences (auto-dictate) | +| `/api/tracking/event` | POST | Record usage tracking event | +| `/api/metrics` | POST | Record productivity metrics | -### Tone Options -Voice for the final document: -- `inspirational`, `practical`, `analytical`, `persuasive`, `cautious` +### Admin Endpoints -## Key Components +| Endpoint | Method | Purpose | +|----------|--------|---------| +| `/api/auth/role` | GET | Check if current user is admin | +| `/api/admin/dashboard` | GET | Analytics dashboard data | +| `/api/admin/user-metrics` | GET | User metrics matrix | +| `/api/admin/sync-app-docs` | POST | Sync per-app CLAUDE.md files to document store | +| `/api/admin/persona-overrides` | GET | List persona DB overrides | +| `/api/admin/agent-prompts` | GET | List LLM task prompt overrides | -### Workspace.tsx (Orchestrator) -Central hub managing: -- **Phases**: `input` → `blank-document` → `workspace` -- **State**: document, lenses, provocations, outline, versions -- **Versioning**: Full history with diff comparison +## Domain Concepts -### ReadingPane.tsx -The document canvas: -- Editable mode (pencil toggle) -- Text selection → voice/edit actions -- Download functionality +### Persona Hierarchy (14 Built-in Perspectives) -### ProvocationsDisplay.tsx -Challenge cards that drive iteration: -- Voice recording on each card -- Status: pending, addressed, rejected, highlighted -- Context passed to merge for intelligent integration +``` +Master Researcher (root) — orchestrates all domains, refreshes every 7 days +│ +├── Business Domain +│ ├── Think Bigger ─ Scale impact: retention, reach, accessibility +│ ├── CEO ─ Mission-first: clarity, accountability, trust +│ └── Product Manager ─ Business value: user stories, success metrics +│ +├── Technology Domain +│ ├── Architect ─ System design: boundaries, APIs, data flow +│ ├── Data Architect ─ Fit-for-purpose data, Key Ring identifiers +│ ├── QA Engineer ─ Testing: edge cases, error handling, reliability +│ ├── UX Designer ─ User flows: discoverability, accessibility +│ ├── Tech Writer ─ Documentation: clarity, naming, context +│ ├── Security Engineer ─ Auth, data privacy, compliance +│ └── Cybersecurity ─ Threat modeling, attack surface, incident response +│ +└── Marketing Domain + ├── Growth Strategist ─ Acquisition, activation, retention, funnel economics + ├── Brand Strategist ─ Positioning, differentiation, voice consistency + └── Content Strategist ─ Audience-channel fit, distribution, SEO, measurement +``` -### VoiceRecorder.tsx -Natural input via speech: -- Web Speech API transcription -- Auto-merge into document -- Works on provocations and text selections +Each persona has distinct challenge and advice prompts: + +| ID | Domain | Label | Focus | +|----|--------|-------|-------| +| `thinking_bigger` | business | Think Bigger | Scale impact: retention, cost-to-serve, accessibility | +| `ceo` | business | CEO | Mission-first: clarity, accountability, trust | +| `product_manager` | business | Product Manager | Business value: user stories, success metrics | +| `architect` | technology | Architect | System design: boundaries, APIs, data flow | +| `data_architect` | technology | Data Architect | Fit-for-purpose data, Key Ring, governance | +| `quality_engineer` | technology | QA Engineer | Testing: edge cases, error handling, reliability | +| `ux_designer` | technology | UX Designer | User flows: discoverability, accessibility, error states | +| `tech_writer` | technology | Tech Writer | Documentation: clarity, naming, missing context | +| `security_engineer` | technology | Security Engineer | Auth, data privacy, compliance | +| `cybersecurity_engineer` | technology | Cybersecurity | Threat modeling, attack surface, defense-in-depth | +| `growth_strategist` | marketing | Growth Strategist | Acquisition, activation, retention, funnel economics | +| `brand_strategist` | marketing | Brand Strategist | Positioning, differentiation, voice consistency | +| `content_strategist` | marketing | Content Strategist | Audience-channel fit, distribution, measurement | + +### Hierarchy Governance Rules + +The Master Researcher is the root agent that governs the entire persona hierarchy: + +1. **Freshness**: The Master Researcher must refresh all persona definitions every **7 days**. Personas with `lastResearchedAt` older than 7 days (or `null`) are considered stale and flagged for re-research. +2. **Domain completeness**: Every domain (business, technology, marketing) must have sufficient coverage for its core knowledge worker roles. The Master Researcher evaluates gaps and proposes new personas when a domain is under-represented. +3. **No orphans**: Every persona (except the root) must have a valid `parentId` and `domain`. Personas without hierarchy placement are invalid. +4. **Challenge ≠ Advice**: Each persona has separate `challenge` and `advice` prompts. Challenges identify gaps and weaknesses without offering solutions. Advice is a separate invocation that provides concrete, actionable recommendations. These are never combined. +5. **Non-negotiable behaviors**: Each persona defines explicit non-negotiable behaviors (what it always does) and forbidden behaviors (what it never does). These constraints ensure consistency across research refreshes. +6. **Computer-first filter**: Only knowledge worker roles where computer-based tasks are central qualify as personas. Roles with significant non-digital, in-person work are excluded. +7. **No overlap**: Each persona must challenge a distinct dimension. If two personas overlap significantly, they should be merged or one should be removed. +8. **Structured definition**: Every persona must include: id, label, icon, role, description, color, prompts (challenge + advice), summary, domain, parentId, and lastResearchedAt. + +### Instruction Types (7 Classifications) + +The `/api/write` endpoint classifies instructions before processing: +- `expand` — Add depth, examples, supporting details +- `condense` — Remove redundancy, tighten prose +- `restructure` — Reorganize content, modify headings, reorder sections +- `clarify` — Simplify language, improve accessibility +- `style` — Adjust voice and tone +- `correct` — Fix grammar, spelling, logic errors +- `general` — Fallback for mixed instructions -## Design Principles +### Tone Options +- `inspirational`, `practical`, `analytical`, `persuasive`, `cautious` -1. **Document-Centric** — The document is the product, not a chat log -2. **Productive Resistance** — AI challenges, doesn't just assist -3. **User as Author** — You write; AI provokes better writing -4. **Voice-First Feedback** — Speaking is faster than typing for ideation -5. **Iterative Shaping** — Documents evolve through multiple passes +## Key Shared Components + +### NotebookWorkspace.tsx (Primary Orchestrator) +The main interface since the notebook refactor. Unified 3-panel resizable layout: +- **Left panel**: ContextSidebar (document/folder tree, pin, search) + ChatDrawer (user-to-user messaging) +- **Center panel**: SplitDocumentEditor (multi-tab document + chart editing) + objective input +- **Right panel**: Research (streaming AI chat) | Notes (captured context + voice transcripts) | Provo (persona discussion) | Generate (document generation) +- **State**: document, objective, personas, capturedContext, pinnedDocIds, discussionMessages, versions, editHistory +- **Mobile**: Falls back to tabbed single-panel layout (context | document | chat tabs) + +### notebook/ Components +| Component | Purpose | +|-----------|---------| +| `NotebookTopBar.tsx` | Session header: name, version count, New Session, admin controls | +| `NotebookLeftPanel.tsx` | Collapsible left panel with Context / Chat / Video tabs | +| `ContextSidebar.tsx` | Full document/folder tree browser with pin/unpin, search, inline rename/delete | +| `NotebookCenterPanel.tsx` | Document editor + objective, preview overlay for context docs | +| `NotebookRightPanel.tsx` | 4-tab right panel: Research, Notes, Provo, Generate | +| `NotebookResearchChat.tsx` | Streaming research chat via `/api/chat/stream` (SSE) | +| `ProvoThread.tsx` | Multi-persona discussion thread: challenges, accept/dismiss, respond | +| `TranscriptPanel.tsx` | Notes management: add notes (text/voice), save to Context Store, evolve document | +| `SplitDocumentEditor.tsx` | Multi-tab editor with smart buttons (Expand/Condense/Restructure/Clarify/Style/Correct) | +| `PersonaAvatarRow.tsx` | Persona selector row with toggle avatars | + +### bschart/ Components +Visual diagram/flowchart designer on infinite canvas: +- `BSChartWorkspace.tsx` — Main container with state management and voice command integration +- `hooks/useVoiceChartCommands.ts` — Natural language voice commands → chart operations +- `hooks/useChartState.ts` — Node/connector state management +- Supports: ERD, flowcharts, architecture diagrams with drag/drop and voice creation + +### ProvokeText.tsx (ADR-mandated) +Universal text component — see ADR below. All text display/editing must use this. + +App-specific components are documented in each app's `apps//CLAUDE.md`. ## Design System ### Theme - **Primary**: Warm amber (#B35C1E) -- **Accent**: Thoughtful blue (200°, 60%, 45%) +- **Accent**: Thoughtful blue (200, 60%, 45%) - **Aesthetic**: Aged paper/ink, intellectual warmth - **Mode**: Dark mode supported @@ -203,31 +406,311 @@ Natural input via speech: | Variable | Description | |----------|-------------| -| `AI_INTEGRATIONS_OPENAI_API_KEY` | OpenAI API key | -| `AI_INTEGRATIONS_OPENAI_BASE_URL` | OpenAI API base URL | +| `AI_INTEGRATIONS_OPENAI_API_KEY` | OpenAI API key (auto-injected by Replit AI Integrations) | +| `AI_INTEGRATIONS_OPENAI_BASE_URL` | Replit proxy base URL for OpenAI requests (auto-injected) | +| `ANTHROPIC_API_KEY` | Anthropic API key (or `ANTHROPIC_KEY`) | +| `GEMINI_API_KEY` | Google Gemini API key (via OpenAI-compatible endpoint) | +| `LLM_PROVIDER` | Force provider: `openai`, `gemini`, or `anthropic` (auto-detects by default) | | `DATABASE_URL` | PostgreSQL connection string | +| `ENCRYPTION_SECRET` | AES-GCM key for document encryption | +| `CLERK_PUBLISHABLE_KEY` | Clerk frontend authentication | +| `CLERK_SECRET_KEY` | Clerk backend secret key | +| `PLAYWRIGHT_CHROMIUM_PATH` | Path to Chromium for screenshots | + +### Replit AI Integrations + +OpenAI credentials are managed automatically via **Tools > AI Integrations** in Replit. When you enable OpenAI in that panel, Replit injects `AI_INTEGRATIONS_OPENAI_API_KEY` and `AI_INTEGRATIONS_OPENAI_BASE_URL` into the server process environment. Requests are proxied through Replit and billed to your Replit credits. + +The LLM adapter in `server/llm.ts` auto-detects these variables. You can verify the active provider at runtime via `GET /api/llm-status`. ## Development Notes +### Code Quality — Fix Pre-existing Errors + +When running `npm run check` or `npm run build`, **always fix all TypeScript errors** — including pre-existing ones in files you didn't change. Do not ignore or skip errors just because they existed before your changes. Every check run should leave the codebase in a better state. If a pre-existing error would take significant effort to fix, flag it to the user rather than silently ignoring it. + ### Adding API Routes 1. Define Zod schema in `shared/schema.ts` 2. Add endpoint in `server/routes.ts` 3. Use `safeParse()` for validation +4. All LLM calls go through `llm.generate()` / `llm.stream()` from `server/llm.ts` + +### Routing (`App.tsx`) + +| Route | Component | Purpose | +|-------|-----------|---------| +| `/` | `NotebookWorkspace` | Default home — notebook 3-panel layout | +| `/app/:templateId` | `NotebookWorkspace` | Template-specific workspace | +| `/store` | `ContextStore` | Standalone Context Store page | +| `/admin` | `Admin` | Admin analytics dashboard | +| `/pricing` | `Pricing` | Pricing page | ### State Management -- Local state in Workspace.tsx for app-wide concerns +- Local state in NotebookWorkspace.tsx for app-wide concerns - React Query for server state caching +- Context pinning: `pinnedDocIds` (Set) + `pinnedDocContents` (cache) in NotebookWorkspace +- Captured context: `capturedContext` (ContextItem[]) passed to all LLM calls - No Redux/Zustand — keep it simple +### Database Tables (`shared/models/chat.ts`) + +| Table | Purpose | +|-------|---------| +| `connections` | User-to-user connections (pending/accepted/blocked) | +| `conversations` | Chat conversations between users | +| `messages` | Individual chat messages | +| `chatPreferences` | User chat settings | + +### ADR: Dual Schema Management — ensureTables + Drizzle (CRITICAL) + +The database schema is managed by **two independent systems** that MUST stay in perfect sync. Mismatches between them cause recurring migration items on every Replit deploy. This has been a repeated source of bugs. + +**The two systems:** + +| System | File | Runs when | Purpose | +|--------|------|-----------|---------| +| `ensureTables()` | `server/db.ts` | App startup (`npm run dev` / `npm run start`) | Safety net: creates tables/columns/indexes via raw SQL so the app works even if `drizzle-kit push` was never run | +| Drizzle schema | `shared/models/chat.ts` | `npm run db:push` (and Replit auto-runs it during Deploy) | Authoritative schema: `drizzle-kit push` introspects the database and generates DDL to match this schema | + +**Why this causes problems:** On Replit, deployment runs `drizzle-kit push` BEFORE the app starts. So `ensureTables()` hasn't run yet when Drizzle checks the database. If the database is missing structures that only `ensureTables()` would create, Drizzle generates migration items. Worse: if `ensureTables()` creates structures that differ subtly from the Drizzle schema (wrong index direction, wrong constraint name, extra FK constraints), Drizzle detects a mismatch and generates DROP/CREATE statements every single deploy. + +**Mandatory rules when changing database schema:** + +1. **Always edit BOTH files.** Every schema change must be reflected in both `shared/models/chat.ts` (Drizzle) AND `server/db.ts` (`ensureTables`). Never edit one without the other. + +2. **The Drizzle schema is the source of truth.** Write the Drizzle definition first, then make `ensureTables()` produce the exact same DDL. Never the reverse. + +3. **Match index definitions exactly:** + - If Drizzle says `index("idx_foo").on(table.bar)` → ensureTables must say `CREATE INDEX IF NOT EXISTS idx_foo ON tablename(bar)` with no extra modifiers (no `DESC`, no `NULLS FIRST`, etc.) + - Drizzle's default sort is ASC. Only add `DESC` to ensureTables if the Drizzle schema explicitly uses `.desc()` + +4. **Match constraint names exactly:** + - Drizzle `.unique()` on a column generates a constraint named `{table}_{column}_unique` + - PostgreSQL inline `UNIQUE` in CREATE TABLE generates `{table}_{column}_key` — a DIFFERENT name + - ensureTables must explicitly create constraints with Drizzle-compatible names: `ALTER TABLE ... ADD CONSTRAINT {table}_{column}_unique UNIQUE(...)` wrapped in `DO $$ BEGIN ... EXCEPTION WHEN duplicate_object THEN NULL; END $$;` + +5. **No FK REFERENCES in ensureTables** unless the Drizzle schema also defines them (via `.references()`). Currently the Drizzle schema has ZERO foreign keys. Adding `REFERENCES` in ensureTables creates constraints that Drizzle doesn't expect, causing recurring DROP CONSTRAINT statements. + +6. **New table checklist:** + - Add `pgTable()` definition in `shared/models/chat.ts` with all columns, indexes, and unique constraints + - Add `CREATE TABLE IF NOT EXISTS` in `ensureTables()` matching the Drizzle definition exactly + - Add the table name to `tablesFilter` in `drizzle.config.ts` + - Add any Drizzle-named unique constraints (`{table}_{column}_unique`) via `ALTER TABLE ADD CONSTRAINT` + +7. **New column checklist:** + - Add the column to the `pgTable()` definition in `shared/models/chat.ts` + - Add `ALTER TABLE ... ADD COLUMN IF NOT EXISTS` in the ensureTables `DO $$ BEGIN ... END $$` block + - If the column has `.unique()`, also add the named constraint + +8. **Verify with `npm run db:push --dry-run`** (or check the Replit deploy dialog) — there should be ZERO pending migration items if both systems are in sync and the app has been started at least once. + ### Error Handling - Zod validation on all API inputs - Defensive null-checks in components - Toast notifications for user feedback +### ADR: Always Use ProvokeText for Text Display + +**All text panels in the application MUST use `ProvokeText`** — never raw `
`, `

`, `