Upgrade ai-pdf-chatbot to NotebookLM-style learning workspace

ai-pdf-chatbot/.env.example

@@ -15,3 +15,11 @@ NEXT_PUBLIC_BETTER_AUTH_URL=http://localhost:3000
 # Same secret InsForge uses internally; the bridge route signs HS256
 # with this. Get it via `npx @insforge/cli secrets get JWT_SECRET`.
 INSFORGE_JWT_SECRET=replace-with-output-of-cli-secrets-get-JWT_SECRET
+
+# ─── Audio Overview (optional) ────────────────────────────────────────
+# OpenAI key for TTS-driven workspace audio overviews (NotebookLM-style
+# two-host podcast). Leave blank to disable — the workspace audio tab
+# will surface a friendly "configure to enable" prompt instead of
+# crashing. Embeddings + chat still route through InsForge AI without
+# this key.
+OPENAI_API_KEY=

ai-pdf-chatbot/README.md

@@ -1,9 +1,9 @@
 <a href="https://insforge.dev">
-  <h1 align="center">InsForge AI PDF Chatbot</h1>
+  <h1 align="center">InsForge AI Notebook</h1>
 </a>
 
 <p align="center">
-  Upload PDFs and chat with them. RAG with citations powered by InsForge pgvector + InsForge AI.
+  Open-source NotebookLM for students. Workspaces of PDFs come with auto-generated mindmaps, spaced-repetition flashcards, two-host podcast summaries, and RAG chat that highlights cited passages inside the source PDF.
 </p>
 
 <p align="center">
@@ -17,18 +17,44 @@
 
 ## Features
 
+NotebookLM-style learning features:
+
+- **Workspaces.** Group related PDFs, chats, mindmaps, and flashcards into one notebook-style container. Cross-document RAG inside a workspace.
+- **Inline PDF viewer.** Click a `[n]` citation, the source PDF slides in from the right at the cited page with the passage highlighted.
+- **Mindmap.** Workspace-wide concept tree, generated from PDF summaries and rendered as an interactive Markmap.
+- **Spaced-repetition flashcards.** Generated from the PDF content, dropped into a per-workspace SRS queue (SM-2 lite, three review grades).
+- **Audio Overview.** Two-host podcast summary, opt-in via `OPENAI_API_KEY`. Prompt adapted from [open-notebooklm](https://github.com/gabrielchua/open-notebooklm).
+
+Production-grade plumbing under the hood:
+
 - Next.js 16 App Router
 - PDF upload (≤ 10 MB) with server-side extraction via `pdfjs-dist`
 - Vector search on InsForge pgvector (`vector(1536)` + ivfflat cosine)
-- Streaming chat with bracketed `[n]` source citations
-- **Better Auth** for email + password sign-in — user/session tables live in your InsForge Postgres
-- HS256 bridge JWT from BA → InsForge: RLS reads `requesting_user_id()` so every user only sees their own documents and chats
+- Streaming RAG chat with bracketed `[n]` source citations (NDJSON)
+- **Better Auth** for email + password sign-in. User/session tables live in your InsForge Postgres.
+- HS256 bridge JWT from BA → InsForge: RLS reads `requesting_user_id()` so every user only sees their own data.
 - shadcn/ui + Tailwind 4 design tokens
 
 ## Demo
 
 [aipdfchat.insforge.site](https://aipdfchat.insforge.site) — sign up with any email, upload a PDF, ask away.
 
+## Why this template
+
+| | NotebookLM | ChatPDF | LangChain RAG demo | **This template** |
+|---|---|---|---|---|
+| Workspaces + cross-PDF RAG | yes | no | no | yes |
+| Inline PDF passage highlight | partial | no | no | yes |
+| Mindmap from PDFs | yes | no | no | yes |
+| Spaced-repetition flashcards | no | partial | no | yes |
+| Two-host audio podcast | yes | no | no | yes (opt-in) |
+| Self-hosted, your data | no | no | yes | yes |
+| Open source, swap any model | no | no | yes | yes |
+| Multi-tenant out of the box | no | yes | no | yes (Better Auth + RLS) |
+| Ready to ship a real product | no | yes | no | yes |
+
+If you want to fork a NotebookLM-style study tool and run it on infrastructure you control, this is the closest open-source starting point.
+
 ## Quick Launch
 
 Fastest path:
@@ -93,6 +119,7 @@ npm install
 - **pgvector extension** — the migration runs `create extension if not exists vector;` against the `public` schema. Supported on any standard InsForge project.
 - **Node 18+** for `node:test` and the ESM `pdfjs-dist` build.
 - **InsForge SMTP** configured if you want password resets to send mail. Cloud projects: configured automatically. Self-hosted: `PUT /api/auth/smtp-config`.
+- **OpenAI API key (optional)** — required only for the Audio Overview tab. Set `OPENAI_API_KEY` in `.env.local`; without it the workspace audio tab shows a friendly "configure to enable" prompt and everything else still works.
 
 ## Architecture
 
@@ -128,6 +155,25 @@ Chat ──► /api/chat (NDJSON stream) ──► embed(question) ──► mat
                   persist assistant message with citations payload
 ```
 
+## Audio Overview
+
+The Audio tab generates a NotebookLM-style two-host podcast summary of every PDF in a workspace.
+
+**How it works:**
+
+1. `lib/ai/audio-script.ts` calls InsForge AI (chat completion) with a producer-style prompt adapted from [open-notebooklm](https://github.com/gabrielchua/open-notebooklm). Sarah hosts and interviews Mike, a subject-matter expert. The asymmetric framing prevents the "two co-hosts agreeing with each other" failure mode.
+2. `lib/audio/tts.ts` synthesizes each turn through OpenAI TTS (`gpt-4o-mini-tts`, `nova` voice for Sarah, `onyx` for Mike), 4 turns in parallel.
+3. mp3 frames are concatenated naively (good enough for a study tool; swap in `ffmpeg.wasm` if you need gapless playback).
+4. The final mp3 lands in the `audio-overviews` storage bucket (public read), and `workspaces.audio_url` + `audio_script` are cached so the tab renders instantly on revisit.
+
+**Cost:** about $0.006 per regeneration (700 chars TTS at `tts-1` pricing). The chat completion that drafts the script runs on InsForge AI and counts against your existing AI quota.
+
+**Without `OPENAI_API_KEY`:** the Audio tab shows a friendly "configure to enable" prompt. Every other feature works.
+
+**Want a different TTS?** Edit `lib/audio/tts.ts`. ElevenLabs and Cartesia have OpenAI-compatible HTTP endpoints. The voice-per-speaker mapping (`VOICE_BY_SPEAKER`) is the only thing to swap.
+
+**Want a better script?** Set `UTILITY_MODEL` in `lib/ai/constants.ts` from `gpt-4o-mini` to `gpt-4o`. Cost goes from $0.006 to ~$0.04 per generation, but you start getting analogies, anecdotes, and "aha moments" the mini model can't produce.
+
 ## Customizing
 
 - **Switch embedding model:** edit `lib/ai/constants.ts` (`EMBEDDING_MODEL`, `EMBEDDING_DIMENSIONS`). If the dimension changes, also `ALTER TABLE document_chunks ALTER COLUMN embedding TYPE vector(N)` and re-ingest existing documents.

ai-pdf-chatbot/app/api/chat/route.ts

@@ -13,6 +13,9 @@ type Body = {
   chatId?: string;
   input: string;
   documentIds?: string[];
+  // If set when chatId is absent, the new chat is created under that
+  // workspace and RAG retrieval is scoped to that workspace's documents.
+  workspaceId?: string | null;
 };
 
 function extractDeltaText(content: unknown): string {
@@ -44,20 +47,32 @@ export async function POST(req: Request) {
   const client = createInsforgeServerClient({ accessToken: auth.accessToken });
 
   let chatId = body.chatId;
+  let workspaceId: string | null = body.workspaceId ?? null;
   if (!chatId) {
     const ins = await client.database
       .from('chat_sessions')
       .insert({
         user_id: ownerId,
+        workspace_id: workspaceId,
         title: body.input.trim().slice(0, 60) || 'New chat',
         document_ids: body.documentIds ?? [],
       })
-      .select('id')
+      .select('id, workspace_id')
       .single();
     if (ins.error || !ins.data) {
       return NextResponse.json({ error: ins.error?.message ?? 'Failed to create chat' }, { status: 500 });
     }
-    chatId = (ins.data as { id: string }).id;
+    chatId = (ins.data as { id: string; workspace_id: string | null }).id;
+    workspaceId = (ins.data as { id: string; workspace_id: string | null }).workspace_id;
+  } else {
+    // For existing chats, read the workspace_id off the row so a stale
+    // client-side cache can't widen retrieval scope.
+    const lookup = await client.database
+      .from('chat_sessions')
+      .select('workspace_id')
+      .eq('id', chatId)
+      .single();
+    workspaceId = (lookup.data as { workspace_id: string | null } | null)?.workspace_id ?? null;
   }
 
   const lastMsg = await client.database
@@ -89,7 +104,7 @@ export async function POST(req: Request) {
     async start(controller) {
       const send = (obj: unknown) => controller.enqueue(encodeNdjson(obj));
       try {
-        const chunks = await retrieveForQuestion(client, ownerId, inputText, docIds);
+        const chunks = await retrieveForQuestion(client, ownerId, inputText, docIds, workspaceId);
         const citations = toCitations(chunks);
         send({ type: 'chat', chatId: resolvedChatId });
         send({ type: 'citations', data: citations });

ai-pdf-chatbot/app/api/chats/route.ts

@@ -5,17 +5,30 @@ import { getCurrentAuthState } from '@/lib/auth-state';
 export const runtime = 'nodejs';
 export const dynamic = 'force-dynamic';
 
-export async function GET() {
+export async function GET(req: Request) {
   const auth = await getCurrentAuthState();
   if (!auth.viewer.isAuthenticated || !auth.accessToken) {
     return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
   }
+
+  // Same workspace filter shape as the documents list: ?workspace=<id> or
+  // ?workspace=unsorted. No param returns every chat the user owns.
+  const url = new URL(req.url);
+  const workspace = url.searchParams.get('workspace');
+
   const client = createInsforgeServerClient({ accessToken: auth.accessToken });
-  const { data, error } = await client.database
+  let query = client.database
     .from('chat_sessions')
-    .select('id, title, document_ids, created_at, last_message_at')
+    .select('id, workspace_id, title, document_ids, created_at, last_message_at')
     .order('last_message_at', { ascending: false });
 
+  if (workspace === 'unsorted') {
+    query = query.is('workspace_id', null);
+  } else if (workspace) {
+    query = query.eq('workspace_id', workspace);
+  }
+
+  const { data, error } = await query;
   if (error) return NextResponse.json({ error: error.message }, { status: 500 });
   return NextResponse.json({ chats: data ?? [] });
 }
@@ -26,13 +39,18 @@ export async function POST(req: Request) {
     return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
   }
 
-  const body = (await req.json().catch(() => ({}))) as { title?: string; documentIds?: string[] };
+  const body = (await req.json().catch(() => ({}))) as {
+    title?: string;
+    documentIds?: string[];
+    workspaceId?: string | null;
+  };
   const client = createInsforgeServerClient({ accessToken: auth.accessToken });
 
   const { data, error } = await client.database
     .from('chat_sessions')
     .insert({
       user_id: auth.viewer.id,
+      workspace_id: body.workspaceId ?? null,
       title: body.title?.trim() || 'New chat',
       document_ids: body.documentIds ?? [],
     })

ai-pdf-chatbot/app/api/documents/[id]/flashcards/route.ts

@@ -41,7 +41,7 @@ export async function POST(_req: Request, { params }: { params: Promise<{ id: st
 
   const docRes = await client.database
     .from('documents')
-    .select('id, storage_bucket, storage_key, file_name, status')
+    .select('id, workspace_id, storage_bucket, storage_key, file_name, status')
     .eq('id', id)
     .single();
 
@@ -50,6 +50,7 @@ export async function POST(_req: Request, { params }: { params: Promise<{ id: st
   }
   const doc = docRes.data as {
     id: string;
+    workspace_id: string | null;
     storage_bucket: string;
     storage_key: string;
     file_name: string;
@@ -80,10 +81,14 @@ export async function POST(_req: Request, { params }: { params: Promise<{ id: st
 
   const rows = cards.map((c, i) => ({
     document_id: id,
+    workspace_id: doc.workspace_id,
     user_id: auth.viewer.id,
     question: c.question,
     answer: c.answer,
     sort_order: i,
+    // Fresh cards land in the SRS queue immediately; defaults on the
+    // ease/interval/reps columns take care of the rest.
+    due_at: new Date().toISOString(),
   }));
 
   const ins = await client.database

ai-pdf-chatbot/app/api/documents/[id]/route.ts

@@ -5,6 +5,43 @@ import { getCurrentAuthState } from '@/lib/auth-state';
 export const runtime = 'nodejs';
 export const dynamic = 'force-dynamic';
 
+export async function PATCH(req: Request, { params }: { params: Promise<{ id: string }> }) {
+  const { id } = await params;
+  const auth = await getCurrentAuthState();
+  if (!auth.viewer.isAuthenticated || !auth.accessToken) {
+    return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
+  }
+
+  const body = (await req.json().catch(() => ({}))) as { workspace_id?: string | null };
+  // Only workspace_id is mutable via this route today. Adding other fields
+  // here keeps the surface flat instead of one route per column.
+  if (!('workspace_id' in body)) {
+    return NextResponse.json({ error: 'No mutable fields provided' }, { status: 400 });
+  }
+
+  const client = createInsforgeServerClient({ accessToken: auth.accessToken });
+  const { error } = await client.database
+    .from('documents')
+    .update({ workspace_id: body.workspace_id ?? null, updated_at: new Date().toISOString() })
+    .eq('id', id);
+
+  if (error) return NextResponse.json({ error: error.message }, { status: 500 });
+
+  // Cascade the workspace_id to all flashcards generated from this doc so
+  // workspace-wide review queues see them. Documents that don't have
+  // flashcards yet just no-op. Failures here would silently leave the
+  // doc and its flashcards out of sync, so surface them.
+  const cascade = await client.database
+    .from('document_flashcards')
+    .update({ workspace_id: body.workspace_id ?? null })
+    .eq('document_id', id);
+  if (cascade.error) {
+    return NextResponse.json({ error: cascade.error.message }, { status: 500 });
+  }
+
+  return NextResponse.json({ ok: true });
+}
+
 export async function DELETE(_req: Request, { params }: { params: Promise<{ id: string }> }) {
   const { id } = await params;
   const auth = await getCurrentAuthState();

ai-pdf-chatbot/app/api/documents/route.ts

@@ -5,18 +5,31 @@ import { getCurrentAuthState } from '@/lib/auth-state';
 export const runtime = 'nodejs';
 export const dynamic = 'force-dynamic';
 
-export async function GET() {
+export async function GET(req: Request) {
   const auth = await getCurrentAuthState();
   if (!auth.viewer.isAuthenticated || !auth.accessToken) {
     return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
   }
 
+  // ?workspace=<id> scopes the list to one workspace.
+  // ?workspace=unsorted returns documents with workspace_id IS NULL.
+  // No param: returns all of the user's documents.
+  const url = new URL(req.url);
+  const workspace = url.searchParams.get('workspace');
+
   const client = createInsforgeServerClient({ accessToken: auth.accessToken });
-  const { data, error } = await client.database
+  let query = client.database
     .from('documents')
-    .select('id, file_name, file_size, mime_type, status, error, page_count, summary, suggested_questions, created_at')
+    .select('id, workspace_id, file_name, file_size, mime_type, status, error, page_count, summary, suggested_questions, created_at')
     .order('created_at', { ascending: false });
 
+  if (workspace === 'unsorted') {
+    query = query.is('workspace_id', null);
+  } else if (workspace) {
+    query = query.eq('workspace_id', workspace);
+  }
+
+  const { data, error } = await query;
   if (error) return NextResponse.json({ error: error.message }, { status: 500 });
   return NextResponse.json({ documents: data ?? [] });
 }

ai-pdf-chatbot/app/api/documents/upload/route.ts

@@ -17,6 +17,18 @@ export async function POST(req: Request) {
 
   const formData = await req.formData();
   const file = formData.get('file');
+  const rawWorkspaceId = formData.get('workspaceId');
+  // Validate the client-supplied workspace id format up-front so a bad
+  // value returns a 400 instead of leaking a 500 from a downstream
+  // `invalid input syntax for type uuid` Postgres error.
+  const UUID_RE = /^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$/;
+  let workspaceId: string | null = null;
+  if (typeof rawWorkspaceId === 'string' && rawWorkspaceId) {
+    if (!UUID_RE.test(rawWorkspaceId)) {
+      return NextResponse.json({ error: 'workspaceId must be a UUID' }, { status: 400 });
+    }
+    workspaceId = rawWorkspaceId;
+  }
   if (!(file instanceof File)) {
     return NextResponse.json({ error: 'No file provided' }, { status: 400 });
   }
@@ -33,6 +45,7 @@ export async function POST(req: Request) {
     .from('documents')
     .insert({
       user_id: auth.viewer.id,
+      workspace_id: workspaceId,
       file_name: file.name,
       file_size: file.size,
       mime_type: file.type,