Skip to content

Upgrade ai-pdf-chatbot to NotebookLM-style learning workspace#50

Merged
CarmenDou merged 8 commits into
mainfrom
feat/ai-pdf-chatbot-v2
Jun 3, 2026
Merged

Upgrade ai-pdf-chatbot to NotebookLM-style learning workspace#50
CarmenDou merged 8 commits into
mainfrom
feat/ai-pdf-chatbot-v2

Conversation

@CarmenDou
Copy link
Copy Markdown
Contributor

@CarmenDou CarmenDou commented Jun 3, 2026

Summary

Upgrades the ai-pdf-chatbot template from a single-PDF RAG chatbot into an open-source NotebookLM-style study workspace. Five new top-level capabilities land on top of the existing chat + pgvector pipeline:

  • Workspaces — notebook-style containers grouping PDFs, chats, mindmap, flashcards, and audio
  • Inline PDF viewer + highlight — citation clicks open react-pdf in a drawer at the cited page with the passage highlighted via text-layer prefix match
  • Mindmap (Markmap) — LLM-generated workspace concept tree rendered with d3
  • Spaced-repetition flashcards — SM-2 lite scheduler with three review grades, per-workspace due queue
  • Audio Overview — opt-in two-host podcast summary via OpenAI TTS, prompt adapted from open-notebooklm

Also includes:

  • Migration fixes uncovered during fresh-link smoke test: SET maintenance_work_mem = '128MB' (ivfflat index creation OOMs on the 16MB default) and a trailing NOTIFY pgrst, 'reload schema' (PostgREST schema cache doesn't auto-refresh after db import).
  • Sidebar architecture: ChatShell hoisted into app/chat/layout.tsx so the sidebar list survives navigation between /chat and /chat/[id] instead of unmounting and flashing.
  • Share button visual cleanup: kept as outline regardless of state, with a small emerald dot indicating active sharing (Notion / Linear pattern, no more black/white flip).
  • Marketplace positioning: renamed from AI PDF Chatbot to AI Notebook to reflect the NotebookLM-style upgrade and form a product family with the existing Workspace template. README rewritten to lead with the new capabilities; slug, demo URL, and directory name unchanged.

Marketplace fields after this PR

  • name: AI Notebook (was AI PDF Chatbot)
  • description: positions the template as "open-source NotebookLM for students" and enumerates the capabilities (workspaces / mindmaps / flashcards / audio / inline highlights)
  • features: [RAG, react-pdf, OpenAI TTS, Better Auth] — kept as a tech-stack list to match the convention of other templates (admin-dashboard: [TanStack Table, Recharts, Real-time Chat, Auth], workspace: [BlockNote, Nested Pages, Public Share Links], etc.). The marketplace UI shows these as the card-level tech badges, while the user-facing capabilities (Workspaces, Mindmap, Flashcards, Audio Overview) are in the description, which is also rendered on the card.
  • tags: [pdf, study-tool, notebooklm, rag, flashcards, pgvector] for search

Verification

Backend was end-to-end verified against an InsForge V2 backend branch (curl walkthrough of register, workspace CRUD, PDF upload, RAG chat, mindmap, flashcard generation + grading, audio generation). The Audio Overview script was generated and listened to.

Notes on cubic review feedback

Most P1/P2 findings were addressed in commit 5466fd5: API routes now surface DB errors instead of masking them as "no documents" / 0 counts / silent successes, frontend loaders no longer hang on failed fetches, PDF viewer clears stale highlights and clamps page navigation, and lib/ai/audio-script.ts no longer logs raw model output (which could carry PDF-derived user content).

One intentional choice was preserved: the sidebar chat list is not re-fetched after each assistant turn in an existing chat. Re-sorting on every message reintroduced the visual jitter that hoisting ChatShell into the layout was specifically meant to fix. The trade-off (mild list staleness, recoverable on next mount) is documented inline at lib/stream/use-chat-stream.ts.

Test plan

  • npm install && npm run setup && npm run dev
  • Register a new account, create a workspace, upload a PDF, run a chat
  • Click a [n] citation in the assistant reply — PDF drawer should slide in at the right page with the cited passage highlighted yellow
  • Open a different citation with no snippet — previous highlight should clear (cubic fix)
  • Mindmap tab → "Generate" → SVG tree should render and pan/zoom
  • Flashcards tab → generate from a document → "Start reviewing" → grade with again/hard/good/easy; verify cards re-queue per SRS schedule
  • When 0 cards are due, the "Start reviewing" button should be disabled and not navigate (cubic fix)
  • (Optional) Add OPENAI_API_KEY to .env.local, restart, hit Audio tab → "Generate" → audio should play, transcript should render with Sarah/Mike per-turn cards
  • Trigger a network failure (devtools offline) on workspace list / review queue → UI should toast an error, not hang on loader (cubic fix)
  • Switch between two chats in the sidebar — the list should not flash on navigation
  • Confirm the share button stays outline-styled and shows the emerald dot when a share link is active

🤖 Generated with Claude Code

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

20 issues found across 48 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="ai-pdf-chatbot/components/audio-overview.tsx">

<violation number="1" location="ai-pdf-chatbot/components/audio-overview.tsx:38">
P2: Network/request failures are unhandled in audio generation, so users get no error toast when `fetch` throws.</violation>
</file>

<file name="ai-pdf-chatbot/lib/ai/audio-script.ts">

<violation number="1" location="ai-pdf-chatbot/lib/ai/audio-script.ts:126">
P1: Parse-error logging includes raw model output text, which can leak user document-derived content to server logs.</violation>
</file>

<file name="ai-pdf-chatbot/lib/pdf/pdf-worker.ts">

<violation number="1" location="ai-pdf-chatbot/lib/pdf/pdf-worker.ts:13">
P3: Worker configuration is deferred to a useEffect and lives in a separate module from the react-pdf component usage, contradicting the documented react-pdf requirement.</violation>
</file>

<file name="ai-pdf-chatbot/app/workspaces/[id]/review/page.tsx">

<violation number="1" location="ai-pdf-chatbot/app/workspaces/[id]/review/page.tsx:38">
P2: Failed/errored queue loads leave `cards` as `null`, so the review page can get stuck in a permanent loading state.</violation>
</file>

<file name="ai-pdf-chatbot/lib/stream/use-chat-stream.ts">

<violation number="1" location="ai-pdf-chatbot/lib/stream/use-chat-stream.ts:74">
P2: Sidebar chat list no longer refreshes after messages in existing chats, causing stale ordering/recency data.</violation>
</file>

<file name="ai-pdf-chatbot/components/pdf-drawer.tsx">

<violation number="1" location="ai-pdf-chatbot/components/pdf-drawer.tsx:58">
P2: The presigned-URL fetch effect is over-subscribed by depending on `ctx`, causing redundant refetches whenever viewer state changes within the same document.</violation>
</file>

<file name="ai-pdf-chatbot/app/api/workspaces/[id]/audio/route.ts">

<violation number="1" location="ai-pdf-chatbot/app/api/workspaces/[id]/audio/route.ts:54">
P2: Database query failures are misreported as "no ready documents" because `docsRes.error` is not handled.</violation>
</file>

<file name="ai-pdf-chatbot/app/workspaces/page.tsx">

<violation number="1" location="ai-pdf-chatbot/app/workspaces/page.tsx:28">
P2: Initial workspace load can hang forever on network errors because `refresh` does not handle rejected fetches.</violation>
</file>

<file name="ai-pdf-chatbot/app/api/documents/[id]/route.ts">

<violation number="1" location="ai-pdf-chatbot/app/api/documents/[id]/route.ts:33">
P2: The flashcard cascade update ignores database errors and still returns success, which can silently leave documents and flashcards out of sync.</violation>
</file>

<file name="ai-pdf-chatbot/app/workspaces/[id]/page.tsx">

<violation number="1" location="ai-pdf-chatbot/app/workspaces/[id]/page.tsx:96">
P2: Non-404 workspace fetch failures are unhandled, leaving the page stuck in a permanent loading state.</violation>

<violation number="2" location="ai-pdf-chatbot/app/workspaces/[id]/page.tsx:375">
P2: `disabled` does not work with `Button asChild` + `Link`, so users can still navigate to review when no cards are due.</violation>
</file>

<file name="ai-pdf-chatbot/app/api/workspaces/route.ts">

<violation number="1" location="ai-pdf-chatbot/app/api/workspaces/route.ts:31">
P2: `POST /api/workspaces` can throw on malformed field types because `.trim()` is called before verifying `name`/`description` are strings.</violation>
</file>

<file name="ai-pdf-chatbot/app/api/workspaces/[id]/flashcards/due/route.ts">

<violation number="1" location="ai-pdf-chatbot/app/api/workspaces/[id]/flashcards/due/route.ts:48">
P2: The document-name lookup ignores query errors and can return a successful response with incorrect fallback file names.</violation>
</file>

<file name="ai-pdf-chatbot/app/api/workspaces/[id]/mindmap/route.ts">

<violation number="1" location="ai-pdf-chatbot/app/api/workspaces/[id]/mindmap/route.ts:61">
P2: Database errors from the documents fetch are masked as a 409 "no documents" response because `docsRes.error` is never checked.</violation>
</file>

<file name="ai-pdf-chatbot/app/api/documents/upload/route.ts">

<violation number="1" location="ai-pdf-chatbot/app/api/documents/upload/route.ts:21">
P2: `workspaceId` is not validated before insert, so malformed IDs trigger DB errors and return 500 instead of a 400 validation response.</violation>
</file>

<file name="ai-pdf-chatbot/components/pdf-viewer.tsx">

<violation number="1" location="ai-pdf-chatbot/components/pdf-viewer.tsx:44">
P2: Previous citation highlights are not cleared when `highlightPrefix` becomes null, leaving stale highlighted text in the viewer.</violation>

<violation number="2" location="ai-pdf-chatbot/components/pdf-viewer.tsx:102">
P2: Next-page navigation can set an out-of-range page number before `numPages` is loaded.</violation>
</file>

<file name="ai-pdf-chatbot/components/workspace-picker.tsx">

<violation number="1" location="ai-pdf-chatbot/components/workspace-picker.tsx:30">
P2: Failed workspace-list requests leave the picker stuck in an infinite loading state.</violation>
</file>

<file name="ai-pdf-chatbot/app/api/workspaces/[id]/route.ts">

<violation number="1" location="ai-pdf-chatbot/app/api/workspaces/[id]/route.ts:43">
P2: GET masks count-query failures by defaulting null counts to 0, returning incorrect stats instead of an error.</violation>

<violation number="2" location="ai-pdf-chatbot/app/api/workspaces/[id]/route.ts:69">
P2: PATCH returns success even when no workspace row matches, so clients can get false-positive updates.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread ai-pdf-chatbot/lib/ai/audio-script.ts Outdated
Comment thread ai-pdf-chatbot/components/audio-overview.tsx
Comment thread ai-pdf-chatbot/app/workspaces/[id]/review/page.tsx Outdated
// new chat was created. Continuing a conversation already on
// the list shouldn't trigger a full sidebar refetch — that's
// the flash we were trying to avoid.
if (!params.chatId && resolvedChatId && typeof window !== 'undefined') {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Sidebar chat list no longer refreshes after messages in existing chats, causing stale ordering/recency data.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At ai-pdf-chatbot/lib/stream/use-chat-stream.ts, line 74:

<comment>Sidebar chat list no longer refreshes after messages in existing chats, causing stale ordering/recency data.</comment>

<file context>
@@ -62,10 +67,20 @@ export function useChatStream() {
+            // new chat was created. Continuing a conversation already on
+            // the list shouldn't trigger a full sidebar refetch — that's
+            // the flash we were trying to avoid.
+            if (!params.chatId && resolvedChatId && typeof window !== 'undefined') {
+              window.dispatchEvent(new Event('chats:changed'));
+            }
</file context>

Comment thread ai-pdf-chatbot/components/pdf-drawer.tsx Outdated
Comment thread ai-pdf-chatbot/components/pdf-viewer.tsx Outdated
Comment thread ai-pdf-chatbot/components/workspace-picker.tsx Outdated
Comment thread ai-pdf-chatbot/app/api/workspaces/[id]/route.ts Outdated
Comment thread ai-pdf-chatbot/app/api/workspaces/[id]/route.ts
Comment thread ai-pdf-chatbot/lib/pdf/pdf-worker.ts Outdated
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="ai-pdf-chatbot/lib/stream/use-chat-stream.ts">

<violation number="1" location="ai-pdf-chatbot/lib/stream/use-chat-stream.ts:74">
P2: Sidebar chat list no longer refreshes after messages in existing chats, causing stale ordering/recency data.</violation>
</file>

<file name="registry.json">

<violation number="1" location="registry.json:60">
P2: Features list replaced user-facing capabilities with implementation details, contradicting the PR description's stated intent. The PR description explicitly says: "registry entries updated to reflect Workspaces, Mindmap, Flashcards, and Audio Overview as marketplace features" — but the diff removes exactly those features. Marketplace users browsing templates see the features array; "react-pdf", "OpenAI TTS", and "Better Auth" are library/provider names that don't communicate what the template does. The description field (unchanged) still markets workspaces, mindmaps, flashcards, and audio overviews, but the features list no longer reflects them for filtering/discovery.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread registry.json
"framework": "nextjs",
"features": ["RAG", "pgvector", "OpenRouter", "Better Auth"],
"tags": ["rag", "pdf", "chatbot", "embeddings", "pgvector", "langchain-alternative"],
"features": ["RAG", "react-pdf", "OpenAI TTS", "Better Auth"],
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Features list replaced user-facing capabilities with implementation details, contradicting the PR description's stated intent. The PR description explicitly says: "registry entries updated to reflect Workspaces, Mindmap, Flashcards, and Audio Overview as marketplace features" — but the diff removes exactly those features. Marketplace users browsing templates see the features array; "react-pdf", "OpenAI TTS", and "Better Auth" are library/provider names that don't communicate what the template does. The description field (unchanged) still markets workspaces, mindmaps, flashcards, and audio overviews, but the features list no longer reflects them for filtering/discovery.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At registry.json, line 60:

<comment>Features list replaced user-facing capabilities with implementation details, contradicting the PR description's stated intent. The PR description explicitly says: "registry entries updated to reflect Workspaces, Mindmap, Flashcards, and Audio Overview as marketplace features" — but the diff removes exactly those features. Marketplace users browsing templates see the features array; "react-pdf", "OpenAI TTS", and "Better Auth" are library/provider names that don't communicate what the template does. The description field (unchanged) still markets workspaces, mindmaps, flashcards, and audio overviews, but the features list no longer reflects them for filtering/discovery.</comment>

<file context>
@@ -53,11 +53,11 @@
     "category": "ai",
     "framework": "nextjs",
-    "features": ["Workspaces", "Mindmap", "Flashcards", "Audio Overview"],
+    "features": ["RAG", "react-pdf", "OpenAI TTS", "Better Auth"],
     "tags": ["pdf", "study-tool", "notebooklm", "rag", "flashcards", "pgvector"],
     "cover": "assets/covers/ai-pdf-chatbot.png",
</file context>
Suggested change
"features": ["RAG", "react-pdf", "OpenAI TTS", "Better Auth"],
"features": ["Workspaces", "Mindmap", "Flashcards", "Audio Overview"],

Copy link
Copy Markdown
Member

@jwfing jwfing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - approved.

@CarmenDou CarmenDou merged commit e72dbe0 into main Jun 3, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants