The per-product widget chat (apps/dashboard/app/api/widget/[publicKey]/chat/route.ts) waits for the full LLM reply before returning, so the user stares at nothing for ~2s. This is the biggest perceived-latency problem.
Goal
Stream tokens so text appears live (first token < ~1s), like ChatGPT/Fin.
Scope
- Change the chat route to return a streamed response (ReadableStream / SSE) instead of buffering the full reply.
- Update the Studio preview (
features/products/test-preview-panel.tsx) and the embed widget (public/embed.js) to consume the stream and render incrementally.
- Plain-text replies stream; the confirm-action (
{reply, action}) case can stay non-streamed for v1.
Acceptance
- First token visible in well under 1s on a warm route.
- Streaming works in both the Studio preview and the embedded widget.
- No regression to the confirm-action flow.
The per-product widget chat (
apps/dashboard/app/api/widget/[publicKey]/chat/route.ts) waits for the full LLM reply before returning, so the user stares at nothing for ~2s. This is the biggest perceived-latency problem.Goal
Stream tokens so text appears live (first token < ~1s), like ChatGPT/Fin.
Scope
features/products/test-preview-panel.tsx) and the embed widget (public/embed.js) to consume the stream and render incrementally.{reply, action}) case can stay non-streamed for v1.Acceptance