fix(chat): add heartbeat pings and abort streams on client disconnect - #8
Conversation
Abort the LLM loops and judge when the client disconnects so a closed tab or proxy reset doesn't keep generating an answer nobody reads. Emit a keep-alive ping on a configurable interval (STREAM_HEARTBEAT_MS, default 10s) so silent phases — cold first step, tool calls, best-of-N judge — never look like a stalled origin to nginx (60s), Cloudflare (100s), or EdgeOne (15s), which reset the stream after the 200 has been sent and surface as ERR_HTTP2_PROTOCOL_ERROR mid-answer. Also drop the Connection: keep-alive header, which is forbidden in HTTP/2. On the web side, coalesce markdown rendering behind requestAnimationFrame so a hidden, throttled tab keeps draining the response instead of stalling the HTTP/2 flow-control window, and show a stream-interrupted notice when the connection drops before completion.
|
Warning Review limit reached
Next review available in: 36 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThe PR adds configurable stream heartbeats, abort propagation for active LLM work, centralized NDJSON emission, disconnect-safe cleanup, richer tool events, and client handling for hidden-tab rendering and interrupted partial responses. ChangesChat streaming resilience
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/api/routes/chat.ts`:
- Around line 409-413: Update the heartbeat setup around the initial emit and
heartbeat interval so the first `ping` is emitted only when
`config.streamHeartbeatMs > 0`; preserve the existing disabled behavior by
avoiding both the initial ping and interval when the value is zero or negative.
- Around line 392-394: Update the disconnect listener in the chat route to use
reply.raw instead of request.raw, and invoke onClientGone only when the response
closes before reply.raw.writableEnded. Keep the existing premature-close guard
and response handling unchanged.
In `@web/src/App.svelte`:
- Around line 602-613: Update the rollback logic around ensureAiItem so
reasoning-only failures keep history and displayItems consistent. When
fullContent is empty but reasoning produced an assistant item, remove the
transient assistant state from displayItems or preserve the corresponding user
message in history for retries; ensure isRetry does not cause the retry request
to omit that user message.
- Around line 602-613: Update the streaming logic in App.svelte to track receipt
of the server finish event separately from the HTTP read result, including
preserving the associated turn statistics and interruption text. Use this finish
state when handling clean EOF and read errors after finish so completed turns
are not treated as interrupted and fullContent answers retain their stats; keep
genuine pre-finish failures on the existing error path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: a4dc3745-2d6a-469e-b721-1421a26db281
📒 Files selected for processing (6)
.env.examplesrc/agent/synthesize.tssrc/api/routes/chat.tssrc/config/index.tsweb/src/App.svelteweb/src/lib/i18n.ts
Three fixes from review of the heartbeat/abort change. `request.raw` emits 'close' as soon as the request body has been read, which is measurably before the response finishes streaming — a probe on Node v24 reports `REQ close | res.writableEnded=false` while the handler is still writing chunks. Guarding on that aborted every healthy turn the moment it started streaming. The response's own 'close' distinguishes the two cases: `writableFinished` is true when the body was flushed and false when the connection died under it. `STREAM_HEARTBEAT_MS=0` documents the heartbeat as disabled, but the initial ping was emitted unconditionally. Both the first ping and the interval now hang off the same check. The assistant bubble is created by the first delta of any kind, reasoning included, so a stream that broke after reasoning but before any answer text left a bubble with no matching `history` turn. Retrying from it deleted the *previous* turn's assistant message and re-sent a history missing the question, so the rollback now removes that bubble too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Abort the LLM loops and judge when the client disconnects so a closed tab or proxy reset doesn't keep generating an answer nobody reads. Emit a keep-alive ping on a configurable interval (STREAM_HEARTBEAT_MS, default 10s) so silent phases — cold first step, tool calls, best-of-N judge — never look like a stalled origin to nginx (60s), Cloudflare (100s), or EdgeOne (15s), which reset the stream after the 200 has been sent and surface as ERR_HTTP2_PROTOCOL_ERROR mid-answer. Also drop the Connection: keep-alive header, which is forbidden in HTTP/2.
On the web side, coalesce markdown rendering behind requestAnimationFrame so a hidden, throttled tab keeps draining the response instead of stalling the HTTP/2 flow-control window, and show a stream-interrupted notice when the connection drops before completion.
Summary by CodeRabbit
New Features
Bug Fixes