Skip to content

fix(responses): keep public streams alive#2666

Open
ryanngit wants to merge 2 commits into
decolua:masterfrom
ryanngit:responses-stream-heartbeat
Open

fix(responses): keep public streams alive#2666
ryanngit wants to merge 2 commits into
decolua:masterfrom
ryanngit:responses-stream-heartbeat

Conversation

@ryanngit

@ryanngit ryanngit commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • return explicit streaming /v1/responses requests as SSE before account routing or provider headers complete
  • emit comment keepalives for generic clients while provider headers are pending
  • emit ignorable SSE events for Codex before and after provider headers so Codex idle timers observe activity
  • inject Codex keepalives only between complete SSE events, preserving fragmented upstream bytes and downstream backpressure
  • propagate downstream cancellation to provider fetches and avoid account cooldown for client-aborted attempts
  • frame delayed JSON/transport failures as schema-complete response.failed events
  • keep stream:false, omitted stream, invalid JSON, and provider-routing behavior unchanged

Root cause

Two independent timeout layers existed:

  1. Long account selection or provider header waits could exceed a reverse-proxy response-header timeout. The proxy returned 524 before 9Router emitted headers.
  2. After provider headers, Codex wraps parsed SSE events in an idle timeout. SSE comments are discarded by its event parser, so comment heartbeats did not reset that timer. A provider event arriving near 125 seconds could lose the race and produce idle timeout waiting for SSE.

Modern Codex UA handling is route-local here. Shared client detection and native provider pass-through remain unchanged.

Verification

  • focused Responses route/stream matrix: 12/12 passed
  • full unit differential from original PR: patched 1035 passed, 26 failed, 24 skipped; stock v0.5.35 1024 passed, same 26 failed assertions, 24 skipped
  • changed-file ESLint and git diff --check: passed
  • PR-range Gitleaks: 2 commits scanned, no leaks
  • npm run build: passed, TypeScript checked, 130 routes generated
  • 140-second public delayed-header canary from original PR: first byte 232 ms, five keepalives, one provider request, one completion, no 524/retry
  • 130-second isolated delayed-provider canary: five 25-second Codex event keepalives, then normal response.completed and [DONE]
  • public cancellation canary from original PR: provider request aborted, no active/completed orphan, no account lock/cooldown mutation
  • independent review: initial shared-detector scope issue fixed; re-review found no blocker

@ryanngit
ryanngit force-pushed the responses-stream-heartbeat branch from d47cfc0 to f3e3ac7 Compare July 19, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant