fix(rate-limit): gracefully handle GLM/Z.AI in-stream burst limits (HTTP 200 + [1302])

[jsboige]

Problem

Recurring GLM dropouts — Claude Code crashes with "empty or malformed response (HTTP 200)" mid-turn, requiring a manual continue / reprends stp to recover. Every one of those manual recoveries follows the same client-side crash.

Root cause (confirmed via CLAUDISH_CAPTURE_DIR captures)

Z.AI delivers its instantaneous burst / RPM limit as HTTP 200 followed by an in-stream SSE error ([1302]) that carries no message envelope. Because both response.ok and FallbackHandler key off HTTP status, a 200-with-in-stream-error is invisible to them — it flowed straight to Claude Code as a bare error and crashed the turn.

Key finding: the errors are instant (0–4 ms), not timeouts. The user's intuition that large context → more crashes is real but indirect — big context drives request bursting/concurrency (and cache misses), which trips the burst limit. It is not a duration/TTFB problem. Condensing helps by reducing request volume.

The 5h sustained quota has headroom (≤50% even at peak rush); only the instantaneous burst limit is hit, occasionally.

Fix — three proportionate, fail-open layers

1. Parser safety-net (anthropic-sse.ts): finalizeWithError() always emits a clean terminal message (synthetic message_start if none was seen, + message_delta end_turn + message_stop) and suppresses bare event: error. A stream error can no longer surface as a raw client crash; rate-limit errors get a friendly retry notice.

2. Transport backoff (anthropic-compat.ts): AnthropicProviderTransport.enqueueRequest retries genuine HTTP 429 with jittered exponential backoff (2/4/8/16/30s + 0–1s, honoring Retry-After).

3. In-stream peek + temporize (stream-peek.ts + composed-handler.ts): peek the first bytes of anthropic-sse responses via ReadableStream.tee() (no bytes lost downstream; bounded ~2s window catches the 0–4ms error without delaying slow-but-healthy big-context responses). On [1302], temporize with jittered backoff (1/2/4s + 0–750ms) and retry the same provider up to 3× (the sustained quota has headroom). On persistence, return 429 so FallbackHandler switches provider (Z.AI → GLM Coding — a separate quota). Fails open on any unexpected condition.

Jitter throughout de-synchronizes the 6-machine cluster (one shared key per provider) so retries don't re-collide on the burst limit.

Also enables CLAUDISH_CAPTURE_DIR in docker-compose for ongoing production diagnosis.

On "shared/adaptive backoff"

The design is per-request jittered exponential backoff, intentionally not a shared AIMD pacer. More 429s → more requests back off → offered load drops (implicit aggregate self-throttling), which is sufficient for the stated load. A coordinated pacer that makes new arrivals probe more cautiously during a storm is the correct next step only if the headroom premise stops holding — deferred to ROADMAP.md with explicit trigger conditions to avoid over-engineering.

Tests

• stream-peek.test.ts (5): full-byte passthrough on healthy, multi-read past leading ping, [1302]→rate-limit, non-RL→other-error (still streamed), no-body→fail-open.
• anthropic-compat.test.ts (+2): 429 retry-then-success; non-429 no-retry.
• format-translation.test.ts: [bug] error handling: undefined is not an object #106 regression rewritten to the no-crash contract + new start-of-stream rate-limit fixture/test.
• Build + tsc --noEmit clean. (One unrelated pre-existing MiniMaxModelDialect context-window test failure from the Firebase catalog migration — not touched here.)

🤖 Generated with Claude Code