Fix failover and health-monitor recovery for 429/5xx outages

[andreasfoo]

Summary

Fixes critical bugs in the failover and health-monitor recovery paths that prevented traffic from returning to recovered services after transient outages (429 rate limits, 5xx errors, and Anthropic's 529 overloaded status). The issues affected both streaming and non-streaming request modes.

Key Changes

Failover Circuit Breaker Recovery

• failover_dispatch.go: Changed isRetryableStatus() from an enumerated set to a range check (429 || 5xx), ensuring provider-specific codes like Anthropic's 529 and Cloudflare's 52x family trigger failover instead of being silently dropped
• failover_dispatch.go: Added success recording for buffered 2xx responses in DispatchWithPriorityFailover(). Previously, only committed gates recorded successes, leaving half-open probe slots claimed forever on non-streaming traffic — preventing breaker closure and traffic return to T0
• generic_stream_interceptor.go: Introduced sendEvent() helper that commits the failover gate before forwarding each client-bound SSE event. Streaming producers must signal on first real chunk; without this, multi-service rules buffered entire streams and never saw committed gates, making successful attempts invisible to the breaker

Health Monitor Recovery Window

• health_monitor.go: Fixed NewHealthMonitor() to apply default recovery timeout when config specifies zero. Previously, absent health_monitor config sections yielded zero recovery windows, causing 429-marked services to "auto-recover" on the next request, defeating the documented rate-limit window
• health_monitor.go: Updated UpdateConfig() to apply the same zero-value defaults for consistency

Service ID Parsing

• service_id.go: Added ParseServiceID() function to split canonical "provider/model" IDs (inverse of FormatServiceID)
• server.go: Fixed health probe function to use ParseServiceID() instead of splitting on ":" (which never matched the "/" format). This bug caused probes to fail for every service, pushing recovery windows forward indefinitely

Test Coverage & Virtual Models

• failover_dispatch_test.go: Updated TestIsRetryableStatus() to verify the full 5xx range (529, 520, 599) and 429 are retryable
• defaults_shared.go: Added virtual-fail-529 mock spec for Anthropic's overloaded_error, enabling end-to-end testing of the 529 failover path
• error_integration_test.go: Updated count assertion for extended error specs (5 → 6)

End-to-End Timeline Tests

• failover_timeline_test.go (new): Comprehensive e2e test suite scripting wall-clock failover scenarios using a fake clock:
  • TestFailoverTimeline_PrimaryDownThenRecover: Two-tier failover with 429/500/529 outages in streaming and non-streaming modes, verifying traffic returns after breaker recovery
  • TestFailoverTimeline_ThreeTierCascade: Three-tier cascade with multiple simultaneous failures, confirming requests flow through all tiers and honest error propagation when all are down
• failover_timeline.go (new): SwitchableUpstream fixture providing runtime-flippable vmodel upstreams and SetupTimelineFailoverRoute() helper for multi-tier test harnesses

Documentation

• .design/tier-routing.md: Clarified health recovery probe behavior and serviceID parsing format

Implementation Details

The root causes were:

1. Enumerated status codes silently dropped provider-specific 5xx variants (529, 52x), breaking failover for exactly the outages they signal
2. Buffered gate success recording only on commit, leaving non-streaming responses (c.JSON, buffering stream producers) invisible to the breaker
3. Streaming gate commit timing in MCP interceptor — entire streams buffered before any commit, preventing half-open probes from closing breakers
4. Zero-value health monitor defaults never applied, causing