request-retry looks like it covers certain HTTP status codes, but I don't think it clearly covers the case where the upstream returns 200 OK and then emits a streamed SSE error like:
error.type = service_unavailable_error
error.code = server_is_overloaded
From an operator/client point of view, this seems like the same class of transient failure as an HTTP 503, just encoded differently.
Would it make sense to treat this streamed overload error as retryable and send it through the existing request-retry / fallback path?
What I'm asking for:
- if a stream emits
service_unavailable_error + server_is_overloaded before a successful response is produced, mark it retryable
- reuse the normal retry/fallback flow, including credential rotation if applicable
- document whether this is covered by
request-retry or by separate streaming retry rules
The reason I'm asking is that right now overload handling seems inconsistent depending on whether the upstream reports it as HTTP 503 or as a streamed SSE error after 200 OK.
Related:
request-retrylooks like it covers certain HTTP status codes, but I don't think it clearly covers the case where the upstream returns200 OKand then emits a streamed SSE error like:error.type = service_unavailable_errorerror.code = server_is_overloadedFrom an operator/client point of view, this seems like the same class of transient failure as an HTTP
503, just encoded differently.Would it make sense to treat this streamed overload error as retryable and send it through the existing
request-retry/ fallback path?What I'm asking for:
service_unavailable_error+server_is_overloadedbefore a successful response is produced, mark it retryablerequest-retryor by separate streaming retry rulesThe reason I'm asking is that right now overload handling seems inconsistent depending on whether the upstream reports it as HTTP
503or as a streamed SSE error after200 OK.Related: