Skip to content

feat: retry streamed server_is_overloaded SSE errors #2712

@axsuul

Description

@axsuul

request-retry looks like it covers certain HTTP status codes, but I don't think it clearly covers the case where the upstream returns 200 OK and then emits a streamed SSE error like:

  • error.type = service_unavailable_error
  • error.code = server_is_overloaded

From an operator/client point of view, this seems like the same class of transient failure as an HTTP 503, just encoded differently.

Would it make sense to treat this streamed overload error as retryable and send it through the existing request-retry / fallback path?

What I'm asking for:

  • if a stream emits service_unavailable_error + server_is_overloaded before a successful response is produced, mark it retryable
  • reuse the normal retry/fallback flow, including credential rotation if applicable
  • document whether this is covered by request-retry or by separate streaming retry rules

The reason I'm asking is that right now overload handling seems inconsistent depending on whether the upstream reports it as HTTP 503 or as a streamed SSE error after 200 OK.

Related:

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions