Resilient outbound-HTTP toolkit for Go: retry, backoff, transient-error classification, and more.
A resilient outbound-HTTP toolkit for Go providing jittered exponential backoff, transient-error classification, Retry-After parsing, HTTP status mapping, secret redaction, body draining, a transparent retrying http.RoundTripper with body replay, and a configurable redirect allowlist. Zero dependencies beyond the Go standard library and pgregory.net/rapid (test only).
The toolkit presents three retry doors sharing one option vocabulary:
Do[T]retries a typed operation you own (any closure returning(T, error)).GetBytesretries an HTTP GET and returns bounded, redaction-safe body bytes.NewRetryRoundTripperretries transparently inside a stdlibhttp.RoundTripper.
NewRetryClient assembles the retrying client (transport + an explicit, required redirect policy) in one call.
v4 tightens what counts as success: CheckHTTPStatus returns nil for 2xx only, so a 3xx surfaced by a non-following redirect policy is now an error (see Status checking and Migrating from v3). v5 hardens signatures with no behavior change: the redaction helpers take the secret as the named Secret type, and WithSameHost / WithPreserveMethod take a bool (see Migrating from v4).
go get github.com/cplieger/httpx/v5@latest
// Bounded-bytes GET with retry
body, err := httpx.GetBytes(ctx, client, url,
httpx.WithMaxAttempts(3),
httpx.WithBaseDelay(time.Second),
)
// Generic typed retry
result, err := httpx.Do(ctx, func(ctx context.Context) (T, error) {
return doWork(ctx)
}, httpx.WithMaxAttempts(3), httpx.WithLabel("fetch"))
// Retry rate limits too (opt-in): a *RateLimitError is retried after
// min(its Retry-After hint, maxWait)
result, err := httpx.Do(ctx, fn,
httpx.WithRateLimitRetry(30*time.Second),
)
// Retry ONLY rate limits (transients fail fast; e.g. when transient retry is
// handled by an outer layer)
_, err := httpx.Do(ctx, func(ctx context.Context) (struct{}, error) {
return struct{}{}, download(ctx)
}, httpx.WithRateLimitOnly(5*time.Minute), httpx.WithMaxAttempts(3))
// Retrying client: transport retry + an explicit redirect policy in one call.
// The policy parameter is required (there is no safe universal default; nil
// panics). No Client.Timeout is set; bound totals with a context deadline or
// TransportConfig.MaxElapsedTime, and single attempts on the base transport.
client := httpx.NewRetryClient(nil, httpx.DefaultRedirectPolicy, httpx.TransportConfig{
MaxAttempts: 4,
BaseDelay: time.Second,
})
// Transparent retrying RoundTripper (inspired by hashicorp/go-retryablehttp)
rt := httpx.NewRetryRoundTripper(http.DefaultTransport, httpx.TransportConfig{
MaxAttempts: 4,
OnRetry: func(attempt int, req *http.Request, resp *http.Response, err error) {
log.Printf("retry #%d for %s", attempt, req.URL)
},
PrepareRetry: func(req *http.Request) error {
req.Header.Set("Authorization", "Bearer "+freshToken())
return nil
},
})
client := &http.Client{Transport: rt, CheckRedirect: httpx.DefaultRedirectPolicy}
// Retry POST/PUT with body replay (opt-in, requires GetBody)
client := httpx.NewRetryClient(nil, httpx.DefaultRedirectPolicy, httpx.TransportConfig{
MaxAttempts: 4,
RetryNonIdempotent: true,
})
payload := []byte(`{"key":"value"}`)
req, _ := http.NewRequest("POST", url, bytes.NewReader(payload))
req.GetBody = func() (io.ReadCloser, error) {
return io.NopCloser(bytes.NewReader(payload)), nil
}
resp, err := client.Do(req)
// PermanentError: signal "do not retry" (mirrors cenkalti/backoff)
if configErr != nil {
return httpx.Permanent(configErr) // will not be retried
}
// Custom redirect policy
policy := httpx.RedirectPolicyFunc(
httpx.WithAllowedHosts("api.example.com"),
httpx.WithAllowedSuffixes(".cdn.example.com"),
httpx.WithMaxHops(3),
)
// Refuse a redirect that would change the method (POST -> GET across a
// 301/302/303) instead of silently re-issuing it as a GET. The refused hop
// surfaces the 3xx response itself, which CheckHTTPStatus reports as an error.
policy = httpx.RedirectPolicyFunc(httpx.WithSameHost(true), httpx.WithPreserveMethod(true))
// Status checking: nil for 2xx only. A 3xx (surfaced by a non-following
// policy) is an error.
if err := httpx.CheckHTTPStatus(resp); err != nil { /* typed: Auth/RateLimit/HTTPStatus */ }
// Pin a private / self-signed CA as the SOLE trust anchor (verification stays
// ON, TLS 1.2 minimum). The caller reads the PEM bytes (file, secret, env),
// keeping the helper I/O-free.
tr, err := httpx.CATransport(pemBytes)
client := &http.Client{Transport: tr, CheckRedirect: httpx.DefaultRedirectPolicy}
// ...or compose the pinned transport with retry:
client = httpx.NewRetryClient(tr, httpx.DefaultRedirectPolicy, httpx.TransportConfig{MaxAttempts: 3})
// Transient error classification
if httpx.IsTransient(err) { /* safe to retry */ }
// Limit response body size
rc := httpx.LimitedBody(resp, 1<<20) // 1 MB cap
defer rc.Close()v5 is signature hardening only: no behavior changes. The redaction helpers take the secret as a named type, Secret, so the value-to-hide and the text-to-scan cannot be transposed at a call site (a reversed call turns the redactor into a leak); the two presence-style redirect options take the bool their neighbour WithAllowSchemeDowngrade always did; and the two capability interfaces embed error.
| v4 | v5 |
|---|---|
RedactSecretString(s, secret) (both plain string) |
secret is the named type Secret: RedactSecretString(s, httpx.Secret(token)) |
RedactTransportError(err, prefix, secret) (prefix and secret both plain string) |
RedactTransportError(err, prefix, httpx.Secret(token)) |
WithSameHost() |
WithSameHost(true); false is the no-option default (only allowlisted targets are followed) |
WithPreserveMethod() |
WithPreserveMethod(true); false is the no-option default (a method-changing hop is followed as net/http rewrites it) |
Transient = IsTransient() bool |
Transient = error + IsTransient() bool |
RetryAfterHint = RetryAfterHint() time.Duration |
RetryAfterHint = error + RetryAfterHint() time.Duration |
module path suffix /v4 |
github.com/cplieger/httpx/v5 in go.mod and every import |
Go 1.26 toolchain (go 1.26.6) |
Go 1.27 toolchain (go 1.27.0); an older toolchain refuses to build the module |
A call passing an untyped string constant (RedactSecretString(s, "token")) compiles unchanged; only a string variable in the secret position needs the httpx.Secret(...) conversion. A transposed call (the secret where the text belongs) is now a compile error, which is the point. RedactSecret(err, secret) is unchanged: its parameters already differ in type, so a swap never compiled.
The two interface changes need no work in a normal implementor: an implementor is an error already, or it could not appear in the chain errors.As walks. Only a type that declared IsTransient()/RetryAfterHint() without an Error() method stops satisfying; such a type was never reachable through either interface's only access path. What the change buys the caller is errors.AsType: errors.AsType[httpx.Transient](err) and errors.AsType[httpx.RetryAfterHint](err) now compile, where before errors.AsType's [E error] constraint refused both and go fix -errorsastype produced code that did not build.
Two changes, one of them breaking.
| v3 | v4 |
|---|---|
CheckHTTPStatus returns nil for 200-399 |
BREAKING: nil for 2xx only; every other status errors, a 3xx included (*HTTPStatusError) |
| a method-changing redirect hop is followed as a GET | opt in to refusing it with RedirectPolicyFunc(..., WithPreserveMethod(true)) (additive, off by default) |
github.com/cplieger/httpx/v3 |
github.com/cplieger/httpx/v5 in go.mod and every import; coming from v3 you cross the v5 signature changes in the same bump, so apply Migrating from v4 too |
What changes: a 3xx that reaches your code now returns *HTTPStatusError{Code} instead of nil.
Who is affected: only callers that pair a non-following redirect policy (RefuseAllRedirects, or any CheckRedirect returning http.ErrUseLastResponse) with CheckHTTPStatus. A followed redirect never surfaces a 3xx, so a client on DefaultRedirectPolicy or an allowlist policy sees no difference. GetBytes and the RetryRoundTripper are untouched: GetBytes already returned a *StatusError for a 3xx (it now derives that from the classifier instead of restating the band), and the transport never classified statuses through it. DoConditional still errors on a 3xx, but the error is now *HTTPStatusError{Code} instead of the plain unexpected status %d fallback; a caller matching on that message must switch to errors.As.
What to do: delete the hand-rolled band check next to the call: a guard like if resp.StatusCode < 200 || resp.StatusCode >= 300 { ... } sitting after CheckHTTPStatus (written precisely because the classifier accepted 3xx) is now redundant. If a 3xx must stay non-fatal for one call site, check resp.StatusCode there instead of widening the classifier back.
Do[T]: generic retry with jittered exponential backoff. When a transient error implementsRetryAfterHint, its pre-capped duration replaces the backoff for the next wait (the exponential base keeps advancing). Rate-limit handling is opt-in per call:WithRateLimitRetry(maxWait)adds*RateLimitErrorto the retryable set;WithRateLimitOnly(maxWait)retries nothing else.WithAttemptTimeout(d)adds a retryable per-attempt bound. Counts total attempts (a non-positive count clamps to 1).GetBytes: HTTP GET with exponential backoff on 408/429/5xx and transient transport errors (timeouts, connection resets, DNS failures; seeIsTransient); other 4xx and non-transient transport errors return immediately. Honors Retry-After (capped atRetryAfterCap). Counts total attempts.NewRetryRoundTripper(base, TransportConfig{...}): create a retryinghttp.RoundTripper.TransportConfig{}is ready to use (3 attempts, 1s base delay, default policy);MaxAttempts: -1means exactly one attempt.
Shared by both loop doors (Option): WithMaxAttempts, WithBaseDelay, WithLogger, WithExhaustedLevel. Do-only (DoOption): WithLabel, WithAttemptTimeout, WithRateLimitRetry, WithRateLimitOnly. GetBytes-only (GetOption): WithHeaders, WithMaxBodyBytes. Passing an option to the wrong door is a compile error. A non-positive rate-limit maxWait falls back to RetryAfterCap (60s), so the inter-attempt wait is always positive (never a hot spin); supplying both rate-limit modes is a configuration error.
NewClient(timeout): simple client with the same-hostDefaultRedirectPolicypreinstalled.NewRetryClient(base, policy, cfg): retrying client;policyis required (nil panics; a nilCheckRedirectwould silently mean net/http's follow-anywhere default). Sets noClient.Timeout(it would cap the whole retry sequence); see Timeouts and deadlines for bounding attempts and totals.ContextWithDefaultTimeout(ctx, def): boundctxbydefonly when the caller brought no deadline. A caller deadline is never undercut or extended;def <= 0means no default; the returned cancel is always non-nil. The per-request timeout rule for an API client.
CATransport(pem): build an*http.Transport(cloned fromhttp.DefaultTransport, so pooling, timeouts, and proxy settings are preserved) that pins the CA certificate(s) inpemas the sole trust anchors. Verification stays on (InsecureSkipVerifyis never set) with a TLS 1.2 minimum. Returns the concrete, mutable transport so it composes withNewRetryRoundTripper. The caller reads the PEM bytes (file, secret, env), keeping the helper I/O-free.ErrNoCertsInPEM: returned byCATransportwhenpemyields no certificates (a loud error instead of a silently-empty pool).CloneDefaultTransport(): a private clone ofhttp.DefaultTransportthat is yours to mutate (a per-attemptResponseHeaderTimeout,MaxIdleConnsPerHost, the base ofNewRetryRoundTripper) without reconfiguring every other client in the process. Errors whenhttp.DefaultTransporthas been replaced by a non-*http.Transport.
The github.com/cplieger/httpx/v5/certtest subpackage supplies throwaway self-signed CA material for tests, the companion to CATransport. Only _test.go files import it, so its certificate-generation code never links into a production binary.
certtest.SelfSignedCA(tb): a fresh self-signed CA certificate, PEM-encoded. Each call generates a new key, so two certs are mutually untrusted (handy for asserting a pin is enforced).certtest.WriteSelfSignedCA(tb): the same certificate written to aca.pemfile undertb.TempDir(), returning the path.
CheckRetry: pluggable retry policy,func(ctx, resp, err) (bool, error). The default retries transient transport errors and 429/502/503/504 (deliberately narrower thanGetBytes, which retries 408 and every 5xx).OnRetry: per-attempt callback for observability/metrics (the transport's only seam; it logs nothing itself)PrepareRetry: mutate the request before a retry (e.g., re-sign tokens)MaxElapsedTime: hard total-time ceiling across retries, including honored Retry-After (checked between attempts)RetryNonIdempotent: opt-in POST/PUT/PATCH/DELETE retry withGetBodyreplay
IsTransient: classify errors as transient (retryable); respectsPermanentError. A caller's expired or canceled context is terminal; anhttp.Client.Timeoutor transportResponseHeaderTimeoutis a per-attempt bound and IS retried (see Timeouts and deadlines)AttemptTimeout(err)/IsAttemptTimeout(err): mark a timeout as the expiry of a bound over ONE attempt, making it retryable, and test for that mark. The mirror ofPermanent, and the only way an error that CARRIEScontext.DeadlineExceededbecomes retryable; the mark keeps the deadline visible toerrors.Is(err, context.DeadlineExceeded)for the caller's own callers.WithAttemptTimeoutapplies it for you (see Timeouts and deadlines).RetryAfterHint: an interface (error+RetryAfterHint() time.Duration) an error implements to supply the next retry wait.Dohonors it when the error is transient and the duration is positive; the implementer must cap the value, since httpx applies no ceiling of its own here.Transient: an interface (error+IsTransient() bool) an error implements to declare its own retryability, consulted byIsTransient. Both capability interfaces embederror, likenet.Error, so a caller can writeerrors.AsType[httpx.Transient](err).CheckHTTPStatus: map an HTTP status to a typed error; success is 2xx only (see Status checking)IsRetryableStatus(code): does the retry loop treat this status as transient? True for 408, 429, and any 5xx. It is the same rule the built-in retry uses, not a copy of it:GetBytes's attempt function calls this function, so the two cannot drift. For the caller that runsGetByteswithWithMaxAttempts(1)inside its own retry budget and therefore classifies the returned*StatusErroritself (see Nesting a door in your own retry loop). TheRetryRoundTripper's default policy is narrower (429/502/503/504); pass this predicate throughTransportConfig.CheckRetryto widen it.ParseRetryAfter/ParseRetryAfterResponse: parse a Retry-After header (capped atRetryAfterCap/ raw)
CheckHTTPStatus(resp) returns nil for exactly 2xx (200-299) and an error for every other status:
| Status | Result |
|---|---|
| 2xx | nil |
| 3xx | *HTTPStatusError{Code} (not transient, not a client or server error) |
| 401 / 403 | *AuthError (the message carries (401) / (403)) |
| 429 | *RateLimitError with the raw, uncapped Retry-After hint |
| other 4xx / 5xx | *HTTPStatusError{Code} (transient for 502/503/504) |
| 1xx | *HTTPStatusError{Code} (not a completed response) |
The 2xx-only window is a v4 breaking change (v3 returned nil for the whole 200-399 band). A 3xx only ever reaches a caller when the client is configured not to follow redirects: RefuseAllRedirects, or any CheckRedirect returning http.ErrUseLastResponse, which net/http hands back as the 3xx response itself with a nil error. Under the old window that redirect stub classified as success, so a token-bearing client that deliberately refuses the hop (exactly the client RefuseAllRedirects exists for) then treated the unfollowed redirect as a completed request. CheckHTTPStatus is the status handling that policy delegates to, and it now reports the 3xx as the failure it is.
A 3xx is deliberately an *HTTPStatusError rather than a new type, so it flows through the existing plumbing unchanged: IsTransient is false (only 502/503/504 are transient), IsServerError and IsClientError are both false, and LogSafeError and the redaction helpers pass it through (it embeds no URL). There is no second, stricter classifier; this is the only one. See Migrating from v3 for the migration.
Permanent(err): wrap an error to signal "do not retry" (mirrors cenkalti/backoff)IsPermanent(err): check whether an error is wrapped as permanentPermanentError: the wrapper type (supportserrors.Is/errors.As/Unwrap)MarkTransient(err): wrap an error to signal "retry this". The mirror ofPermanent, for a failure your operation knows is self-healing where the shared policy cannot know (a server-side fault delivered inside a 200 envelope, an upstream whose plain 500 always clears). It saves you declaring a one-methodTransientwrapper, and cannot forgetUnwrapthe way a hand-rolled one does. The mark is the outermost verdict, so it overrides a non-transient verdict already on the error, butIsTransient's standing rejections still win:Permanent, an*AuthError, a*RateLimitError(retry a rate limit by naming a wait budget instead:WithRateLimitRetry), and a caller-context error stay terminal. For a per-attempt timeout carrying a deadline, useAttemptTimeout.
Running a door with WithMaxAttempts(1) inside your own retry loop is the sanctioned way to avoid multiplying the two attempt counts (a 3-attempt door inside a 3-attempt loop is 9 requests). The door then makes no retry decision, so you make it. The two exported predicates are the door's own rule, so your loop and the built-in one classify identically:
// One attempt per outer attempt; this loop owns the budget.
body, err := httpx.GetBytes(ctx, client, url, httpx.WithMaxAttempts(1))
if err != nil {
// A self-healing status is worth another of MY attempts. GetBytes
// deliberately does NOT mark its exhaustion error transient: after
// WithMaxAttempts(1) that decision is the caller's policy.
if se, ok := errors.AsType[*httpx.StatusError](err); ok && httpx.IsRetryableStatus(se.Code) {
return httpx.MarkTransient(err)
}
return err // auth/config failure: terminal, fail on the first attempt
}The exhaustion error still implements RetryAfterHint, so the upstream's already-capped Retry-After survives into the enclosing Do and is waited instead of the jittered backoff.
JitteredBackoff: equal jitter,[backoff/2, backoff]SafeDouble/SleepCtx: overflow-safe doubling, context-aware sleep
Drain/DrainClose: drain a body for connection reuse (64 KB limit). A failed drain logs a bare Debug line and never the read error itself: that text is written by the far end (see URL redaction in logs and errors)LimitedBody: wrap a response body with a size capReadLimitedBody: read a body to a cap (closing it) with overflow detection, returning*ResponseTooLargeErrorinstead of a silently truncated body
Validators{ETag, LastModified}: the cache validators captured from a previous 200, replayed on the next requestConditionalResult{Validators, Body, NotModified}: one conditional-request outcomeDoConditional(client, req, v, maxBodyBytes): one conditional attempt;valone decides what is replayed (pre-existing conditional headers are cleared, empty fields unsent). A 304 returnsNotModifiedwith zeroValidators(keep the ones you sent); a 200 returns the bounded body plus fresh validators; anything else is an error, with transport errors reduced viaLogSafeErrorso no raw URL reaches caller error text. Single-shot by design: wrap it inDofor retry, rebuild the request per attempt, persist body and validators together, and send zeroValidatorswhen the cached body is unusable. Validators are checked in both directions (header field-value grammar, 1 KiB cap): an invalid upstream value is captured as empty and an invalid replayed field is unsent, so a poisoned validator degrades to an unconditional GET and self-heals on the next clean 200. Full semantics in the godoc.
DefaultRedirectPolicy: same-host-only (used byNewClient); refuses a same-hosthttps->httpdowngrade, allows anhttp->httpsupgrade.RefuseAllRedirects: follows no redirect; returnshttp.ErrUseLastResponse, so the client surfaces the 3xx response itself (nil error) andCheckHTTPStatusreports it as an error. The policy for a token-bearing client of an API that issues no redirects: Go forwards custom headers (X-Plex-Token,X-Api-Key) across redirects, so a hostile 302 would exfiltrate the credential.DockerGitHubRedirectPolicy: example allowlist policy for docker.com/github.com.RedirectPolicyFunc: build a custom redirect allowlist from functional options:WithAllowedHosts,WithAllowedSuffixes,WithSameHost(true)(also allow the original request's host; the same-origin building block;falseis the no-option default),WithMaxHops,WithAllowSchemeDowngrade, andWithPreserveMethod. Every policy refuses anhttps->httpdowngrade by default, even to an allowlisted or same-host target, so an auth header is never forwarded onto a cleartext hop; anhttp->httpsupgrade is always allowed, andWithAllowSchemeDowngrade(true)opts out of the refusal.WithPreserveMethod(true): refuses a hop that would change the request method instead of rewriting the method back (falseis the no-option default: the hop is followed as net/http rewrites it). net/http downgrades a POST/PUT/PATCH/DELETE to a GET across a 301/302/303 and drops the body (RFC 9110 §15.4, Go issue 18570); only 307/308 carry the method forward. The refusal returnshttp.ErrUseLastResponse, so the 3xx surfaces to the caller (nil error) and errors underCheckHTTPStatus, the same pairingRefuseAllRedirectsrelies on. The comparison is against the original request, so a POST kept by a 307 and then downgraded by a 302 is refused at the second hop; an emptyviachain fails closed. The hop cap, allowlist, and scheme-downgrade refusals (hard errors) keep precedence, and the option grants nothing on its own: with no allowlist and noWithSameHost(true)the policy still refuses everything.CheckRedirect: thehttp.Client.CheckRedirectfunction shape as a type alias; every shipped policy is one.
RedactTransportError/RedactSecret/RedactSecretString: secret redaction (error- and string-level).RedactSecretStringandRedactTransportErrortake the secret as the namedSecrettype, so the value-to-hide and the text-to-scan cannot be transposed. A reversed call would turn the redactor into a leak, and it no longer compiles.LogSafeError: reduce a URL-embedding transport*url.Errorto its underlying cause (everything else passes through,errors.Is/Aspreserved). The same reduction httpx applies to every transport error it logs; equivalent toRedactTransportError(err, "", "").
AuthError/RateLimitError/HTTPStatusError/StatusErrorResponseTooLargeError: returned byGetByteswhen the response exceedsWithMaxBodyBytes(carriesLimit; no body is returned)ErrRateLimited/ErrServerError: sentinel errorsPermanentError: do-not-retry sentinel wrapper
Do and GetBytes log via log/slog and accept WithLogger to override the default logger per call. Per-attempt "retrying" lines are logged at Debug; a retry that recovers is normal operation, not a degraded state. The terminal "retries exhausted" / "rate limit retries exhausted" lines are at Warn, except under WithMaxAttempts(1), where they drop to Debug: a one-attempt budget retried nothing, so the door is a single attempt inside the caller's own retry loop, and that loop owns both the retry policy and the warning. WithExhaustedLevel(level) overrides that line's level outright, for callers whose own failure log carries strictly more context than the library's can (the tracker, the item, an onset latch): demoting the library's copy keeps one report of one event without discarding the per-attempt Debug diagnostics a discard logger would also throw away. It applies to a multi-attempt budget too, and is the only way to raise the line above Warn. GetBytes also emits a Warn "slow upstream response" when a single attempt's response takes longer than 10s (timed per attempt, so backoff sleeps are not counted as upstream latency). The RetryRoundTripper logs nothing itself; observe its retries through the OnRetry hook, where redaction is the caller's responsibility. Drain/DrainClose emit one Debug line, failed to drain response body, carrying no attributes. A failed drain only forfeits connection reuse, and the read error that caused it is remote-authored text this package refuses to log (see below).
To avoid leaking credentials into logs (CWE-532, the class of go-retryablehttp CVE-2024-6104), GetBytes never logs or returns a raw request URL:
- Every logged
urlattribute is redacted: the whole userinfo component is replaced withREDACTED(stronger thanurl.URL.Redacted, which masks only the password and would leave a username-only API token in the clear) and query values are replaced withREDACTED(query values commonly carry API keys and tokens). Query keys, scheme, host, and path are kept for debugging. StatusError.Error()renders that same redacted URL, so the secret stays out of returned errors too; the rawStatusError.URLfield remains available for programmatic use.- Transport errors (
*url.Error, which embed the full URL) are reduced to their underlying cause before logging. The reduction is exported asLogSafeErrorso callers wrapping transport errors into their own messages can apply the same one.
That reduction is type-based (it strips an envelope this library itself added), so it cannot sanitize text the far end wrote. Drain/DrainClose therefore drop the body-read error instead of logging it: net/http renders a malformed chunked trailer as malformed MIME header: missing colon: "<remote bytes>", and for a URL that carries its credential in the path (a webhook token) an edge echoing the request URI puts that credential in those bytes. The drain site logs on the package-level default logger, which no option can reroute, so the value is dropped there rather than left for callers to handle. Nothing an operator can act on is lost, because a drain runs only where the body is already being discarded and its outcome is reported by the path that discarded it.
GetBytes and the RetryRoundTripper report exhaustion differently; match your error handling to the one you use:
GetBytesreturnsnilbody and a wrapped error:retries exhausted after <elapsed>: <lastErr>(unwrap witherrors.Is/errors.As). A response that overflowsWithMaxBodyBytesreturns*ResponseTooLargeError(no body). When the last attempt carried aRetry-After, the exhaustion error also implementsRetryAfterHintwith that capped wait, so a caller runningGetByteswithWithMaxAttempts(1)inside its ownDoloop (the pattern that avoids multiplying the two attempt budgets) still honors the upstream-requested delay instead of falling back to jittered backoff. It deliberately does not implementTransient: whether an exhausted GET is worth another outer attempt stays the caller's policy.RetryRoundTripperreturns the last response with a nil error, even when that response is a retryable 5xx (e.g. a 503), mirroring how a non-retried request behaves. A caller that checks onlyerr != nilwill treat an exhausted 503 as success, so inspectresp.StatusCodeand close the body. (A budget abort viaMaxElapsedTimedoes return an error.)
httpx retries transient failures, not budget expiry. IsTransient classifies a caller's expired or canceled context as non-transient, while a connection reset, a DNS error, a net.Error timeout, and a 429/5xx are transient. A caller's context deadline means "the budget is exhausted, stop"; the expiry of a bound over ONE attempt means "this attempt failed, try again".
The two are told apart by whether the context.DeadlineExceeded value is in the error's unwrap chain, not by errors.Is: net/http's own timeout error reports errors.Is(err, context.DeadlineExceeded) == true without ever carrying the sentinel, so errors.Is cannot distinguish the bounds net/http installs from the caller's own. Where the difference is genuinely unknowable (a bound the net package reports, since it maps every expired context onto one shared i/o timeout value), the verdict stays terminal, and AttemptTimeout(err) is how the code that installed the bound opts back in.
- Total budget: a context deadline. Pass a
context.WithTimeout(or a caller-supplied deadline) as the single authoritative bound. It spans every attempt and every backoff sleep (SleepCtxcaps the backoff by it); on expiry the call ends, terminal, not retried. - Per-attempt bound under
Do:WithAttemptTimeout(d). Each attempt runs under a context bounded byd, and that bound's expiry is retried, the gRPC per-try-timeout model. A caller deadline nearer thandstill governs (context keeps the earlier deadline), sodcaps one attempt and never extends the total. The expiry is marked as the attempt's only while the caller's own context is still live, so a caller out of budget stays terminal. The attempt context is canceled whenfnreturns, so read or drain the body insidefn. - Per-attempt bound on the client or transport:
http.Client.TimeoutandResponseHeaderTimeout, retried. net/http reports both through its own timeout error, which claims to matchcontext.DeadlineExceededbut never carries it, so both classify as per-attempt and are retried, including under theRetryRoundTripper's default policy. UsePermanent(err)in aTransportConfig.CheckRetryif you want one of them to stop the loop instead. - Per-attempt bound anywhere else: mark it with
AttemptTimeout(err). Acontext.WithTimeouta caller derives itself insidefncarries the real sentinel, and a net-level bound (anet.DialerTimeout) is indistinguishable from a caller deadline thenetpackage mapped; both stay terminal until marked.WithAttemptTimeoutis the first case done for you. GetBytestakes no per-attempt option: the door owns its request and has no callback to bound. Bound it with theClient.Timeoutof the client you pass, or run it as one attempt insideDo(WithMaxAttempts(1)on the GET,WithAttemptTimeout(d)on theDo), which is also the composition that avoids multiplying two attempt budgets.
Under NewRetryRoundTripper / NewRetryClient the retry loop runs inside client.Do, so http.Client.Timeout is not per-attempt at all: it caps the whole retry sequence and a slow attempt that trips it aborts the remaining retries, which is why NewRetryClient sets none. Put the per-attempt bound on the base transport (ResponseHeaderTimeout, retried) and the total on the caller's context or TransportConfig.MaxElapsedTime. Neither the between-attempt MaxElapsedTime check nor an expired context can interrupt an attempt already stalled inside the base transport; only a transport-level timeout can.
The following features are intentionally not provided:
| Feature | Rationale |
|---|---|
| Circuit breaker | Orthogonal pattern excluded by all comparables. Compose externally with sony/gobreaker. |
| Retry budget / token bucket | None of the comparables implement it. Disproportionate complexity (~150 LOC + shared mutable state) for a focused library. |
| Multiple jitter strategies (full, decorrelated) | Equal jitter is the recommended default per AWS Builders' Library. Full jitter risks near-zero delays. |
ErrorHandler for exhaustion |
Current fmt.Errorf("retries exhausted: %w", lastErr) is sufficient. Callers unwrap. |
| Response body on error | Adds API complexity (ownership of body close). Use Do[T] with custom logic. |
| Idempotency key injection | Application-level concern, not a retry library's responsibility. |
| Configurable Retry-After cap | A raisable cap would regress the fixed-60s DoS ceiling (ParseRetryAfter); rate-limit waits are capped by the caller-owned maxWait arguments. |
Issues and PRs are welcome. See CONTRIBUTING.md for the conventions and how to run the checks locally.
This project is built with care and follows security best practices, but it is intended for personal / self-hosted use. No guarantees of fitness for production environments. Use at your own risk.
This project was built with AI-assisted tooling using Claude, GPT, and Kiro. The human maintainer defines architecture, supervises implementation, and makes all final decisions.
Apache-2.0. See LICENSE.