Overview
Traffic shaping and response caching capabilities: RFC 9111 compliant caching, request coalescing, request mirroring, and traffic splitting.
Response Caching (RFC 9111)
Shared cache semantics per RFC 9111. Cacheable response determination, Cache-Control directive handling (max-age, no-cache, no-store, public, private, must-revalidate), ETag, Last-Modified, Vary, Age, and Expires support.
Cache Key Composition
Method + host + path + query string + Vary header values. Vary-aware keying ensures correct content negotiation.
Conditional Requests
If-None-Match (ETag) and If-Modified-Since (Last-Modified) support. Return 304 Not Modified when conditions are met.
Cache Invalidation
Multiple strategies: PURGE method for explicit invalidation, TTL-based expiration, and tag-based bulk invalidation.
Stale Serving
stale-while-revalidate: serve stale while revalidating in the background. stale-if-error: fall back to stale when upstream errors occur.
Cache Storage Backends
Pluggable backend trait. In-memory default (DashMap with TTL eviction). Redis/Valkey for distributed multi-instance deployments.
Request Coalescing
Collapse concurrent identical requests behind a single upstream fetch. Prevents cache stampede on popular resources.
Request Mirroring
Async fire-and-forget copy of requests to a shadow cluster. Shadow response is discarded. Enables testing new upstreams with production traffic.
Traffic Splitting
Percentage-based weighted routing between clusters. Enables canary deployments and gradual rollouts.
Semantic Caching
Embedding-based prompt similarity for AI inference deduplication. Separate from RFC 9111 caching; uses vector similarity rather than exact key matching.
Related Issues
Overview
Traffic shaping and response caching capabilities: RFC 9111 compliant caching, request coalescing, request mirroring, and traffic splitting.
Response Caching (RFC 9111)
Shared cache semantics per RFC 9111. Cacheable response determination, Cache-Control directive handling (max-age, no-cache, no-store, public, private, must-revalidate), ETag, Last-Modified, Vary, Age, and Expires support.
Cache Key Composition
Method + host + path + query string + Vary header values. Vary-aware keying ensures correct content negotiation.
Conditional Requests
If-None-Match (ETag) and If-Modified-Since (Last-Modified) support. Return 304 Not Modified when conditions are met.
Cache Invalidation
Multiple strategies: PURGE method for explicit invalidation, TTL-based expiration, and tag-based bulk invalidation.
Stale Serving
stale-while-revalidate: serve stale while revalidating in the background. stale-if-error: fall back to stale when upstream errors occur.
Cache Storage Backends
Pluggable backend trait. In-memory default (DashMap with TTL eviction). Redis/Valkey for distributed multi-instance deployments.
Request Coalescing
Collapse concurrent identical requests behind a single upstream fetch. Prevents cache stampede on popular resources.
Request Mirroring
Async fire-and-forget copy of requests to a shadow cluster. Shadow response is discarded. Enables testing new upstreams with production traffic.
Traffic Splitting
Percentage-based weighted routing between clusters. Enables canary deployments and gradual rollouts.
Semantic Caching
Embedding-based prompt similarity for AI inference deduplication. Separate from RFC 9111 caching; uses vector similarity rather than exact key matching.
Related Issues