Add scoped and config-driven HttpClient middleware

[DamianReeves]

Summary

kyo-http already has a strong foundation for HTTP middleware through HttpFilter: route-level filters, client-side helpers such as bearerAuth, basicAuth, addHeader, and logging, and a HttpFilter.Factory ServiceLoader SPI for globally discovered server and client filters.

This proposal is to add first-class programmatic client middleware configuration so users can apply client-wide behavior without attaching a filter to every route and without needing to publish a ServiceLoader factory. The main supported scenarios are auth, logging, tracing, metrics, request IDs, and other cross-cutting policies that should apply to all outgoing HTTP requests in a scoped block or reusable client config.

Motivation

A common HTTP client workflow is to build a client once with a policy pipeline:

HttpClient.withConfig(
    HttpClientConfig()
        .baseUrl("https://api.example.com")
        .filter(HttpFilter.client.bearerAuth(token))
        .filter(HttpFilter.client.addHeader("X-App", "kyo"))
) {
    HttpClient.getText("/users")
}

or to add a temporary policy around a block:

HttpClient.withFilter(HttpFilter.client.addHeader("X-Request-Id", requestId)) {
    HttpClient.postJson[CreateUser]("/users", body)
}

Today this is possible at the route level:

val route = HttpRoute
    .getText("/users")
    .filter(HttpFilter.client.bearerAuth(token))

and possible globally through HttpFilter.Factory.clientFilter, but there is no ergonomic middle ground for application code that wants scoped or config-owned client middleware.

Existing behavior

Current kyo-http support includes:

• HttpRoute.filter(...), which applies filters per route.
• HttpFilter.client, which includes client helpers for auth, custom headers, and logging.
• HttpFilter.Factory.clientFilter, which lets libraries contribute globally discovered client filters through ServiceLoader.
• Server-side equivalents via HttpFilter.server and HttpFilter.Factory.serverFilter.

The client backend already composes discovered client filters with route filters before sending the request. The proposed change extends that pipeline with programmatic filters.

Proposed API

Add a filter field to HttpClientConfig:

case class HttpClientConfig(
    baseUrl: Maybe[HttpUrl] = Absent,
    timeout: Duration = 5.seconds,
    connectTimeout: Duration = 30.seconds,
    followRedirects: Boolean = true,
    maxRedirects: Int = 10,
    retrySchedule: Maybe[Schedule] = Absent,
    retryOn: HttpStatus => Boolean = _.isServerError,
    transportConfig: HttpTransportConfig = HttpTransportConfig.default,
    tls: HttpTlsConfig = HttpTlsConfig.default,
    clientFilter: HttpFilter.Passthrough[Nothing] = HttpFilter.noop
)

Add builder methods:

def filter(f: HttpFilter.Passthrough[Nothing]): HttpClientConfig =
    copy(clientFilter = clientFilter.andThen(f))

def filters(fs: Seq[HttpFilter.Passthrough[Nothing]]): HttpClientConfig =
    copy(clientFilter = fs.foldLeft(clientFilter)(_.andThen(_)))

Add scoped helpers on HttpClient:

def withFilter[A, S](filter: HttpFilter.Passthrough[Nothing])(v: A < S)(using Frame): A < S

def withFilters[A, S](filters: Seq[HttpFilter.Passthrough[Nothing]])(v: A < S)(using Frame): A < S

The intended composition order for normal HTTP requests is:

HttpFilter.Factory.composedClient
    .andThen(config.clientFilter)
    .andThen(scopedClientFilter)
    .andThen(route.filter)
    .andThen(pool/send)

This preserves the existing SPI behavior while giving application code two programmatic attachment points:

• config filters, for reusable client policy
• scoped filters, for block-local additions

Route filters remain the most specific layer.

WebSocket handshakes

WebSocket connections currently bypass the normal HttpClient send path. The upgrade request is built directly in WebSocketCodec.requestUpgradeWith, so route-level and client-level HTTP filters do not naturally apply to the handshake.

This proposal should make client filters apply to the WebSocket HTTP upgrade handshake as well:

HttpClient.withConfig(_.filter(HttpFilter.client.bearerAuth(token))) {
    HttpClient.webSocket("wss://api.example.com/events") { ws =>
        // authenticated handshake
    }
}

For WebSockets, the pipeline would be:

HttpFilter.Factory.composedClient
    .andThen(config.clientFilter)
    .andThen(scopedClientFilter)
    .andThen(upgrade/send)

Route filters would not apply unless typed WebSocket routes are introduced later.

This should be limited to the handshake. Message-level WebSocket middleware is a different abstraction because, after the 101 response, there is no normal HttpResponse[Out]; there is a long-lived HttpWebSocket. If message interception becomes useful, it should probably be modeled separately, for example as HttpWebSocketMiddleware.

Supported scenarios

This would support:

• Auth headers applied to every HTTP request and WebSocket handshake.
• Request IDs or correlation IDs applied at a scoped boundary.
• Logging and tracing that wraps outbound requests without route boilerplate.
• Metrics and timing instrumentation from libraries through the existing SPI and from application code through config.
• Reusable API clients that carry base URL, timeout, TLS, retry, and middleware policy in one config value.
• Tests that install a temporary client filter without modifying route definitions or global ServiceLoader state.

Pros

• Reuses the existing HttpFilter model instead of introducing a parallel middleware type for basic request/response interception.
• Keeps ServiceLoader SPI immutable and cacheable while adding programmatic configuration.
• Avoids repeating .filter(...) on every route when the behavior is client-wide.
• Gives clear precedence: SPI, config, scoped block, route.
• Makes WebSocket authentication and tracing work through the same client policy used for HTTP requests.
• Preserves the current fast path when all filters are noop.

Possible cons and design questions

• HttpClientConfig would contain a function-like value, so equality and case-class ergonomics may become less useful. If this is a concern, the filter could live in the private HttpClient local context instead, but then middleware would not travel with reusable config values.
• Applying filters to WebSocket handshakes requires representing the handshake as a request-like value before WebSocketCodec writes bytes. Care is needed so required WebSocket headers such as Upgrade, Connection, Sec-WebSocket-Key, and Sec-WebSocket-Version remain correct.
• The exact ordering should be documented. SPI first is useful for library instrumentation, while route last keeps endpoint-specific behavior closest to the send.
• Retrying and redirect behavior should be documented. Existing client filters run at the pool/send layer, so they apply per wire attempt rather than only once per logical request.
• HttpFilter.Passthrough[Nothing] is intentionally conservative for config and scoped filters. If users need typed field additions or custom error types at client-config scope, that likely needs a separate design.

Testing expectations

The change should include shared kyo-http tests covering:

• Config filters apply to HttpClient convenience methods.
• Config filters apply to explicit client.sendWith.
• Scoped HttpClient.withFilter applies only inside the block.
• Config and scoped filters compose in the documented order.
• Route filters still compose after config and scoped filters.
• HttpFilter.noop preserves existing behavior and fast path.
• WebSocket handshakes receive headers from SPI, config, and scoped filters.
• WebSocket frame messages are not intercepted by HTTP client filters.

Documentation

The README filter section should distinguish four layers:

1. Route filters: endpoint-specific behavior via HttpRoute.filter.
2. Config filters: reusable client policy via HttpClientConfig.filter.
3. Scoped filters: temporary block-local policy via HttpClient.withFilter.
4. SPI filters: library-level global instrumentation via HttpFilter.Factory.

It should also call out that WebSocket support applies to the HTTP upgrade handshake only, not to messages after the connection is established.