From 755e460b20404fc4ea1b7079fbd9c458ecab3153 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 13 Jun 2025 16:26:11 +1200 Subject: [PATCH 1/7] Authentication API Signed-off-by: Tom Bentley --- proposals/003-authentication-api.md | 223 ++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 proposals/003-authentication-api.md diff --git a/proposals/003-authentication-api.md b/proposals/003-authentication-api.md new file mode 100644 index 0000000..b14cf40 --- /dev/null +++ b/proposals/003-authentication-api.md @@ -0,0 +1,223 @@ + +# Authentication API + +This proposal describes a public API to expose client identity to `Filter` implementations. + +## Current situation + +The `Filter` and `FilterContext` APIs currently don't directly expose any authenticated client identity information. +Specifically: + +* If clients are authenticating to the proxy using mTLS then filters don't have access to a `Subject` or `Principal` corresponding to the client TLS certificate. +* If clients are authenticating using the `SaslAuthenticate` KRPC the only way a `Filter` can know about that is by intercepting those frames. + - For "SASL passthrough", a `Filter` implementation could try to _infer_ the client's identity by watching for a successful `SaslAuthenticateResponse` + returned by the broker. This is a technique that each identity-dependent filter in the chain could use, but each needs to reimplement the logic. + - For "SASL termination", a `Filter` implementation could handle the `SaslAuthenticateRequest` frames itself + - If it makes its own `SaslAuthenticateRequests` (with different credentials, and/or a different mechanism) to the server then subsequent filters in the chain need to use the "SASL passthrough" technique to be aware of the broker-facing identity, and there's no way for them to know the original identity. + - If it doesn't (so that there is no `SaslAuthenticate` intersection with the broker) then subsequent filters in the chain have no way of knowing the client's identity. + +## Motivation + +The lack of this API makes implementing client identity aware plugins difficult, or impossible. + +Goals: + +* Enable `Filters` to access a client's identity using a single, consistent API, irrespective of what authentication mechanism is implemented, TLS or SASL, and whether it's implemented by the proxy runtime (in the TLS case), or a prior Filter in the chain (in the SASL termination case). +* Don't require a `Filter` to handle `SaslAuthenticate` unless it is performing SASL termination. + +Non-goals: + +* Defining an API for exposing the identity of a _broker_ to `Filters` (in cases where the proxy mututally authenticates the broker). + +## Proposal + +### Public API changes + +Add the following interface to the `kroxylicious-api` module to allow a `Filter` implementation to opt into consuming authentication outcomes: + +```java +package io.kroxylicious.proxy.filter; + +import javax.security.auth.Subject; +import javax.security.auth.login.LoginException; + +// This is a new interface in the api module +interface ClientSubjectAwareFilter extends Filter { + // Called when a client authenticates, or reauthenticates + void onClientAuthentication(Subject clientSubject, FilterContext context); + // Called when a client fails authentication, or reauthentication + default void onClientAuthenticationFailure(LoginException exception, FilterContext context) { + } +} +``` + +Add the following two methods to `FilterContext` to allow SASL terminating `Filter` implementations to propagate their authentication outcomes to upstream filters: + +```java +public interface FilterContext { + + void clientAuthenticationSuccess(Subject subject); + + void clientAuthenticationFailure(LoginException e); + +} +``` + +### Runtime changes + +1. Change the `kroxylicious-runtime` module so a `Subject` gets instantiated on initial connection. +2. On successful completion of a TLS handshake, associate a `javax.security.auth.x500.X500Principal` with that `Subject`. +3. On construction of the filter chain fire an internal message down the pipeline. +4. Change the `FilterHandler` and `FilterContext` to supply the subject from such a message to the `Filter` implementation. + +### An example identity-consuming `Filter` + +This section sketches how a Filter implementation would consume client identity: + +```java + public class ExampleIdentityConsumingFilter implements ProduceRequestFilter, ClientSubjectAwareFilter { + private Subject subject; + + public void onClientAuthentication(Subject clientSubject, FilterContext context) { + // Store the subject for use later + this.subject = clientSubject; + } + + @Override + public CompletionStage onProduceRequest(short apiVersion, RequestHeaderData header, ProduceRequestData request, FilterContext context) { + if (subject == null) { + return null; // TODO return an error reponse, e.g. Errors.SASL_AUTHENTICATION_FAILED, or Errors.UNKNOWN_SERVER_ERROR + } + else if (authorized(subject)) { + // TODO do something and forward the request + return null; + } + else { + return null; // TODO return an error reponse: Errors.TOPIC_AUTHORIZATION_FAILED + } + } + + private static boolean authorized(Subject subject) { + // TODO this is not abstracted from the type of Principal + // We really want a model like Kafka's org.apache.kafka.server.authorizer.Authorizer + // See org.apache.kafka.server.authorizer.AuthorizableRequestContext for what gets exposed to Authorizer. + // I guess this is why KafkaPrincipal#principalType exists + // so that authorizers can just query without knowing about disparate types + return subject.getPrincipals().contains(new X500Principal("dn=admin,org=whatever")) + || subject.getPrincipals().contains(new RolePrincipal("admin")); + } + } +``` + +Notes: + +* The choice of `Principal` implementations is left open. In particular a `Filter` could, but doesn't have to, add a `KafkaPrincipal` to the subject. + Likewise a Filter could, but doesn't have to, make use of JDK-defined Principals like `javax.security.auth.x500.X500Principal`, + `javax.security.auth.kerberos.KerberosPrincipal`. It should be noted that `Filters` that add principals and `Filters` that query principals (including making authorization decisions) need to use common principal types. It is therefore recommended that `Filters` use `KafkaPrincipal`. + +### An example SASL Terminating `Filter ` + +This section sketches how a SASL terminating `Filter` might work. + +The proxy config would look like this: + +```yaml +virtualClusters: + - name: my-cluster + filters: + - my_authn_filter +filterDefinitions: + - name: my_authn_filter + type: MyClientAuthnFilter + config: + jaasConfigFile: client-jaas.login.config + jaasContextName: client_auth_stack +``` + +where the `client-jaaas.login.config` file might look like this: + +``` +client_auth_stack { + org.apache.kafka.common.security.plain.PlainLoginModule required + user_alice="pa55word" + user_bob="changeit" + ; +}; +``` + +The implementation would look something like this: + +```java +public class ExampleSaslTerminatingFilter implements SaslAuthenticateRequestFilter, ClientSubjectAwareFilter { + private Subject subject; + + @Override + public void onClientAuthentication(Subject clientSubject, FilterContext context) { + this.subject = clientSubject; + } + + @Override + public CompletionStage onSaslAuthenticateRequest(short apiVersion, + RequestHeaderData header, + SaslAuthenticateRequestData request, + FilterContext context) { + Configuration config = new ConfigFile(URI.create("client-jaas.login.config")); + + // TODO the callback handler to use depends on the context in the jaas config + // so how do we know which handler to instantiate? + // Kafka's SaslChannelBuilder basically does its own parsing of the jaas configuration to figure it out. + CallbackHandler callbackHandler = new PlainServerCallbackHandler(); + + try { + LoginContext loginContext = new LoginContext("client_auth_stack", subject, callbackHandler, config); + loginContext.login(); + // here we propagate the subject along the pipeline + // using the new clientAuthentication() method which + // broadcasts the subject to all plugins in the upstream direction + context.clientAuthenticationSuccess(loginContext.getSubject()); + return null; // TODO return a success response to the client + } + catch (LoginException e) { + // here we propagate the failure along the pipeline + // using the new clientAuthenticationFailure() method which + // broadcasts some represnetation of the error + // to all plugins in the upstream direction + context.clientAuthenticationFailure(e); + return null; // TODO return an error response to the client + } + } +} +``` + +Notes: + +* by implementing `ClientSubjectAwareFilter` and reusing the `clientSubject`, it's possible to add a username principal to any existing `X500Principal` added to the object by the runtime. +* `org.apache.kafka.common.security.plain.PlainLoginModule` (and probably other Kafka `LoginModule` implementations) don't work with how JASS `LoginModules` were originally designed to support stackable modules. For example it adds the `username` as a _credential_ of the `Subject`, rather than adding it as _principal_, and `login` is hard coded to return `true`. + + +### Design choices + +* Use JAAS because: + - the Kafka broker and clients already use JAAS + - we'd rather avoid adding a dependency on those not-publicly-supportewd Kafka classes in kroxylicious-api and kroxylicious-runtime, but + - we recognise that 3rd part Filter authors might want to make that choice + - we have no appetite to build-out our own API, nor to pick up a dependency on a 3rd party framework +* Use `Subject` to convey identity information, rather than `Principal` because + - this is more sympathetic with the conceptpaul model of JAAS. + - it allows attaching multiple `Principals` to client `Subjects`, and to be consumed by `ClientSubjectAwareFilter` (e.g. know the client TLS certificate DN _and_ the SASL SCRAM-SHA username). + +## Affected/not affected projects + +The `kroxylicous` repo. + +## Compatibility + +This change would be backwards compatible for Filter developers and proxy users (i.e. all existing proxy configurations files would still be valid). + +## Rejected alternatives + +* Why not just add `public Principal clientPrincipal();` to `FilterContext`? + - It doesn't support multiple principals. +* OK, so why not just add `public Subject clientSubject();` to `FilterContext`? + - It makes the `Subject` a property of the connection/Netty channel. The public API presented here allows a `Filter` to change the `Principals` presented to subsequent `Filters` in the chain, enabling Filters to implement use cases like impersonation. + From 7990532b554a6b047c1f7a00b6f2fc88c5d7a676 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 4 Jul 2025 15:26:55 +1200 Subject: [PATCH 2/7] More APIs, lots of them! Signed-off-by: Tom Bentley --- proposals/003-authentication-api.md | 413 +++++++++++++++++++++++----- 1 file changed, 347 insertions(+), 66 deletions(-) diff --git a/proposals/003-authentication-api.md b/proposals/003-authentication-api.md index b14cf40..d5051cd 100644 --- a/proposals/003-authentication-api.md +++ b/proposals/003-authentication-api.md @@ -1,123 +1,220 @@ -# Authentication API +# Authentication APIs -This proposal describes a public API to expose client identity to `Filter` implementations. +This proposal describes a set of public APIs to expose client identity to plugin implementations. + +## Terminology + +Let's define some terminology to make discussion easier. For the purposes of this proposal we consider TLS and SASL as the only authentication mechanisms we care about. + +Let's first define terms for TLS: + +* A **mutual authentication mechanism** is one which proves the identity of a server to a client. +* When the proxy is configured to accept TLS connections from clients it is performing **TLS Termination**, which does not imply mutual authentication. +* A **TLS client certificate** is a TLS certificate for the client-side of a TLS Handshake. For a given client and server pairing a proxy might have _two_ of these the Kafka client's TLS client certificate, and the proxy's own TLS client certificate for its connection to the server. +* When the proxy configured to require TLS client certificates from clients and validates these against a set of trusted signing certificates (CA certificate) it is performing **client mutual TLS authentication** ("client mTLS"). +* A **TLS server certificate** is a TLS certificate for the server-side of a TLS Handshake. As above, there could be two of these for a given connection through a proxy. +* When the proxy is configured to use a TLS client certificate when making a TLS connection to a server, we will use the term **server mutual TLS authentication_** ("server mTLS"). + +Now let's talk about SASL. In the following the word "component" is generalising over filters, other plugins, and the proxy as a whole: + +* a component which forwards a client's `SaslAuthenticate` requests to the server, and conveys the responses back to the client, is performing **SASL Passthrough**. +* SASL Passthrough is one way to for a proxy to be **identity preserving**, which means that a proxy's principal is the same as the broker's principal for all clients connecting through that proxy. +* a component performing SASL Passthrough and looking at the requests and responses to infer the client's principal is performing **SASL Passthrough Sniffing**. Note that this technique does not work with all SASL mechanisms. +* a component that responds to a client's `SaslAuthenticate` requests _itself_, without forwarding those requests to the server, is performing **SASL Termination**. +* a component that injects its own `SaslAuthenticate` requests into a SASL exchange with the server, is performing **SASL Initiation**. + +When _all_ the filters/plugins on the path between client and server a performing "SASL passthrough" then the proxy as a whole is performing "SASL passthrough". Alternatively, if any filters/plugins on the path between client and server is performing "SASL Termination", then we might say that the proxy as a whole is performing "SASL Termination". + +It is possible for a proxy to be perform neither, one, or both, of SASL Termination and SASL Initiation. + +Finally, let's define some ideas that from JAAS: + +* A **subject** represents a participant in the protcol (a client or server). +* A **principal** identifies a subject. A subject may have zero or more principals. +Subjects that haven't authenticated will have no principals. +A subject gains a principal following a successful TLS handshake. +A a subject also gains a principal following a successful `SaslAuthenticate` exchange. +* A **credential** is information used to prove the authenticity of a principal. +* A **public credential**, such as a TLS certificate, need not be kept a secret. +* A **private credential**, such as a TLS private key or a password, must be kept secret, otherwise the authenticity of a principal is compromised. ## Current situation The `Filter` and `FilterContext` APIs currently don't directly expose any authenticated client identity information. Specifically: -* If clients are authenticating to the proxy using mTLS then filters don't have access to a `Subject` or `Principal` corresponding to the client TLS certificate. -* If clients are authenticating using the `SaslAuthenticate` KRPC the only way a `Filter` can know about that is by intercepting those frames. - - For "SASL passthrough", a `Filter` implementation could try to _infer_ the client's identity by watching for a successful `SaslAuthenticateResponse` - returned by the broker. This is a technique that each identity-dependent filter in the chain could use, but each needs to reimplement the logic. - - For "SASL termination", a `Filter` implementation could handle the `SaslAuthenticateRequest` frames itself - - If it makes its own `SaslAuthenticateRequests` (with different credentials, and/or a different mechanism) to the server then subsequent filters in the chain need to use the "SASL passthrough" technique to be aware of the broker-facing identity, and there's no way for them to know the original identity. - - If it doesn't (so that there is no `SaslAuthenticate` intersection with the broker) then subsequent filters in the chain have no way of knowing the client's identity. +* If proxy uses client mTLS, then filters don't have access to a `Subject` or `Principal` corresponding to the client's TLS client certificate. +* If clients are authenticating using SASL the only way a `Filter` can know about that is by intercepting those frames. + - identity-using filters in the chain must _each_ implement SASL passthrough sniffing. + - but this is usually incompatible with use of a filter performing SASL Termination or SASL Initiation. ## Motivation -The lack of this API makes implementing client identity aware plugins difficult, or impossible. +The lack of API support makes implementing client identity aware plugins difficult, or impossible. Goals: -* Enable `Filters` to access a client's identity using a single, consistent API, irrespective of what authentication mechanism is implemented, TLS or SASL, and whether it's implemented by the proxy runtime (in the TLS case), or a prior Filter in the chain (in the SASL termination case). -* Don't require a `Filter` to handle `SaslAuthenticate` unless it is performing SASL termination. +* Allow the possibility for new KRPC intercepting plugins in the future by not assuming that `Filters` are the kind of KRPC intercepting plugin. We'll use the term **plugin**, unless saying something specifically about `Filters`. +* Enable plugins to access a client's identity using a single, consistent API, irrespective of which authentication mechanism(s) are being used, TLS or SASL, and whether they're implemented by the proxy runtime (in the TLS case), or a prior plugin in the chain (in the SASL termination case). +* Don't require a plugin to handle `SaslAuthenticate` unless it is performing SASL termination or initiation. +* Provide a flexible API to make serving niche use cases possible (though perhaps not simple). Non-goals: -* Defining an API for exposing the identity of a _broker_ to `Filters` (in cases where the proxy mututally authenticates the broker). +* Defining an API for exposing the identity of a _broker_ to plugins (in cases where the proxy mututally authenticates the broker). ## Proposal -### Public API changes +### Proposed API for learning about client authentication outcomes + +Plugin implementations require an API through which to learn about client authentication outcomes. -Add the following interface to the `kroxylicious-api` module to allow a `Filter` implementation to opt into consuming authentication outcomes: +For this purpose we will add the following new interface to the new package `io.kroxylicious.proxy.authentication` in the `kroxylicious-api` module: ```java -package io.kroxylicious.proxy.filter; +package io.kroxylicious.proxy.authentication; import javax.security.auth.Subject; import javax.security.auth.login.LoginException; -// This is a new interface in the api module -interface ClientSubjectAwareFilter extends Filter { +// Allows a plugin to opt-in to being away of client-facing authentication outcomes +interface ClientSubjectAware { + // Called when a client authenticates, or reauthenticates - void onClientAuthentication(Subject clientSubject, FilterContext context); + void onClientAuthentication(Subject clientSubject, ClientAuthenticationContext context); + // Called when a client fails authentication, or reauthentication - default void onClientAuthenticationFailure(LoginException exception, FilterContext context) { + default void onClientAuthenticationFailure(LoginException exception, ClientAuthenticationContext context) { } } ``` -Add the following two methods to `FilterContext` to allow SASL terminating `Filter` implementations to propagate their authentication outcomes to upstream filters: +This interface may be implemented by `Filters` to learn about client authentication outcomes. +In the case of client mTLS the runtime will populate the `Subject` with a `X500Principal` corresponding to the TLS client certificate. -```java -public interface FilterContext { +The `ClientAuthenticationContext` is implemented by the runtime and may be used by the `ClientSubjectAware` implementation to query information available at that point in time. - void clientAuthenticationSuccess(Subject subject); - - void clientAuthenticationFailure(LoginException e); - +```java +// TODO do we really need this? +// In particular, if a pluin implements ClientSubjectAware then it's in a position to mutate the `clientSubject`, +// which will be visible to later plugins +// If the runtime cloned the subject clientAuthenticationSuccessbefore each invocation then each plugin would have its own view +package io.kroxylicious.proxy.authentication; + +public interface ClientAuthenticationContext { + + /** + * Forward a successful authentication outcome towards + * the broker, invoking upstream filters. + * @param subject The subject + */ + void forwardClientAuthenticationSuccess(Subject clientSubject); + + /** + * Forward a failed authentication outcome towards + * the broker, invoking upstream filters. + * @param e The exception + */ + void forwardClientAuthenticationFailure(LoginException e); } ``` -### Runtime changes - -1. Change the `kroxylicious-runtime` module so a `Subject` gets instantiated on initial connection. -2. On successful completion of a TLS handshake, associate a `javax.security.auth.x500.X500Principal` with that `Subject`. -3. On construction of the filter chain fire an internal message down the pipeline. -4. Change the `FilterHandler` and `FilterContext` to supply the subject from such a message to the `Filter` implementation. -### An example identity-consuming `Filter` +### Example: An identity-consuming `Filter` -This section sketches how a Filter implementation would consume client identity: +This shows a filter implementing `ClientSubjectAware` and using the `clientSubject` to drive an authorization decision. ```java - public class ExampleIdentityConsumingFilter implements ProduceRequestFilter, ClientSubjectAwareFilter { - private Subject subject; + public class ExampleIdentityConsumingFilter implements ProduceRequestFilter, ClientSubjectAware { + + private Subject clientSubject; - public void onClientAuthentication(Subject clientSubject, FilterContext context) { + public void onClientAuthentication(Subject clientSubject, ClientAuthenticationContext context) { // Store the subject for use later - this.subject = clientSubject; + this.clientSubject = clientSubject; } @Override public CompletionStage onProduceRequest(short apiVersion, RequestHeaderData header, ProduceRequestData request, FilterContext context) { - if (subject == null) { - return null; // TODO return an error reponse, e.g. Errors.SASL_AUTHENTICATION_FAILED, or Errors.UNKNOWN_SERVER_ERROR + if (clientSubject == null) { + return ...; // return an error reponse, e.g. Errors.SASL_AUTHENTICATION_FAILED, or Errors.UNKNOWN_SERVER_ERROR } - else if (authorized(subject)) { - // TODO do something and forward the request - return null; + else if (authorized(clientSubject)) { + return ...; // do something and forward the request } else { - return null; // TODO return an error reponse: Errors.TOPIC_AUTHORIZATION_FAILED + return ...; // return an error reponse: Errors.TOPIC_AUTHORIZATION_FAILED } } - private static boolean authorized(Subject subject) { + private static boolean authorized(Subject clientSubject) { // TODO this is not abstracted from the type of Principal // We really want a model like Kafka's org.apache.kafka.server.authorizer.Authorizer // See org.apache.kafka.server.authorizer.AuthorizableRequestContext for what gets exposed to Authorizer. // I guess this is why KafkaPrincipal#principalType exists // so that authorizers can just query without knowing about disparate types - return subject.getPrincipals().contains(new X500Principal("dn=admin,org=whatever")) - || subject.getPrincipals().contains(new RolePrincipal("admin")); + return clientSubject.getPrincipals().contains(new X500Principal("dn=admin,org=whatever")); } } ``` -Notes: -* The choice of `Principal` implementations is left open. In particular a `Filter` could, but doesn't have to, add a `KafkaPrincipal` to the subject. - Likewise a Filter could, but doesn't have to, make use of JDK-defined Principals like `javax.security.auth.x500.X500Principal`, - `javax.security.auth.kerberos.KerberosPrincipal`. It should be noted that `Filters` that add principals and `Filters` that query principals (including making authorization decisions) need to use common principal types. It is therefore recommended that `Filters` use `KafkaPrincipal`. +Note a common authorization API is not in scope of this proposal. + +The choice of `Principal` implementations is left open. In particular a plugin could, but doesn't have to, add a `KafkaPrincipal` to the subject. +Likewise a plugin could, but doesn't have to, make use of JDK-defined Principals like `javax.security.auth.x500.X500Principal`, or + `javax.security.auth.kerberos.KerberosPrincipal`. +It should be noted that plugins that add principals and plugins that query principals (including making authorization decisions) need to use common principal types. +It is therefore recommended that plugins use `KafkaPrincipal`. **TODO: Really? Why not just define our own ProxyPrincipal(type, name) and be done with it?** + +Audit logging clients is another use case for this API, in addition to this authorization example. + -### An example SASL Terminating `Filter ` +### Proposed API for announcing client authentication outcomes -This section sketches how a SASL terminating `Filter` might work. +The existing `SaslAuthenticateRequestFilter` and `SaslAuthenticateResponseFilter` continue to provide the mechanism for protocol-level +interaction with clients and server. +However, such SASL terminating plugins require an API through which to anounce their authentication outcomes to filters. + +For this purpose we will add the following two methods to the existing `FilterContext` interface in the `io.kroxylicious.proxy.filter` package. + +```java +package io.kroxylicious.proxy.filter; + +public interface FilterContext { + + // ... existing methods ... + + + /** + * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) + * to announce a successful authentication outcome to subsequent plugins. + * @param subject The authenticated subject. + */ + void clientAuthenticationSuccess(Subject subject); + + /** + * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) + * to announce a failed authentication outcome to subsequent plugins. + * @param exception An exception describing the authentication failure. + */ + void clientAuthenticationFailure(LoginException exception); + +} +``` + +It's worth noting that `LoginException` has a number of subclasses in `javax.security.auth.login`. + +This API allows SASL-terminating plugin to announce its authentication outcomes to later filters in the filter chain which have implemented `ClientSubjectAware`. +Note that client authentication outcomes only propagate towards the server, not back towards the client. +So a `ClientSubjectAware` before a SASL-terminating plugin will not receive the announcement. +**TODO justify this** + + +### Example: A SASL Terminating `Filter` + +This shows how a SASL terminating `Filter` would use the new methods on `FilterContext` to inform other filters, such as `ExampleIdentityConsumingFilter` above, about the client. The proxy config would look like this: @@ -126,12 +223,16 @@ virtualClusters: - name: my-cluster filters: - my_authn_filter + - my_authz_filter filterDefinitions: - name: my_authn_filter - type: MyClientAuthnFilter + type: ExampleSaslTerminatingFilter config: jaasConfigFile: client-jaas.login.config jaasContextName: client_auth_stack + - name: my_authz_filter + type: ExampleIdentityConsumingFilter + config: ... ``` where the `client-jaaas.login.config` file might look like this: @@ -148,11 +249,13 @@ client_auth_stack { The implementation would look something like this: ```java -public class ExampleSaslTerminatingFilter implements SaslAuthenticateRequestFilter, ClientSubjectAwareFilter { +public class ExampleSaslTerminatingFilter implements SaslAuthenticateRequestFilter, ClientSubjectAware { + private Subject subject; @Override public void onClientAuthentication(Subject clientSubject, FilterContext context) { + // the clientSubject will have an X500Principal iff the client used mTLS. this.subject = clientSubject; } @@ -175,7 +278,7 @@ public class ExampleSaslTerminatingFilter implements SaslAuthenticateRequestFilt // using the new clientAuthentication() method which // broadcasts the subject to all plugins in the upstream direction context.clientAuthenticationSuccess(loginContext.getSubject()); - return null; // TODO return a success response to the client + return ...; // return a success response to the client } catch (LoginException e) { // here we propagate the failure along the pipeline @@ -183,28 +286,202 @@ public class ExampleSaslTerminatingFilter implements SaslAuthenticateRequestFilt // broadcasts some represnetation of the error // to all plugins in the upstream direction context.clientAuthenticationFailure(e); - return null; // TODO return an error response to the client + return ...; // return an error response to the client } } } ``` -Notes: +By implementing `ClientSubjectAware` and reusing the `clientSubject`, we're adding a principal to any existing `X500Principal` of the subject. + + +### Proposed API for selecting TLS client certificates for server connections + +Initiating a connection to a broker is currently entirely the responsilibity of the runtime, using the `NetFilter` interface. +(note that `NetFilter` is **not** part of the `kroxylicious-api` module.) +This means that the TLS client certificate used is currently always the same, and cannot depend on the client subject. +Adding this new API will allow TLS client certificates for conections to servers to depend on client identity (as learned about using `ClientSubjectAware`). + +```java +package io.kroxylicious.proxy.authentication; + +interface ServerTlsClientCertificateSupplier { + + TlsCredentials tlsCredentials(ServerCredentialContext context); + +} +``` + +where + +```java +package io.kroxylicious.proxy.authentication; + +interface TlsCredentials { + /* Intentionally empty: implemented and accessed only in the runtime */ +} + +interface ServerCredentialContext { + /** The default key for this target cluster (e.g. from the proxy configuration file). */ + TlsCredentials defaultTlsCredentials(); + /** Factory for TlsCredentials */ + TlsCredentials tlsCredentials(Certificate certificate, PrivateKey key, Certificate[] intermediateCertificates); +} +``` + +and adding + +```java + +public interface FilterFactoryContext { + + // ... existing methods ... + + TlsCredentials tlsCredentials(Certificate certificate, PrivateKey key, Certificate[] intermediateCertificates); +} +``` + +It is not proposed to allow dynamic selection of a set of trust anchors. Those should remain under the control of the person configuring the proxy. + + +#### An Example: TLS-to-TLS identity mapping + +This shows how a plugin could choose a TLS client certificate for the broker connection based on the connected Kafka client's TLS identity. + +```java + +class ExampleMTlsFilter implements ClientSubjectAware, ServerCredentialSupplier { + + private Map certs; + + ExampleMTlsFilter(Map certs) { + this.certs = certs; + } + + private Subject clientSubject; + + public void onClientAuthentication(Subject clientSubject, ClientAuthenticationContext context) { + // Store the subject for use later + this.clientSubject = clientSubject; + } + + TlsCredentials certificate(ServerCredentialContext context) { + if (this.clientSubject == null) { + throw new IllegalStateException(); + } + return certs.getOrDefault( + this.clientSubject.getPrincipal(X500Principal.class), + context.defaultTlsCredentials()); + } +} + +class ExampleMTlsFilterFactory implements FilterFactory<, Map> { + Map certs; + public Map initialize(FilterFactoryContext context, C config) { + certs = context.tlsCredentials(...) + } + + ExampleMTlsFilter createFilter(FilterFactoryContext context, I initializationData) { + return new ExampleMTlsFilter(certs) + } + +``` + +An almost identical class could be used with the `ExampleSaslTerminatingFilter` from the previous section for SASL-to-TLS identity mapping. + +### Proposed API for learning about server authentication outcomes -* by implementing `ClientSubjectAwareFilter` and reusing the `clientSubject`, it's possible to add a username principal to any existing `X500Principal` added to the object by the runtime. -* `org.apache.kafka.common.security.plain.PlainLoginModule` (and probably other Kafka `LoginModule` implementations) don't work with how JASS `LoginModules` were originally designed to support stackable modules. For example it adds the `username` as a _credential_ of the `Subject`, rather than adding it as _principal_, and `login` is hard coded to return `true`. +This is the mirror image of `ClientSubjectAware`, but for cases where the broker is using mTLS and/or a SASL mechanism that supports mutual authentication. +It allows plugin behaviour to depend on the server's identity. +```java +package io.kroxylicious.proxy.authentication; + +import javax.security.auth.Subject; +import javax.security.auth.login.LoginException; + +// Allows a plugin to opt-in to being away of client-facing authentication outcomes +interface ServerSubjectAware { + + // Called when a client authenticates, or reauthenticates + void onServerAuthentication(Subject serverSubject, ServerAuthenticationContext context); + + // Called when a client fails authentication, or reauthentication + default void onServerAuthenticationFailure(LoginException exception, ServerAuthenticationContext context) { + } +} +``` + +### Proposed API for initiating a SASL exchange with the broker + +There are two mutually exclusive cases to consider for SASL initiation: + +1. That a SASL initiator plugin should perform SASL authentication _eagerly_, as soon as the underlying connection is ready. +This would be driven by the runtime following a successful TCP handshake on the client-to-proxy connection, and after `ClientSubjectAware` plugins have initially been called with any `X500Principal`, but before any SASL exchange on the client side. +2. That the SASL initiator plugin should perform SASL authentication _lazily_, as soon as a KRPC requests propagates along the chain to the initator plugin. +In this case the `ClientSubjectAware` plugins may have been called with a SASL principal. + +Supporting the former case would allow the broker's principal (as obtained by a `ServerSubjectAware` implementing plugin) to be used in the client-facing SASL exchange. This could be relevant for Kafka clients which validated or made use of the authenticate server (in this case proxy) principal. However, we're noy aware of any clients that actually do this, so we won't consider it further. + +Supporting the latter case allows the proxy's server-faving SASL credentials to depend on the client-facing principal. +Because of variation in behaviour of client libraries there is not a single type of request which will always be the first. +However,the existing `Sasl(Handshake|Authenticate)(Request|Response)Filter` interfaces can be used for the KRPC level implementation. +Such a filter implementation needs to keep some kind of `seenFirstRequest` state if it wants to use `FilterContent.sendRequest()` to insert its own initial requests prior to forwarding requests from the client. + +What's missing is a way for the initiator plugin to inform other plugins of the outcome. For this purpose we will add the following two methods to the existing `FilterContext` interface in the `io.kroxylicious.proxy.filter` package. + +```java +package io.kroxylicious.proxy.filter; + +public interface FilterContext { + + // ... existing methods ... + + + /** + * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) + * to announce a successful authentication outcome to subsequent plugins. + * @param subject The authenticated subject. + */ + void serverAuthenticationSuccess(Subject subject); + + /** + * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) + * to announce a failed authentication outcome to subsequent plugins. + * @param exception An exception describing the authentication failure. + */ + void serverAuthenticationFailure(LoginException exception); + +} +``` + +These are the mirror image of the equivalent client methods, and calling them would similarly result in `ServerSubjectAware`-implememting plugins getting informed of the server's subject. + + +### Use cases + +* SASL-to-SASL identity mapping (in combination with a SASL terminator) +* SASL-to-SASL identity preservation with in-proxy authorization (in combination with a SASL terminator, and an authorizer). +* Audit logging + + +### Implementation + +The preceeding section are intented to lay out a comprehensive API covering a variety of authentication use cases. +This is to encourage review of the proposal that considers all the possible use cases. +However, there is no requirement for them to be implemented in one go/within one release cycle. +Some of the proposed changes will require non-trival changes in the proxy runtime. ### Design choices * Use JAAS because: - the Kafka broker and clients already use JAAS - - we'd rather avoid adding a dependency on those not-publicly-supportewd Kafka classes in kroxylicious-api and kroxylicious-runtime, but - - we recognise that 3rd part Filter authors might want to make that choice + - we'd rather avoid adding a dependency on those not-publicly-supported Kafka classes in `kroxylicious-api` and `kroxylicious-runtime`, + - but we recognise that 3rd part Filter authors might want to make that choice - we have no appetite to build-out our own API, nor to pick up a dependency on a 3rd party framework * Use `Subject` to convey identity information, rather than `Principal` because - - this is more sympathetic with the conceptpaul model of JAAS. - - it allows attaching multiple `Principals` to client `Subjects`, and to be consumed by `ClientSubjectAwareFilter` (e.g. know the client TLS certificate DN _and_ the SASL SCRAM-SHA username). + - this is more sympathetic with the conceptual model of JAAS. + - it allows attaching multiple `Principals` to client `Subjects`, and to be consumed by `ClientSubjectAware` instances (e.g. know the client TLS certificate DN _and_ the SASL SCRAM-SHA username). ## Affected/not affected projects @@ -214,10 +491,14 @@ The `kroxylicous` repo. This change would be backwards compatible for Filter developers and proxy users (i.e. all existing proxy configurations files would still be valid). +## Future work + +This proposal in combination with the proposal on a routing API, would enable client-subject based routing to backing clusters. + ## Rejected alternatives * Why not just add `public Principal clientPrincipal();` to `FilterContext`? - It doesn't support multiple principals. * OK, so why not just add `public Subject clientSubject();` to `FilterContext`? - - It makes the `Subject` a property of the connection/Netty channel. The public API presented here allows a `Filter` to change the `Principals` presented to subsequent `Filters` in the chain, enabling Filters to implement use cases like impersonation. + - It makes the `Subject` a property of the connection/Netty channel. The public API presented here allows a `Filter` to change the `Principals` presented to subsequent `Filters` in the chain, enabling `Filters` to implement a wider variety of use cases. From 5a3e5018c1f7bc5dbcb0b98d46371a82b2b76a12 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 4 Jul 2025 16:28:03 +1200 Subject: [PATCH 3/7] Clarify the API wrt mutual authentication mechanisms Signed-off-by: Tom Bentley --- proposals/003-authentication-api.md | 56 +++++++++++++++++------------ 1 file changed, 34 insertions(+), 22 deletions(-) diff --git a/proposals/003-authentication-api.md b/proposals/003-authentication-api.md index d5051cd..ac93962 100644 --- a/proposals/003-authentication-api.md +++ b/proposals/003-authentication-api.md @@ -78,13 +78,17 @@ package io.kroxylicious.proxy.authentication; import javax.security.auth.Subject; import javax.security.auth.login.LoginException; -// Allows a plugin to opt-in to being away of client-facing authentication outcomes +// Allows a plugin to opt-in to being aware of client-facing authentication outcomes interface ClientSubjectAware { - // Called when a client authenticates, or reauthenticates + /** + * Called when a client authenticates, or reauthenticates, with the proxy. + */ void onClientAuthentication(Subject clientSubject, ClientAuthenticationContext context); - // Called when a client fails authentication, or reauthentication + /** + * Called when a client fails authentication, or reauthentication, with the proxy. + */ default void onClientAuthenticationFailure(LoginException exception, ClientAuthenticationContext context) { } } @@ -96,27 +100,16 @@ In the case of client mTLS the runtime will populate the `Subject` with a `X500P The `ClientAuthenticationContext` is implemented by the runtime and may be used by the `ClientSubjectAware` implementation to query information available at that point in time. ```java -// TODO do we really need this? -// In particular, if a pluin implements ClientSubjectAware then it's in a position to mutate the `clientSubject`, -// which will be visible to later plugins -// If the runtime cloned the subject clientAuthenticationSuccessbefore each invocation then each plugin would have its own view package io.kroxylicious.proxy.authentication; public interface ClientAuthenticationContext { - /** - * Forward a successful authentication outcome towards - * the broker, invoking upstream filters. - * @param subject The subject - */ - void forwardClientAuthenticationSuccess(Subject clientSubject); - - /** - * Forward a failed authentication outcome towards - * the broker, invoking upstream filters. - * @param e The exception + /** + * The subject that the proxy presented to the client. + * This may be null of the authentication mechanism does not support + * mutual authentication. */ - void forwardClientAuthenticationFailure(LoginException e); + Subject proxySubject(); } ``` @@ -400,18 +393,37 @@ package io.kroxylicious.proxy.authentication; import javax.security.auth.Subject; import javax.security.auth.login.LoginException; -// Allows a plugin to opt-in to being away of client-facing authentication outcomes +// Allows a plugin to opt in to being aware of broker-facing authentication outcomes interface ServerSubjectAware { - // Called when a client authenticates, or reauthenticates + /** + * Called when the proxy authenticates, or reauthenticates with a server + * The given serverSubject may not have a principal for the server corresponding + * to the authentication mechanism used if that authentication mechanism does not provide + * mutual authentication. + */ void onServerAuthentication(Subject serverSubject, ServerAuthenticationContext context); - // Called when a client fails authentication, or reauthentication + /** + * Called when the proxy fails authentication, or reauthentication, with a server + */ default void onServerAuthenticationFailure(LoginException exception, ServerAuthenticationContext context) { } } ``` +```java +package io.kroxylicious.proxy.authentication; + +public interface ServerAuthenticationContext { + + /** + * The subject that the proxy presented to the server. + */ + Subject proxySubject(); +} +``` + ### Proposed API for initiating a SASL exchange with the broker There are two mutually exclusive cases to consider for SASL initiation: From 180893fb0a68b5a54b03ae3f170df3ae8c05e745 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 7 Jul 2025 15:03:56 +1200 Subject: [PATCH 4/7] Address come of Keith's comments Signed-off-by: Tom Bentley --- proposals/003-authentication-api.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/proposals/003-authentication-api.md b/proposals/003-authentication-api.md index ac93962..ab913b2 100644 --- a/proposals/003-authentication-api.md +++ b/proposals/003-authentication-api.md @@ -45,7 +45,7 @@ The `Filter` and `FilterContext` APIs currently don't directly expose any authen Specifically: * If proxy uses client mTLS, then filters don't have access to a `Subject` or `Principal` corresponding to the client's TLS client certificate. -* If clients are authenticating using SASL the only way a `Filter` can know about that is by intercepting those frames. +* If clients are authenticating using SASL, the only way a `Filter` can know about that is by intercepting those frames. - identity-using filters in the chain must _each_ implement SASL passthrough sniffing. - but this is usually incompatible with use of a filter performing SASL Termination or SASL Initiation. @@ -59,6 +59,7 @@ Goals: * Enable plugins to access a client's identity using a single, consistent API, irrespective of which authentication mechanism(s) are being used, TLS or SASL, and whether they're implemented by the proxy runtime (in the TLS case), or a prior plugin in the chain (in the SASL termination case). * Don't require a plugin to handle `SaslAuthenticate` unless it is performing SASL termination or initiation. * Provide a flexible API to make serving niche use cases possible (though perhaps not simple). +* Drop support for the "raw" (i.e. not encapsulated within the Kafka protocol) support for SASL, as [Kafka itself has does from Kafka 4.0](https://cwiki.apache.org/confluence/display/KAFKA/KIP-896%3A+Remove+old+client+protocol+API+versions+in+Kafka+4.0) Non-goals: @@ -265,8 +266,12 @@ public class ExampleSaslTerminatingFilter implements SaslAuthenticateRequestFilt CallbackHandler callbackHandler = new PlainServerCallbackHandler(); try { + // Note: In general these methods are not guaranteed to be non-blocking, so + // use of ThreadPoolExecutor is recommended unless an implementor knows + // from other means that blocking is impossible. LoginContext loginContext = new LoginContext("client_auth_stack", subject, callbackHandler, config); loginContext.login(); + // here we propagate the subject along the pipeline // using the new clientAuthentication() method which // broadcasts the subject to all plugins in the upstream direction @@ -322,7 +327,7 @@ interface ServerCredentialContext { } ``` -and adding +and adding the following to allow a filter factory to generate `TlsCredentials` at initialization time so as to take work off the hotter path on which `ServerTlsClientCertificateSupplier.tlsCredentials()` is invoked: ```java @@ -428,14 +433,14 @@ public interface ServerAuthenticationContext { There are two mutually exclusive cases to consider for SASL initiation: -1. That a SASL initiator plugin should perform SASL authentication _eagerly_, as soon as the underlying connection is ready. +1. That a SASL initiator plugin should perform SASL authentication _eagerly_, as soon as the underlying connection to the server is ready. This would be driven by the runtime following a successful TCP handshake on the client-to-proxy connection, and after `ClientSubjectAware` plugins have initially been called with any `X500Principal`, but before any SASL exchange on the client side. 2. That the SASL initiator plugin should perform SASL authentication _lazily_, as soon as a KRPC requests propagates along the chain to the initator plugin. In this case the `ClientSubjectAware` plugins may have been called with a SASL principal. -Supporting the former case would allow the broker's principal (as obtained by a `ServerSubjectAware` implementing plugin) to be used in the client-facing SASL exchange. This could be relevant for Kafka clients which validated or made use of the authenticate server (in this case proxy) principal. However, we're noy aware of any clients that actually do this, so we won't consider it further. +Supporting the former case would allow the broker's principal (as obtained by a `ServerSubjectAware` implementing plugin) to be used in the client-facing SASL exchange. This could be relevant for Kafka clients which validated or made use of the authenticate server (in this case proxy) principal. However, we're not aware of any clients that actually do this, so we won't consider it further. -Supporting the latter case allows the proxy's server-faving SASL credentials to depend on the client-facing principal. +Supporting the latter case allows the proxy's server-facing SASL credentials to depend on the client-facing principal. Because of variation in behaviour of client libraries there is not a single type of request which will always be the first. However,the existing `Sasl(Handshake|Authenticate)(Request|Response)Filter` interfaces can be used for the KRPC level implementation. Such a filter implementation needs to keep some kind of `seenFirstRequest` state if it wants to use `FilterContent.sendRequest()` to insert its own initial requests prior to forwarding requests from the client. From f9110e1810f9a5fbdd723021dd67dbab9a7f2dda Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Tue, 8 Jul 2025 11:57:00 +1200 Subject: [PATCH 5/7] Rewording confusing bits Signed-off-by: Tom Bentley --- proposals/003-authentication-api.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/003-authentication-api.md b/proposals/003-authentication-api.md index ab913b2..a0b7adb 100644 --- a/proposals/003-authentication-api.md +++ b/proposals/003-authentication-api.md @@ -16,15 +16,15 @@ Let's first define terms for TLS: * A **TLS server certificate** is a TLS certificate for the server-side of a TLS Handshake. As above, there could be two of these for a given connection through a proxy. * When the proxy is configured to use a TLS client certificate when making a TLS connection to a server, we will use the term **server mutual TLS authentication_** ("server mTLS"). -Now let's talk about SASL. In the following the word "component" is generalising over filters, other plugins, and the proxy as a whole: +Now let's talk about SASL. In the following, the word "component" is generalising over filters, other plugins, and a proxy virtual cluster as a whole: * a component which forwards a client's `SaslAuthenticate` requests to the server, and conveys the responses back to the client, is performing **SASL Passthrough**. -* SASL Passthrough is one way to for a proxy to be **identity preserving**, which means that a proxy's principal is the same as the broker's principal for all clients connecting through that proxy. +* SASL Passthrough is one way to for a proxy to be **identity preserving**, which means that, for all client principals in the virtual cluster, each of those principals will have the same name as the corresponding client principal in the broker. * a component performing SASL Passthrough and looking at the requests and responses to infer the client's principal is performing **SASL Passthrough Sniffing**. Note that this technique does not work with all SASL mechanisms. * a component that responds to a client's `SaslAuthenticate` requests _itself_, without forwarding those requests to the server, is performing **SASL Termination**. * a component that injects its own `SaslAuthenticate` requests into a SASL exchange with the server, is performing **SASL Initiation**. -When _all_ the filters/plugins on the path between client and server a performing "SASL passthrough" then the proxy as a whole is performing "SASL passthrough". Alternatively, if any filters/plugins on the path between client and server is performing "SASL Termination", then we might say that the proxy as a whole is performing "SASL Termination". +When _all_ the filters/plugins on the path between client and server a performing "SASL passthrough" then the virtual cluster as a whole is performing "SASL passthrough". Alternatively, if any filters/plugins on the path between client and server is performing "SASL Termination", then we might say that the virtual cluster as a whole is performing "SASL Termination". It is possible for a proxy to be perform neither, one, or both, of SASL Termination and SASL Initiation. @@ -339,12 +339,13 @@ public interface FilterFactoryContext { } ``` -It is not proposed to allow dynamic selection of a set of trust anchors. Those should remain under the control of the person configuring the proxy. +It is not proposed to allow dynamic selection of a set of trust anchors. +Those should remain under the control of the person configuring the proxy. #### An Example: TLS-to-TLS identity mapping -This shows how a plugin could choose a TLS client certificate for the broker connection based on the connected Kafka client's TLS identity. +This shows how a plugin could choose a TLS client certificate for the broker connection, based on the connected Kafka client's TLS identity. ```java From ac92938ea451842ae8d059cf1b3fe425ee1dee54 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Tue, 15 Jul 2025 17:38:19 +1200 Subject: [PATCH 6/7] Rewrite based on partial implementation Signed-off-by: Tom Bentley --- proposals/003-authentication-api.md | 796 ++++++++++++++++------------ 1 file changed, 462 insertions(+), 334 deletions(-) diff --git a/proposals/003-authentication-api.md b/proposals/003-authentication-api.md index a0b7adb..472b5fe 100644 --- a/proposals/003-authentication-api.md +++ b/proposals/003-authentication-api.md @@ -5,36 +5,43 @@ This proposal describes a set of public APIs to expose client identity to plugin ## Terminology -Let's define some terminology to make discussion easier. For the purposes of this proposal we consider TLS and SASL as the only authentication mechanisms we care about. +Let's define some terminology to make discussion easier. +For the purposes of this proposal we consider TLS and SASL as the only authentication mechanisms we care about. Let's first define terms for TLS: * A **mutual authentication mechanism** is one which proves the identity of a server to a client. -* When the proxy is configured to accept TLS connections from clients it is performing **TLS Termination**, which does not imply mutual authentication. -* A **TLS client certificate** is a TLS certificate for the client-side of a TLS Handshake. For a given client and server pairing a proxy might have _two_ of these the Kafka client's TLS client certificate, and the proxy's own TLS client certificate for its connection to the server. -* When the proxy configured to require TLS client certificates from clients and validates these against a set of trusted signing certificates (CA certificate) it is performing **client mutual TLS authentication** ("client mTLS"). -* A **TLS server certificate** is a TLS certificate for the server-side of a TLS Handshake. As above, there could be two of these for a given connection through a proxy. +* When the proxy is configured to accept TLS connections from clients it is performing **TLS Termination**, which does not imply mutual authentication. + (For the avoidance of doubt, Kroxylicious does not support TLS passthrough; the alternative to TLS Termination is simply using TCP as the application transport). +* A **TLS client certificate** is a TLS certificate for the client-side of a TLS Handshake. + For a given client and server pairing a proxy might have _two_ of these: the Kafka client's TLS client certificate, and the proxy's own TLS client certificate for its connection to the server. +* When the proxy is configured to require TLS client certificates from clients and validates these against a set of trusted signing certificates (CA certificate) it is performing **client mutual TLS authentication** ("client mTLS"). +* A **TLS server certificate** is a TLS certificate for the server-side of a TLS Handshake. + As above, there could be two of these for a given connection through a proxy. * When the proxy is configured to use a TLS client certificate when making a TLS connection to a server, we will use the term **server mutual TLS authentication_** ("server mTLS"). Now let's talk about SASL. In the following, the word "component" is generalising over filters, other plugins, and a proxy virtual cluster as a whole: * a component which forwards a client's `SaslAuthenticate` requests to the server, and conveys the responses back to the client, is performing **SASL Passthrough**. * SASL Passthrough is one way to for a proxy to be **identity preserving**, which means that, for all client principals in the virtual cluster, each of those principals will have the same name as the corresponding client principal in the broker. -* a component performing SASL Passthrough and looking at the requests and responses to infer the client's principal is performing **SASL Passthrough Sniffing**. Note that this technique does not work with all SASL mechanisms. +* a component performing SASL Passthrough and looking at the requests and responses to infer the client's principal is performing **SASL Passthrough Inspection**. + Note that this technique does not work with all SASL mechanisms. * a component that responds to a client's `SaslAuthenticate` requests _itself_, without forwarding those requests to the server, is performing **SASL Termination**. -* a component that injects its own `SaslAuthenticate` requests into a SASL exchange with the server, is performing **SASL Initiation**. +* a component that injects its own `SaslAuthenticate` requests into a SASL exchange with the server is performing **SASL Initiation**. -When _all_ the filters/plugins on the path between client and server a performing "SASL passthrough" then the virtual cluster as a whole is performing "SASL passthrough". Alternatively, if any filters/plugins on the path between client and server is performing "SASL Termination", then we might say that the virtual cluster as a whole is performing "SASL Termination". +When _all_ the filters/plugins on the path between client and server are performing "SASL passthrough" (or don't intercept SASL messages at all) then the virtual cluster as a whole is performing "SASL passthrough". +Alternatively, if any filters/plugins on the path between client and server is performing "SASL Termination", then we might say that the virtual cluster as a whole is performing "SASL Termination". It is possible for a proxy to be perform neither, one, or both, of SASL Termination and SASL Initiation. -Finally, let's define some ideas that from JAAS: +Finally, let's define some concepts from JAAS: * A **subject** represents a participant in the protcol (a client or server). -* A **principal** identifies a subject. A subject may have zero or more principals. -Subjects that haven't authenticated will have no principals. -A subject gains a principal following a successful TLS handshake. -A a subject also gains a principal following a successful `SaslAuthenticate` exchange. +* A **principal** identifies a subject. + A subject may have zero or more principals. + Subjects that haven't authenticated will have no principals. + A subject gains a principal following a successful TLS handshake. + A subject also gains a principal following a successful `SaslAuthenticate` exchange. * A **credential** is information used to prove the authenticity of a principal. * A **public credential**, such as a TLS certificate, need not be kept a secret. * A **private credential**, such as a TLS private key or a password, must be kept secret, otherwise the authenticity of a principal is compromised. @@ -46,7 +53,7 @@ Specifically: * If proxy uses client mTLS, then filters don't have access to a `Subject` or `Principal` corresponding to the client's TLS client certificate. * If clients are authenticating using SASL, the only way a `Filter` can know about that is by intercepting those frames. - - identity-using filters in the chain must _each_ implement SASL passthrough sniffing. + - identity-using filters in the chain must _each_ implement SASL passthrough inspection. - but this is usually incompatible with use of a filter performing SASL Termination or SASL Initiation. ## Motivation @@ -55,468 +62,589 @@ The lack of API support makes implementing client identity aware plugins difficu Goals: -* Allow the possibility for new KRPC intercepting plugins in the future by not assuming that `Filters` are the kind of KRPC intercepting plugin. We'll use the term **plugin**, unless saying something specifically about `Filters`. +* Allow the possibility for new kinds of KRPC intercepting plugins in the future by not assuming that `Filters` are the only kind of KRPC intercepting plugin. We'll use the term **plugin**, unless saying something specifically about `Filters`. * Enable plugins to access a client's identity using a single, consistent API, irrespective of which authentication mechanism(s) are being used, TLS or SASL, and whether they're implemented by the proxy runtime (in the TLS case), or a prior plugin in the chain (in the SASL termination case). +* Allow access to TLS- or SASL-specific details by plugins should they need them. * Don't require a plugin to handle `SaslAuthenticate` unless it is performing SASL termination or initiation. * Provide a flexible API to make serving niche use cases possible (though perhaps not simple). * Drop support for the "raw" (i.e. not encapsulated within the Kafka protocol) support for SASL, as [Kafka itself has does from Kafka 4.0](https://cwiki.apache.org/confluence/display/KAFKA/KIP-896%3A+Remove+old+client+protocol+API+versions+in+Kafka+4.0) -Non-goals: - -* Defining an API for exposing the identity of a _broker_ to plugins (in cases where the proxy mututally authenticates the broker). - ## Proposal -### Proposed API for learning about client authentication outcomes +### API for Filters to access client TLS information -Plugin implementations require an API through which to learn about client authentication outcomes. +TLS (in contrast to SASL) is handled entirely by the proxy runtime. +By the time a `Filter` is instantiated the proxy already has an established TLS connection. +All that's required is an API for exposing appropriate details to `Filters`. -For this purpose we will add the following new interface to the new package `io.kroxylicious.proxy.authentication` in the `kroxylicious-api` module: +The following method will be added to the existing `FilterContext` interface: ```java -package io.kroxylicious.proxy.authentication; - -import javax.security.auth.Subject; -import javax.security.auth.login.LoginException; - -// Allows a plugin to opt-in to being aware of client-facing authentication outcomes -interface ClientSubjectAware { - /** - * Called when a client authenticates, or reauthenticates, with the proxy. + * @return The TLS context for the client connection, or empty if the client connection is not TLS. */ - void onClientAuthentication(Subject clientSubject, ClientAuthenticationContext context); - - /** - * Called when a client fails authentication, or reauthentication, with the proxy. - */ - default void onClientAuthenticationFailure(LoginException exception, ClientAuthenticationContext context) { - } -} + Optional clientTlsContext(); ``` -This interface may be implemented by `Filters` to learn about client authentication outcomes. -In the case of client mTLS the runtime will populate the `Subject` with a `X500Principal` corresponding to the TLS client certificate. +Where `ClientTlsContext` is a new interface in the new package `io.kroxylicious.proxy.tls`: -The `ClientAuthenticationContext` is implemented by the runtime and may be used by the `ClientSubjectAware` implementation to query information available at that point in time. +``` +package io.kroxylicious.proxy.tls; -```java -package io.kroxylicious.proxy.authentication; +import java.security.cert.X509Certificate; +import java.util.Optional; -public interface ClientAuthenticationContext { +public interface ClientTlsContext { + /** + * @return The TLS server certificate that the proxy presented to the client during TLS handshake. + */ + X509Certificate proxyServerCertificate(); - /** - * The subject that the proxy presented to the client. - * This may be null of the authentication mechanism does not support - * mutual authentication. + /** + * @return the client's certificate, or empty if no TLS client certificate was presented during TLS handshake. */ - Subject proxySubject(); + Optional clientCertificate(); + } ``` +Having a distinct type, `ClientTlsContext`, means we can easily expose the same information to future plugins that are not filters (and thus do not have access to a `FilterContext`). + -### Example: An identity-consuming `Filter` +### APIs for Filters to produce client SASL information -This shows a filter implementing `ClientSubjectAware` and using the `clientSubject` to drive an authorization decision. +SASL (in contrast to TLS) is embedded in the Kafka protocol (the `SaslHandshake` and `SaslAuthentication` messages), and therefore can be handled by `Filter` implementations. +The goals require decoupling the production and consumption of SASL information by plugins. +Let's consider the production side first: SASL terminators and inspectors require a way of announcing the outcome of a SASL authentication. +For this purpose we will add the following methods to the existing `FilterContext` interface: ```java - public class ExampleIdentityConsumingFilter implements ProduceRequestFilter, ClientSubjectAware { - - private Subject clientSubject; - - public void onClientAuthentication(Subject clientSubject, ClientAuthenticationContext context) { - // Store the subject for use later - this.clientSubject = clientSubject; - } - - @Override - public CompletionStage onProduceRequest(short apiVersion, RequestHeaderData header, ProduceRequestData request, FilterContext context) { - if (clientSubject == null) { - return ...; // return an error reponse, e.g. Errors.SASL_AUTHENTICATION_FAILED, or Errors.UNKNOWN_SERVER_ERROR - } - else if (authorized(clientSubject)) { - return ...; // do something and forward the request - } - else { - return ...; // return an error reponse: Errors.TOPIC_AUTHORIZATION_FAILED - } - } - - private static boolean authorized(Subject clientSubject) { - // TODO this is not abstracted from the type of Principal - // We really want a model like Kafka's org.apache.kafka.server.authorizer.Authorizer - // See org.apache.kafka.server.authorizer.AuthorizableRequestContext for what gets exposed to Authorizer. - // I guess this is why KafkaPrincipal#principalType exists - // so that authorizers can just query without knowing about disparate types - return clientSubject.getPrincipals().contains(new X500Principal("dn=admin,org=whatever")); - } - } -``` + /** + * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) + * to announce a successful authentication outcome with the Kafka client to other plugins. + * After calling this method the result of {@link #clientSaslContext()} will + * be non-empty for this and other filters. + * + * In order to support reauthentication, calls to this method and + * {@link #clientSaslAuthenticationFailure(String, String, Exception)} + * may be arbitrarily interleaved during the lifetime of a given filter instance. + * @param mechanism The SASL mechanism used. + * @param authorizationId The authorization ID. + */ + void clientSaslAuthenticationSuccess(String mechanism, String authorizationId); + /** + * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) + * to announce a failed authentication outcome with the Kafka client. + * It is the filter's responsilbity to return the right error response to a client, and/or disconnect. + * + * In order to support reauthentication, calls to this method and + * {@link #clientSaslAuthenticationSuccess(String, String)} + * may be arbitrarily interleaved during the lifetime of a given filter instance. + * @param mechanism The SASL mechanism used, or null if this is not known. + * @param authorizationId The authorization ID, or null if this is not known. + * @param exception An exception describing the authentication failure. + */ + void clientSaslAuthenticationFailure(String mechanism, String authorizationId, Exception exception); +``` -Note a common authorization API is not in scope of this proposal. +Note that [RFC 4422][RFC4422] defines the "authorization identity" as one of pieces of information transferred via the challenges and responses defined by a mechanism. +In SASL this need not be the same thing as a client's username or other identity, though it usually is in Apache Kafka's use of SASL. +We're sticking with the SASL terminology in this part of the API. -The choice of `Principal` implementations is left open. In particular a plugin could, but doesn't have to, add a `KafkaPrincipal` to the subject. -Likewise a plugin could, but doesn't have to, make use of JDK-defined Principals like `javax.security.auth.x500.X500Principal`, or - `javax.security.auth.kerberos.KerberosPrincipal`. -It should be noted that plugins that add principals and plugins that query principals (including making authorization decisions) need to use common principal types. -It is therefore recommended that plugins use `KafkaPrincipal`. **TODO: Really? Why not just define our own ProxyPrincipal(type, name) and be done with it?** +### APIs for Filters to consume client SASL information -Audit logging clients is another use case for this API, in addition to this authorization example. +Some filters, such as audit loggers, may need to use SASL authentication information specifically. +For such filters we will add the following methods to `FilterContext`: +```java + /** + * Returns the SASL context for the client connection, or empty if the client + * has not successfully authenticated using SASL. + * Filters should use {@link #clientPrincipal()} in preference to this method, unless they require SASL-specific functionality. + * @return The SASL context for the client connection, or empty if the client + * has not successfully authenticated using SASL. + */ + Optional clientSaslContext(); +``` -### Proposed API for announcing client authentication outcomes +Where -The existing `SaslAuthenticateRequestFilter` and `SaslAuthenticateResponseFilter` continue to provide the mechanism for protocol-level -interaction with clients and server. -However, such SASL terminating plugins require an API through which to anounce their authentication outcomes to filters. +```java +package io.kroxylicious.proxy.authentication; -For this purpose we will add the following two methods to the existing `FilterContext` interface in the `io.kroxylicious.proxy.filter` package. +import java.util.Optional; -```java -package io.kroxylicious.proxy.filter; +import io.kroxylicious.proxy.filter.FilterContext; -public interface FilterContext { +/** + * Exposes SASL authentication information to plugins, for example using {@link FilterContext#clientSaslContext()}. + * This is implemented by the runtime for use by plugins. + */ +public interface ClientSaslContext { - // ... existing methods ... + /** + * The name of the SASL mechanism used by the client. + * @return The name of the SASL mechanism used by the client. + */ + String mechanismName(); - /** - * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) - * to announce a successful authentication outcome to subsequent plugins. - * @param subject The authenticated subject. + * Returns the client's authorizationId that resulted from the SASL exchange. + * @return the client's authorizationId. */ - void clientAuthenticationSuccess(Subject subject); + String authorizationId(); /** - * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) - * to announce a failed authentication outcome to subsequent plugins. - * @param exception An exception describing the authentication failure. + * The server identity that the proxy presented to the client using SASL authentication. + * @return the proxy's identity with the client. This will be null + * if the proxy did not supply an identity because the SASL mechanism used + * does not support mutual authentication. */ - void clientAuthenticationFailure(LoginException exception); - + Optional proxyServerId(); } ``` -It's worth noting that `LoginException` has a number of subclasses in `javax.security.auth.login`. - -This API allows SASL-terminating plugin to announce its authentication outcomes to later filters in the filter chain which have implemented `ClientSubjectAware`. -Note that client authentication outcomes only propagate towards the server, not back towards the client. -So a `ClientSubjectAware` before a SASL-terminating plugin will not receive the announcement. -**TODO justify this** - +### APIs for using client principals in a generic way -### Example: A SASL Terminating `Filter` +Most `Filters` don't need to be opinionated about how the client is identified. They only need to know: -This shows how a SASL terminating `Filter` would use the new methods on `FilterContext` to inform other filters, such as `ExampleIdentityConsumingFilter` above, about the client. +* that the client _is_ authenticated, somehow +* the name of the client's identity -The proxy config would look like this: +We want to avoid making `Filter` developers pick a source of authentication information (`clientTlsContext()` or `clientSaslContext()`) in order to maximise the reusabilty of `Filter` implementations. +For this purpose we will make use of `java.security.Principal` by providing the following method on `FilterContext`: -```yaml -virtualClusters: - - name: my-cluster - filters: - - my_authn_filter - - my_authz_filter -filterDefinitions: - - name: my_authn_filter - type: ExampleSaslTerminatingFilter - config: - jaasConfigFile: client-jaas.login.config - jaasContextName: client_auth_stack - - name: my_authz_filter - type: ExampleIdentityConsumingFilter - config: ... -``` - -where the `client-jaaas.login.config` file might look like this: - -``` -client_auth_stack { - org.apache.kafka.common.security.plain.PlainLoginModule required - user_alice="pa55word" - user_bob="changeit" - ; -}; +```java + /** + *

Returns the authenticated principal for the client connection.

+ * + *

The concrete type of principal returned depends on the proxy configuration. + * For example, + * it may be a {@link javax.security.auth.x500.X500Principal} if client identity is TLS-based, + * or it may be a {@link SaslPrincipal} if client identity is SASL based.

+ * + *

Callers should not:

+ *
    + *
  • assume any particular type of principal. + *
  • assume the principal does not change during the lifetime of a filter (due to reauthentication) + *
+ * + * @return The authenticated principal for the client connection, or empty if the client + * has not successfully authenticated. + */ + Optional clientPrincipal(); ``` -The implementation would look something like this: +The choice about what concrete type of `Principal` this method returns will be left to the person configuring the proxy. +They will do this using a new `principalType` property in the `VirtualCluster` configuration YAML. +This will support the values `X500` or `SASL`. +When configured with `X500`, the `Principal` returned by `FilterContext.clientPrincipal()` will be the `javax.security.auth.x500.X500Principal` from the client's `java.security.cert.X509Certificate`. +When configured with `SASL`, the `principal` returned by `FilterContext.clientPrincipal()` will be an instance of `SaslPrincipal`, defined as follows: ```java -public class ExampleSaslTerminatingFilter implements SaslAuthenticateRequestFilter, ClientSubjectAware { - - private Subject subject; +package io.kroxylicious.proxy.authentication; - @Override - public void onClientAuthentication(Subject clientSubject, FilterContext context) { - // the clientSubject will have an X500Principal iff the client used mTLS. - this.subject = clientSubject; - } +import java.security.Principal; +import java.util.Objects; +/** + * A principal established using SASL. + */ +public record SaslPrincipal(String name) implements Principal { @Override - public CompletionStage onSaslAuthenticateRequest(short apiVersion, - RequestHeaderData header, - SaslAuthenticateRequestData request, - FilterContext context) { - Configuration config = new ConfigFile(URI.create("client-jaas.login.config")); - - // TODO the callback handler to use depends on the context in the jaas config - // so how do we know which handler to instantiate? - // Kafka's SaslChannelBuilder basically does its own parsing of the jaas configuration to figure it out. - CallbackHandler callbackHandler = new PlainServerCallbackHandler(); - - try { - // Note: In general these methods are not guaranteed to be non-blocking, so - // use of ThreadPoolExecutor is recommended unless an implementor knows - // from other means that blocking is impossible. - LoginContext loginContext = new LoginContext("client_auth_stack", subject, callbackHandler, config); - loginContext.login(); - - // here we propagate the subject along the pipeline - // using the new clientAuthentication() method which - // broadcasts the subject to all plugins in the upstream direction - context.clientAuthenticationSuccess(loginContext.getSubject()); - return ...; // return a success response to the client - } - catch (LoginException e) { - // here we propagate the failure along the pipeline - // using the new clientAuthenticationFailure() method which - // broadcasts some represnetation of the error - // to all plugins in the upstream direction - context.clientAuthenticationFailure(e); - return ...; // return an error response to the client - } + public String getName() { + return this.name; } } ``` -By implementing `ClientSubjectAware` and reusing the `clientSubject`, we're adding a principal to any existing `X500Principal` of the subject. +The `name` will be the `authorizationId` argument from the SASL Terminator or Inspector's call to `FilterContext.clientSaslAuthenticationSuccess()`. +### API for Filters to access server TLS information -### Proposed API for selecting TLS client certificates for server connections +So far we've only covered authentication on the _client-to-proxy connection_. +To cater for "client-side" proxy deployment topologies we must also consider authentication on the _proxy-to-server connection_. +Both TLS and SASL can provide for mutual authentication, so there may be a server identity which, logically a filter could make use of. -Initiating a connection to a broker is currently entirely the responsilibity of the runtime, using the `NetFilter` interface. -(note that `NetFilter` is **not** part of the `kroxylicious-api` module.) -This means that the TLS client certificate used is currently always the same, and cannot depend on the client subject. -Adding this new API will allow TLS client certificates for conections to servers to depend on client identity (as learned about using `ClientSubjectAware`). +The API for exposing the proxy-to-broker TLS information to `Filters` is very similar to the client one. The following method will be added to `FilterContext`: ```java -package io.kroxylicious.proxy.authentication; + /** + * @return The TLS context for the server connection, or empty if the server connection is not TLS. + */ + Optional serverTlsContext(); +``` + +Where -interface ServerTlsClientCertificateSupplier { +```java +package io.kroxylicious.proxy.tls; + +import java.security.cert.X509Certificate; +import java.util.Optional; - TlsCredentials tlsCredentials(ServerCredentialContext context); +public interface ServerTlsContext { + /** + * @return The TLS server certificate that the proxy presented to the server during TLS handshake, + * or empty if no TLS client certificate was presented during TLS handshake. + */ + Optional proxyClientCertificate(); + /** + * @return the server's TLS certificate. + */ + X509Certificate serverCertificate(); } ``` -where +### APIs for Filters to produce server SASL information + +A SASL Initiator will be able to use the follow methods on `FilterContext` to announce a successful, or failed, authentication with a Kafka server: ```java -package io.kroxylicious.proxy.authentication; + /** + * Allows a filter + * to announce a successful authentication outcome with the Kafka server to other plugins. + * After calling this method the result of {@link #serverSaslContext()} will + * be non-empty for this and other filters. + * This method may be called multiple times over the lifetime of + * a session if reauthentication is required. + * TODO define the semantics around reauth + * @param saslPrincipal The authenticated principal. + */ + void serverSaslAuthenticationSuccess(String mechanism, String serverName); -interface TlsCredentials { - /* Intentionally empty: implemented and accessed only in the runtime */ -} + /** + * Allows a filter + * to announce a failed authentication outcome with the Kafka server. + * @param exception An exception describing the authentication failure. + */ + void serverSaslAuthenticationFailure(String mechanism, String serverName, Exception exception); +``` -interface ServerCredentialContext { - /** The default key for this target cluster (e.g. from the proxy configuration file). */ - TlsCredentials defaultTlsCredentials(); - /** Factory for TlsCredentials */ - TlsCredentials tlsCredentials(Certificate certificate, PrivateKey key, Certificate[] intermediateCertificates); -} +### APIs for Filters to consume server SASL information + +```java + /** + * @return The SASL context for the server connection, or empty if the server + * has not successfully authenticated using SASL. + */ + Optional serverSaslContext(); ``` -and adding the following to allow a filter factory to generate `TlsCredentials` at initialization time so as to take work off the hotter path on which `ServerTlsClientCertificateSupplier.tlsCredentials()` is invoked: +Where: ```java +package io.kroxylicious.proxy.authentication; -public interface FilterFactoryContext { +import java.util.Optional; - // ... existing methods ... +import io.kroxylicious.proxy.filter.FilterContext; - TlsCredentials tlsCredentials(Certificate certificate, PrivateKey key, Certificate[] intermediateCertificates); -} -``` +/** + * Exposes SASL authentication information to plugins, for example using {@link FilterContext#serverSaslContext()} ()}. + * This is implemented by the runtime for use by plugins. + */ +public interface ServerSaslContext { -It is not proposed to allow dynamic selection of a set of trust anchors. -Those should remain under the control of the person configuring the proxy. + /** + * The name of the SASL mechanism used. + * @return The name of the SASL mechanism used. + */ + String mechanismName(); + /** + * Returns the principal returned by the server. + * @return the principal returned by the server, + * or empty if the SASL mechanism used does not support mutual authentication. + */ + String serverId(); -#### An Example: TLS-to-TLS identity mapping +} +``` + +### APIs for using server principals in a generic way -This shows how a plugin could choose a TLS client certificate for the broker connection, based on the connected Kafka client's TLS identity. +This works similarly to the client-facing equivalent: ```java + /** + * Returns the authenticated principal for the server connection, or empty if the server + * has not successfully authenticated, or if the server authentication was not mutual. + * The concrete type of principal returned depends on the proxy configuration. + * For example, + * it may be a {@link javax.security.auth.x500.X500Principal} if server identity is TLS-based, + * or it may be a {@link SaslPrincipal} is client identity is SASL based. + * @return The authenticated principal for the server connection, or empty if the server + * has not successfully authenticated. + */ + Optional serverPrincipal(); +``` -class ExampleMTlsFilter implements ClientSubjectAware, ServerCredentialSupplier { - private Map certs; - - ExampleMTlsFilter(Map certs) { - this.certs = certs; - } +### API for selecting target cluster TLS credentials - private Subject clientSubject; +The APIs presented so far are sufficient to write plugins which: - public void onClientAuthentication(Subject clientSubject, ClientAuthenticationContext context) { - // Store the subject for use later - this.clientSubject = clientSubject; - } +* Propagate SASL, letting a client's SASL identity reach the server unchanged +* Initiate SASL, injecting mechanism-specific credentials prior to letting client-originated requests reach the server. + The selection of the server-facing SASL credentials to use could be based on the client's identity + (e.g. SASL termination and SASL initiation in the same virtual cluster) + +What's missing is an API where the server-facing TLS client certificate is chosen based on the client's identity. +For this purpose we will add a new plugin interface, `ServerTlsCredentialSupplierFactory`. +It will use the usual Kroxylicious plugin mechanism, leveraging `java.util.Service`-based discovery. +However, this plugin is not the same thing as a `FilterFactory`. +Rather, an implementation class will be defined on the TargetCluster configuration object, and instantiated once for each target cluster. +The TargetCluster's `tls` object will gain a `tlsCredentialSupplier` property, supporting `type` and `config` properties (similarly to how filters are configured). +The interface itself is declared like this: - TlsCredentials certificate(ServerCredentialContext context) { - if (this.clientSubject == null) { - throw new IllegalStateException(); - } - return certs.getOrDefault( - this.clientSubject.getPrincipal(X500Principal.class), - context.defaultTlsCredentials()); +```java +package io.kroxylicious.proxy.tls; + +/** + *

A pluggable source of {@link ServerTlsCredentialSupplier} instances.

+ *

ServerTlsCredentialSupplierFactories are:

+ *
    + *
  • {@linkplain java.util.ServiceLoader service} implementations provided by plugin authors
    • + *
  • called by the proxy runtime to {@linkplain #create(Context, Object) create} instances
    • + *
+ * @param The type of configuration. + * @param The type of initialization data. + */ +public interface ServerTlsCredentialSupplierFactory { + I initialize(Context context, C config) throws PluginConfigurationException; + ServerTlsCredentialSupplier create(Context context, I initializationData); + default void close(I initializationData) { } } +``` -class ExampleMTlsFilterFactory implements FilterFactory<, Map> { - Map certs; - public Map initialize(FilterFactoryContext context, C config) { - certs = context.tlsCredentials(...) - } - - ExampleMTlsFilter createFilter(FilterFactoryContext context, I initializationData) { - return new ExampleMTlsFilter(certs) +`ServerTlsCredentialSupplierFactory` is following the convention established by `FilterFactory`, and the `Context` referenced above is similar to the `FilterFactoryContext`: + +```java + interface Context { + + /** + * An executor backed by the single Thread responsible for dispatching + * work to a ServerTlsCredentialSupplier instance for a channel. + * It is safe to mutate ServerTlsCredentialSupplier members from this executor. + * @return executor + * @throws IllegalStateException if the factory is not bound to a channel yet. + */ + ScheduledExecutorService filterDispatchExecutor(); + + /** + * Gets a plugin instance for the given plugin type and name + * @param pluginClass The plugin type + * @param instanceName The plugin instance name + * @return The plugin instance + * @param

The plugin manager type + * @throws UnknownPluginInstanceException If the plugin could not be instantiated. + */ +

P pluginInstance(Class

pluginClass, + String instanceName) + throws UnknownPluginInstanceException; + + /** + * Creates some TLS credentials for the given parameters. + * @param key The key corresponding to the given client certificate. + * @param certificateChain The client certificate corresponding to the given {@code key}, plus any intermediate certificates forming the certificate chain up to (but not including) the TLS certificate trusted by the peer. + * @return The TLS credentials instance. + * @see ServerTlsCredentialSupplier.Context#tlsCredentials(PrivateKey, Certificate[]) + */ + TlsCredentials tlsCredentials(PrivateKey key, + Certificate[] certificateChain); } ``` -An almost identical class could be used with the `ExampleSaslTerminatingFilter` from the previous section for SASL-to-TLS identity mapping. +So what is a `ServerTlsCredentialSupplier` that this factory creates? -### Proposed API for learning about server authentication outcomes +```java +package io.kroxylicious.proxy.tls; -This is the mirror image of `ClientSubjectAware`, but for cases where the broker is using mTLS and/or a SASL mechanism that supports mutual authentication. -It allows plugin behaviour to depend on the server's identity. +import java.security.PrivateKey; +import java.security.cert.Certificate; +import java.util.Optional; +import java.util.concurrent.CompletionStage; -```java -package io.kroxylicious.proxy.authentication; +import io.kroxylicious.proxy.authentication.ClientSaslContext; -import javax.security.auth.Subject; -import javax.security.auth.login.LoginException; - -// Allows a plugin to opt in to being aware of broker-facing authentication outcomes -interface ServerSubjectAware { - - /** - * Called when the proxy authenticates, or reauthenticates with a server - * The given serverSubject may not have a principal for the server corresponding - * to the authentication mechanism used if that authentication mechanism does not provide - * mutual authentication. +/** + * Implemented by a {@link io.kroxylicious.proxy.filter.Filter} that provides + * the credentials for the TLS connection between the proxy and the Kafka server. + */ +public interface ServerTlsCredentialSupplier { + /** + * Return the TlsCredentials for the connection. + * @param context The context. + * @return the TlsCredentials for the connection. */ - void onServerAuthentication(Subject serverSubject, ServerAuthenticationContext context); - + CompletionStage tlsCredentials(Context context); +} +``` + +Where the `Context` will be a inner interface: + +```java /** - * Called when the proxy fails authentication, or reauthentication, with a server + * The context API for {@link ServerTlsCredentialSupplier}. + * This is implemented by the runtime for use by plugins. */ - default void onServerAuthenticationFailure(LoginException exception, ServerAuthenticationContext context) { + interface Context { + Optional clientTlsContext(); + Optional clientSaslContext(); + + /** + * Returns the default credentials for this target cluster (e.g. from the proxy configuration file). + * Implementations of {@link ServerTlsCredentialSupplier} may use this as a fall-back + * or default, for example if the apply a certificiate-per-client-principal pattern + * but are being used with an anonymous principal. + * @return the default credentials. + */ + TlsCredentials defaultTlsCredentials(); + + /** + *

Factory methods for creating TLS credentials for the given parameters.

+ * + *

The equivalent method on {@code FilterFactoryContext} can be used when the credentials + * are known at plugin configuration time.

+ * + * @param key The key corresponding to the given client certificate. + * @param certificateChain The client certificate corresponding to the given {@code key}, plus any intermediate certificates forming the certificate chain up to (but not including) the TLS certificate trusted by the peer. + * @return The TLS credentials instance. + * see io.kroxylicious.proxy.filter.ServerTlsCredentialSupplierFactory.Context#tlsCredentials(PrivateKey, Certificate[]) + */ + TlsCredentials tlsCredentials(Certificate certificate, + PrivateKey key, + Certificate[] intermediateCertificates); } -} ``` +And `TlsCredentials` looks like this: + ```java package io.kroxylicious.proxy.authentication; -public interface ServerAuthenticationContext { - - /** - * The subject that the proxy presented to the server. - */ - Subject proxySubject(); +interface TlsCredentials { + /* Intentionally empty: implemented and accessed only in the runtime */ } ``` -### Proposed API for initiating a SASL exchange with the broker - -There are two mutually exclusive cases to consider for SASL initiation: +// TODO Why not use the JDK's `X500PrivateCredential`? -1. That a SASL initiator plugin should perform SASL authentication _eagerly_, as soon as the underlying connection to the server is ready. -This would be driven by the runtime following a successful TCP handshake on the client-to-proxy connection, and after `ClientSubjectAware` plugins have initially been called with any `X500Principal`, but before any SASL exchange on the client side. -2. That the SASL initiator plugin should perform SASL authentication _lazily_, as soon as a KRPC requests propagates along the chain to the initator plugin. -In this case the `ClientSubjectAware` plugins may have been called with a SASL principal. +### Protections for those configuring a virtual cluster -Supporting the former case would allow the broker's principal (as obtained by a `ServerSubjectAware` implementing plugin) to be used in the client-facing SASL exchange. This could be relevant for Kafka clients which validated or made use of the authenticate server (in this case proxy) principal. However, we're not aware of any clients that actually do this, so we won't consider it further. +So far this proposal has provided a classification of `Filters` consuming the SASL Kafka protocol messages, and described Java APIs to be used by `Filter` developers producing or consumer authenticated identity information. +However, we also need to consider the task of constructing a working proxy (more specifically virtual cluster) from those building blocks. +We would like to make it a startup-time error to configure a virtual cluster in a way that cannot possibly work. +Examples of such illogical configurations include: -Supporting the latter case allows the proxy's server-facing SASL credentials to depend on the client-facing principal. -Because of variation in behaviour of client libraries there is not a single type of request which will always be the first. -However,the existing `Sasl(Handshake|Authenticate)(Request|Response)Filter` interfaces can be used for the KRPC level implementation. -Such a filter implementation needs to keep some kind of `seenFirstRequest` state if it wants to use `FilterContent.sendRequest()` to insert its own initial requests prior to forwarding requests from the client. +* Configuring a virtual cluster with `principalType: SASL` without a SASL terminator or SASL inspector in the virtual cluster's `filters` (because where is the SASL principal going to come from?) +* Configuring a virtual cluster with `principalType: X500` in a virtual cluster not configured for client mTLS (because where is the TLS principal going to come from?) +* Configuring multiple SASL terminators and/or SASL inspectors in the `filters` of a virtual cluster (because there should be a single producer of client identity). Similarly for SASL initiators (becausde there should be a single producer of server identity). -What's missing is a way for the initiator plugin to inform other plugins of the outcome. For this purpose we will add the following two methods to the existing `FilterContext` interface in the `io.kroxylicious.proxy.filter` package. +To provide this kind of fail-safe, the proxy runtime needs to know which filters are SASL inspectors, terminators or initiators, and what sort of identity information a filter consumes. +At proxy start up time, the runtime only knows about `FilterFactories`, not about any `Filter` instances or their types, +The `FilterFactory` service interface doesn't provide a way for the runtime to know what kind of filters it may create. +Therefore we will introduce the following runtime-retained annotation types to be applied to `FilterFactory` implementations: ```java -package io.kroxylicious.proxy.filter; - -public interface FilterContext { - - // ... existing methods ... - - - /** - * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) - * to announce a successful authentication outcome to subsequent plugins. - * @param subject The authenticated subject. - */ - void serverAuthenticationSuccess(Subject subject); +/** + * Annotation to be applied to `FilterFactory` implementations indicating that + * the factory create filters that call {@link FilterContext#clientSaslAuthenticationSuccess()}. + */ +@interface ClientSaslProducer{} + +/** + * Annotation to be applied to `FilterFactory` implementations indicating that + * the factory create filters that call {@link FilterContext#serverSaslAuthenticationSuccess()}. + */ +@interface ServerSaslProducer{} + +/** + * Annotation to be applied to `FilterFactory` implementations indicating that + * the factory create filters the call {@link FilterContext#clientPrincipal()}, + * or {@link FilterContext#clientSaslContext()}. + */ +@interface ClientPrincipalConsumer{ + Class[] value(); +} - /** - * Allows a filter (typically one which implements {@link SaslAuthenticateRequestFilter}) - * to announce a failed authentication outcome to subsequent plugins. - * @param exception An exception describing the authentication failure. - */ - void serverAuthenticationFailure(LoginException exception); - +/** + * Annotation to be applied to `FilterFactory` implementations indicating that + * the factory create filters the call {@link FilterContext#serverPrincipal()}, + * or {@link FilterContext#serverSaslContext()}. + */ +@interface ServerPrincipalConsumer{ + Class[] value(); } ``` -These are the mirror image of the equivalent client methods, and calling them would similarly result in `ServerSubjectAware`-implememting plugins getting informed of the server's subject. +|--------------------------------|-------------------------------------------------|----------------------------------------------------| +| `VirtualCluster.principalType` | FilterFactory annotation | Behaviour | +|--------------------------------|-------------------------------------------------|----------------------------------------------------| +| none | `@ClientPrincipalConsumer(Principal.class)` | Startup error | +| none | `@ClientPrincipalConsumer(X500Principal.class)` | Startup error | +| none | `@ClientPrincipalConsumer(SaslPrincipal.class)` | Startup error | +| `X500` | `@ClientPrincipalConsumer(Principal.class)` | All good (filter doesn't care about concrete type) | +| `X500` | `@ClientPrincipalConsumer(X500Principal.class)` | All good | +| `X500` | `@ClientPrincipalConsumer(SaslPrincipal.class)` | Startup error | +| `SASL` | `@ClientPrincipalConsumer(Principal.class)` | All good (filter doesn't care about concrete type) | +| `SASL` | `@ClientPrincipalConsumer(X500Principal.class)` | Startup error | +| `SASL` | `@ClientPrincipalConsumer(SaslPrincipal.class)` | All good | +|--------------------------------|-------------------------------------------------|----------------------------------------------------| + + +## Affected/not affected projects +The `kroxylicous` repo. -### Use cases +## Compatibility -* SASL-to-SASL identity mapping (in combination with a SASL terminator) -* SASL-to-SASL identity preservation with in-proxy authorization (in combination with a SASL terminator, and an authorizer). -* Audit logging +This change would be backwards compatible for `Filter` developers and proxy users (i.e. all existing proxy configurations files would still be valid). -### Implementation +# Future work -The preceeding section are intented to lay out a comprehensive API covering a variety of authentication use cases. -This is to encourage review of the proposal that considers all the possible use cases. -However, there is no requirement for them to be implemented in one go/within one release cycle. -Some of the proposed changes will require non-trival changes in the proxy runtime. +* Implement a 1st party `SaslInspector` filter. +* Implement a 1st party `SaslTerminator` filter. +* Implement a 1st party `SaslInitiator` filter. +* A `PrincipalBuilder` API for customizing the concrete type of `Principal` exposed to `Filters` using `FilterContext.clientPrincipal()` +* A common authorization API. -### Design choices -* Use JAAS because: - - the Kafka broker and clients already use JAAS - - we'd rather avoid adding a dependency on those not-publicly-supported Kafka classes in `kroxylicious-api` and `kroxylicious-runtime`, - - but we recognise that 3rd part Filter authors might want to make that choice - - we have no appetite to build-out our own API, nor to pick up a dependency on a 3rd party framework -* Use `Subject` to convey identity information, rather than `Principal` because - - this is more sympathetic with the conceptual model of JAAS. - - it allows attaching multiple `Principals` to client `Subjects`, and to be consumed by `ClientSubjectAware` instances (e.g. know the client TLS certificate DN _and_ the SASL SCRAM-SHA username). +## Rejected alternatives -## Affected/not affected projects -The `kroxylicous` repo. -## Compatibility +# References -This change would be backwards compatible for Filter developers and proxy users (i.e. all existing proxy configurations files would still be valid). +SASL was initially defined in [RFC 4422][RFC4422]. +Apache Kafka has built-in support for a number of mechanisms. +Apache Kafka also supports plugging-in custom mechanisms on both the server and the client. -## Future work +|---------------------|---------------------|--------------------------| +| Mechanism | Definition | Kafka implementation KIP | +|---------------------|---------------------|--------------------------| +| PLAIN | [RFC 4616][RFC4616] | [KIP-42][KIP43] | +| GSSAPI (Kerberos v5)| [RFC 4752][RFC4752] | [KIP-12][KIP12] | +| SCRAM | [RFC 5802][RFC5802] | [KIP-84][KIP84] | +| OAUTHBEARER | [RFC 6750][RFC6750] | [KIP-255][KIP255] | +|---------------------|---------------------|--------------------------| -This proposal in combination with the proposal on a routing API, would enable client-subject based routing to backing clusters. +Note that the above list of KIPs is not exhaustive: Other KIPs have further refined some mechanisms, and defined reauthentication. -## Rejected alternatives +[RFC4422]:https://www.rfc-editor.org/rfc/rfc4422 +[RFC4616]:https://www.rfc-editor.org/rfc/rfc4616 +[RFC4752]:https://www.rfc-editor.org/rfc/rfc4752 +[RFC5802]:https://www.rfc-editor.org/rfc/rfc5802 +[RFC6750]:https://www.rfc-editor.org/rfc/rfc6750 +[KIP12]:https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=51809888 +[KIP43]:https://cwiki.apache.org/confluence/display/KAFKA/KIP-43%3A+Kafka+SASL+enhancements +[KIP84]:https://cwiki.apache.org/confluence/display/KAFKA/KIP-84%3A+Support+SASL+SCRAM+mechanisms +[KIP255]:https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75968876 -* Why not just add `public Principal clientPrincipal();` to `FilterContext`? - - It doesn't support multiple principals. -* OK, so why not just add `public Subject clientSubject();` to `FilterContext`? - - It makes the `Subject` a property of the connection/Netty channel. The public API presented here allows a `Filter` to change the `Principals` presented to subsequent `Filters` in the chain, enabling `Filters` to implement a wider variety of use cases. From 8b926229b5a88b9c6b0d56b454ebea30c3ba9944 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Thu, 17 Jul 2025 18:37:59 +1200 Subject: [PATCH 7/7] Review comments Signed-off-by: Tom Bentley --- proposals/003-authentication-api.md | 41 +++++++++++++---------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/proposals/003-authentication-api.md b/proposals/003-authentication-api.md index 472b5fe..5631eb7 100644 --- a/proposals/003-authentication-api.md +++ b/proposals/003-authentication-api.md @@ -14,7 +14,7 @@ Let's first define terms for TLS: * When the proxy is configured to accept TLS connections from clients it is performing **TLS Termination**, which does not imply mutual authentication. (For the avoidance of doubt, Kroxylicious does not support TLS passthrough; the alternative to TLS Termination is simply using TCP as the application transport). * A **TLS client certificate** is a TLS certificate for the client-side of a TLS Handshake. - For a given client and server pairing a proxy might have _two_ of these: the Kafka client's TLS client certificate, and the proxy's own TLS client certificate for its connection to the server. + For a given client and server pairing, a proxy might have _two_ of these: the Kafka client's TLS client certificate, and the proxy's own TLS client certificate for its connection to the server. * When the proxy is configured to require TLS client certificates from clients and validates these against a set of trusted signing certificates (CA certificate) it is performing **client mutual TLS authentication** ("client mTLS"). * A **TLS server certificate** is a TLS certificate for the server-side of a TLS Handshake. As above, there could be two of these for a given connection through a proxy. @@ -43,7 +43,7 @@ Finally, let's define some concepts from JAAS: A subject gains a principal following a successful TLS handshake. A subject also gains a principal following a successful `SaslAuthenticate` exchange. * A **credential** is information used to prove the authenticity of a principal. -* A **public credential**, such as a TLS certificate, need not be kept a secret. +* A **public credential**, such as a TLS certificate, does not need not be kept secret. * A **private credential**, such as a TLS private key or a password, must be kept secret, otherwise the authenticity of a principal is compromised. ## Current situation @@ -81,7 +81,8 @@ The following method will be added to the existing `FilterContext` interface: ```java /** - * @return The TLS context for the client connection, or empty if the client connection is not TLS. + * @return The TLS context for the connection between the Kafka client and the proxy, + * or empty if the client connection is not TLS. */ Optional clientTlsContext(); ``` @@ -101,7 +102,8 @@ public interface ClientTlsContext { X509Certificate proxyServerCertificate(); /** - * @return the client's certificate, or empty if no TLS client certificate was presented during TLS handshake. + * @return the TLS client certificate was presented by the Kafka client to the proxy during TLS handshake, + * or empty if no TLS client certificate was presented. */ Optional clientCertificate(); @@ -128,6 +130,11 @@ For this purpose we will add the following methods to the existing `FilterContex * In order to support reauthentication, calls to this method and * {@link #clientSaslAuthenticationFailure(String, String, Exception)} * may be arbitrarily interleaved during the lifetime of a given filter instance. + * + * This method can only be called by filters created from {@link FilterFactory FilterFactories} which + * have been annotated with {@link ClientSaslProducer @ClientSaslProducer}. + * Calls from filters where this is not the case will be logged but otherwise ignored. + * * @param mechanism The SASL mechanism used. * @param authorizationId The authorization ID. */ @@ -141,6 +148,11 @@ For this purpose we will add the following methods to the existing `FilterContex * In order to support reauthentication, calls to this method and * {@link #clientSaslAuthenticationSuccess(String, String)} * may be arbitrarily interleaved during the lifetime of a given filter instance. + * + * This method can only be called by filters created from {@link FilterFactory FilterFactories} which + * have been annotated with {@link ClientSaslProducer @ClientSaslProducer}. + * Calls from filters where this is not the case will be logged but otherwise ignored. + * * @param mechanism The SASL mechanism used, or null if this is not known. * @param authorizationId The authorization ID, or null if this is not known. * @param exception An exception describing the authentication failure. @@ -194,14 +206,6 @@ public interface ClientSaslContext { * @return the client's authorizationId. */ String authorizationId(); - - /** - * The server identity that the proxy presented to the client using SASL authentication. - * @return the proxy's identity with the client. This will be null - * if the proxy did not supply an identity because the SASL mechanism used - * does not support mutual authentication. - */ - Optional proxyServerId(); } ``` @@ -271,7 +275,8 @@ The API for exposing the proxy-to-broker TLS information to `Filters` is very si ```java /** - * @return The TLS context for the server connection, or empty if the server connection is not TLS. + * @return The TLS context for the connection between the proxy and the Kafka server, + * or empty if the server connection is not TLS. */ Optional serverTlsContext(); ``` @@ -292,7 +297,7 @@ public interface ServerTlsContext { Optional proxyClientCertificate(); /** - * @return the server's TLS certificate. + * @return the TLS server certificate was presented by the Kafka server to the proxy during TLS handshake. */ X509Certificate serverCertificate(); } @@ -353,14 +358,6 @@ public interface ServerSaslContext { * @return The name of the SASL mechanism used. */ String mechanismName(); - - /** - * Returns the principal returned by the server. - * @return the principal returned by the server, - * or empty if the SASL mechanism used does not support mutual authentication. - */ - String serverId(); - } ```