Feat: Add token-broker service for OAuth session and token management

[davidhadas]

Summary

The token-broker service provides centralized OAuth token acquisition for AI agents via PKCE-based authorization flows, token caching with JWT expiry parsing, and session management with long-polling coordination.

Fixes: #455

Related issue(s)

kagenti/kagenti#1435

[rhuss]

Thanks for the work on this, David. I don't want to block your research progress, but a few observations from the productization side might be still ok:

Completeness: The token-broker cannot complete an OAuth flow end-to-end without the Backend service (Phase 3) that handles the user-facing browser interaction. JWT signature validation is also explicitly skipped. As it stands, this is a Phase 1 foundation, not a shippable component.

Keycloak overlap: The core functionality (authorization code + PKCE, token caching, JWT expiry parsing, session management) overlaps significantly with what Keycloak provides natively through identity brokering. The implementation also reimplements standard Go library functionality (golang.org/x/oauth2, golang-jwt/jwt) rather than using them, which increases the maintenance and CVE surface without clear benefit. The legitimate gap (bridging to external providers not federated with Keycloak) can potentially be addressed through Keycloak identity brokering configuration or CIBA, both of which are production-hardened paths that don't require a new service.

Repo integration: The token-broker is architecturally independent (separate binary, separate Deployment, zero imports from operator code), but it shares the operator's go.mod. This means downstream consumers who don't ship the token-broker still inherit its dependencies in SBOM/vulnerability scans and need to carry deletion patches that require go mod tidy after every upstream sync. Since the code is fully decoupled, moving it to its own Go module (separate go.mod, either as a subdirectory module or a separate repo) would make it cleanly excludable without per-sync patching.

For a long term solution, maybe consider extracting the token-broker into its own module/repo and evaluating whether Keycloak identity brokering or CIBA could address the external-provider token acquisition gap before building a custom service. Happy to discuss the Keycloak/CIBA angle further (see also the discussion on #1435).

[davidhadas]

Thanks for the review.
A Backend service - see kagenti/kagenti#1583
JWT signature validation - committed

As it stands, this is a Phase 1 foundation, not a shippable component.

Agreed. We are laying down the HITL foundation. More work is definitely needed, but we are at a phase where we can finally see the landscape and discuss the design decisions, controls, configs, and user experience.

Does this mean we should hold off on merging this PR once it passes review? Or can we merge it post-review with the explicit note that we still have work before it is production-ready?

Repo integration: Understood. So, keep it in this repo but ensure it uses its own isolated go.mod.

Relationship with Keycloak: Let’s keep that specific discussion over to the Epic issue kagenti/kagenti#1435 as it is part of the big picture we need to discuss.

It should be noted here though that token broker service should have a later PR to include integration with KeyCloak supporting use cases in which Kagneti is deployed with KeyCloak and need to offer OAuth 2 against federated systems.

[rhuss]

Thanks David. To be clear: please don't feel blocked by me on merging this. My comments were architectural observations, not merge blockers.

I've come to better appreciate that kagenti's upstream is designed for research incubation and rapid iteration, without stability guarantees that would slow contributors down. That's a valid and valuable mode of operation. The productization constraints I've been raising come from a different set of goals, and it's not fair to apply those as a gate on upstream research work.

Go ahead and merge when it passes review. The foundation you're laying here is solid exploration of the HITL problem space. Separate go.mod would still be appreciated for cleaner downstream consumption, but that's a housekeeping ask, not a blocker.

[davidhadas]

@rhuss ,
The token broker service was moved to its own Go module.

[akram]

CleanupExpiredTokens() is defined but never wired to a ticker in main.go. For long-running instances, expired tokens will accumulate in memory. Suggestion: add a periodic cleanup
goroutine, or clean up lazily on SetToken when the session map exceeds a threshold.

[akram]

the OAuthDiscoverer interface is tightly coupled to the OAuth authorization code + PKCE flow.
If we want to support alternative token acquisition mechanisms (e.g., Keycloak identity brokering, CIBA, or custom IdP flows), it might be worth defining a higher-level TokenAcquirer
interface that abstracts the grant type entirely, with PKCE as one implementation.

[akram]

The API uses two different error response formats: writeError (flat, for AuthBridge) and writeBackendError (nested, for Backend). This makes client integration harder — consumers need
to know which endpoint returns which format. Consider standardizing on a single error envelope.

[akram]

Already discussed in the Slack thread — the replicas: 1 constraint with in-memory state is a known limitation. Just noting that the security posture is good (non-root, read-only
filesystem, capabilities dropped).

[akram]

Keycloak overlap question

Roland's observation about the overlap with Keycloak identity brokering is worth exploring further. The core of what the token-broker implements — authorization code + PKCE, token caching, JWT expiry parsing, session management — maps closely to what Keycloak provides natively. CIBA (Client-Initiated Backchannel Authentication) is also an interesting alternative for the HITL use case: the backend initiates the auth request, the user consents via a separate channel, no browser redirect needed.

Provider-agnostic token brokering interface

That said, tying the HITL flow exclusively to Keycloak would create a hard dependency on a specific IdP. Since we're actively working on decoupling from Keycloak on the AuthBridge side, it raises a broader design question: could we define a provider-agnostic interface for token brokering, where Keycloak is one possible backend but not the only one?

Concretely, the current OAuthDiscoverer interface in interfaces.go is tightly coupled to the authorization code + PKCE flow. A higher-level TokenAcquirer abstraction could support multiple grant types — PKCE, CIBA, identity brokering, or custom IdP flows — so the HITL mechanism works regardless of the underlying IdP (Keycloak, external OIDC provider, or something else entirely).

[davidhadas]

@akram - all good points. Thank you for the review. I will address one by one and present for additional review.

Let record the overall solution directions discussions in the epic kagenti/kagenti#1435 - I think some of your comments related to Keycloak overlap question were already discussed there and maybe you can comment when it was not.

Related to Provider-agnostic token brokering interface, I would like to consider the proposed higher-level TokenAcquirer abstraction and come back with thoughts.