diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml new file mode 100644 index 00000000..a37b5d37 --- /dev/null +++ b/.github/workflows/pages.yml @@ -0,0 +1,54 @@ +name: Publish Book + +on: + workflow_dispatch: + pull_request: + paths: + - ".github/workflows/pages.yml" + - "docs/book/**" + push: + branches: [main] + paths: + - ".github/workflows/pages.yml" + - "docs/book/**" + +permissions: + contents: read + +jobs: + build: + runs-on: ubuntu-latest + concurrency: + group: pages-build-${{ github.ref }} + cancel-in-progress: true + steps: + - uses: actions/checkout@v6.0.2 + - name: Install mdBook + uses: taiki-e/install-action@v2.75.27 + with: + tool: mdbook@0.5.3 + - name: Build book + run: mdbook build docs/book + - name: Upload Pages artifact + uses: actions/upload-pages-artifact@v4 + with: + path: docs/book/book + + deploy: + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + needs: build + runs-on: ubuntu-latest + permissions: + contents: read + pages: write + id-token: write + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + concurrency: + group: pages + cancel-in-progress: false + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 diff --git a/.gitignore b/.gitignore index 0e5210a8..c267df73 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,9 @@ target # Generated by gateway local runs contextforge-gateway-rs.log.* +# Generated by mdBook +docs/book/book/ + # RustRover # JetBrains specific template is maintained in a separate JetBrains.gitignore that can # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore diff --git a/AGENTS.md b/AGENTS.md index c8d643a9..728fc0fd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,149 +1,48 @@ # AGENTS.md -Architecture notes for agents working on `contextforge-gateway-rs`. - -This repo is the Rust dataplane part of ContextForge. It must stay compatible with the external ContextForge control plane in `https://github.com/IBM/mcp-context-forge`, but it must not become a control-plane, UI, IAM, or metrics-storage app. - -## Mental Model - -The gateway accepts downstream MCP streamable HTTP traffic, authenticates the caller, loads that user's virtual-host config, opens or reuses MCP client sessions to configured backend MCP servers, then presents those backends as one merged MCP server. - -```text -MCP client - -> /contextforge-rs/servers/{virtual_host_id}/mcp - -> JWT auth - -> user config lookup - -> virtual host backend list - -> per-backend MCP client sessions - -> merged tools/resources/prompts back to client -``` - -Existing ContextForge control-plane and management concerns live outside this repo. This repo owns the fast dataplane path only. - -## Dataplane Direction - -Current direction: - -- Control plane stays external. It owns management APIs, UI, user/admin workflows, configuration, credentials, policies, and durable storage. -- Dataplane owns the hot request/response path. It consumes runtime config, routes traffic, applies policy/guardrails/plugins, calls upstreams, and emits telemetry. -- Runtime config arrives from the control plane through Redis now, possibly xDS/gRPC later. -- `/servers/{virtual_host_id}/mcp` should behave like the legacy ContextForge MCP endpoint from a client point of view. -- Nginx/front-door routing may send only `/servers/{uuid}/mcp` traffic here while all other ContextForge traffic stays on existing CF paths. -- Platform scope starts with MCP and should keep auth, policy, telemetry, and config ingestion reusable for A2A and LLM/model gateway traffic. -- Plugins may need request/response payload access, not just headers. Performance-sensitive paths should keep CPU, memory, locking, and task boundaries explicit. -- This project is still early development with no external users; prefer the right architecture over preserving unstable APIs or compatibility surfaces. - -## Workspace Layers - -```text -crates/contextforge-gateway-rs - process shell: config parsing, logging, runtime, dependency assembly - -crates/contextforge-gateway-rs-lib - dataplane: listeners, Axum middleware, auth, config lookup, MCP fanout/routing, sessions - -crates/contextforge-gateway-rs-apis - shared contract: user config model and schema generation - -crates/contextforge-load-test - performance harness: end-to-end MCP traffic driver -``` - -Most product behavior belongs in `contextforge-gateway-rs-lib`; avoid adding dataplane logic to the binary crate. - -## Target Pipeline - -The gateway is a bidirectional AI traffic pipeline: - -```text -downstream request - -> authentication / authorization - -> rate limiting - -> routing / protocol selection - -> request payload/header modification - -> optional retrieval / augmentation - -> request guardrails - -> upstream MCP / A2A / model provider - -upstream response - -> response guardrails - -> response payload/header modification - -> metrics / tracing / logging - -> downstream response -``` - -Current code implements the MCP subset. Preserve ordering: auth and config before backend selection; request plugins/mutation before upstream calls; response plugins/mutation before returning; telemetry around both sides. - -The intended runtime shape is a hot request path plus support loops: - -- listener and Hyper/Axum stack on Tokio executors -- JWT/auth middleware -- session extraction -- user/config retrieval -- MCP session management around gateway logic -- request and response plugin hooks -- upstream client boundary -- separate configuration and metrics collection work - -Keep allocation, locking, cross-task communication, and shared mutable state intentional. - -## Request Flow - -Gateway route: - -```text -/contextforge-rs/servers/{virtual_host_id}/mcp -``` - -Startup assembly: - -1. `contextforge-gateway-rs/src/main.rs` parses `Config`. -2. It builds `RedisUserConfigStore`. -3. It builds `Gateway` with config, user config store, and RMCP local session manager. -4. `Gateway::run_gateway` creates an RMCP `StreamableHttpService`. -5. Axum middleware wraps the service and TCP/TLS listeners expose it. - -Per request, middleware populates extensions: - -1. `claims_layer` validates JWT and stores `ContextForgeGatewayClaims`. -2. `user_config_store_layer` uses JWT subject as Redis key and stores `UserConfig`. -3. `SessionIdLayer` reads `Mcp-session-id` into `SessionId`. -4. `virtual_host_id_layer` extracts `{virtual_host_id}` from the path. - -`McpService` reads those extensions through `InitializeCallValidator` or `AuthorizedCallValidator`. - -## Runtime Config Model - -User config is the routing source of truth: - -```text -UserConfig - virtual_hosts: HashMap - -VirtualHost - backends: HashMap - -BackendMCPGateway - url: Url -``` - -Current persistence: - -- User key is `User::new(jwt_subject)`. -- Keys and values are MessagePack-encoded. -- `RedisUserConfigStore` keeps an in-process LRU cache in front of Redis. - -Expected config growth: - -- route selection across multiple MCP endpoints -- principal/virtual-host filters for tools, resources, and prompts -- backend auth/TLS material references -- request/response header pass/add/remove rules -- plugin/CPEX hook settings -- pagination/SSE behavior where protocol handling needs config -- future A2A and LLM routing/provider settings - -Keep persistent config access behind `UserConfigStore`. Do not push Redis details into routing code. +Guidance for agents working on `contextforge-gateway-rs`. + +This repo is the Rust dataplane part of ContextForge. It must stay compatible +with the external ContextForge control plane in +`https://github.com/IBM/mcp-context-forge`, but it must not become a +control-plane, UI, IAM, or metrics-storage app. + +## Architecture + +Architecture documentation lives in The ContextForge Gateway Book under +[docs/book](docs/book/README.md). Read the relevant page before changing the +hot path: + +| Page | Read it for | +| --- | --- | +| [What is ContextForge Gateway?](docs/book/src/what-is-contextforge-gateway.md) | Scope, boundaries, key terms, and the mental model. | +| [System Shape](docs/book/src/system-shape.md) | Crate layout, control-plane boundary, pipeline shape, state ownership, and module boundaries. | +| [Request Flow](docs/book/src/request-flow.md) | Startup, middleware order, initialize fanout, authorized calls, and the response path. | +| [Concurrency And Runtime Model](docs/book/src/concurrency-and-runtime.md) | Executor shapes, shared state and locks, fanout, and cancellation. | +| [Authentication And User Config Lookup](docs/book/src/authentication-and-user-config.md) | JWT validation, config keying, cache behavior, and failure responses. | +| [Security Model And Trust Boundaries](docs/book/src/security-model.md) | Trust boundaries, compromise impact, and transport security posture. | +| [Runtime Configuration](docs/book/src/runtime-configuration.md) | The `UserConfig` model, Redis/MessagePack persistence, and plugin runtime config. | +| [Control-Plane Integration](docs/book/src/control-plane-integration.md) | Redis keys, schemas, token shape, and route parity with the control plane. | +| [Backend Connections And Transports](docs/book/src/backend-connections-and-transports.md) | Downstream, upstream, and config-store transports plus TLS direction. | +| [Session Ownership](docs/book/src/session-ownership.md) | Backend session state, cleanup, and load-balancing constraints. | +| [MCP Routing Semantics](docs/book/src/mcp-routing-semantics.md) | The backend prefix namespace and routing contract. | +| [Architectural Choices](docs/book/src/architectural-choices.md) | Invariants and tradeoffs that must not change accidentally. | + +The book is rendered from `docs/book/src/` and published through GitHub Pages; +see [docs/book/README.md](docs/book/README.md) for build and validation steps. + +## Working Rules + +- Most product behavior belongs in `contextforge-gateway-rs-lib`; avoid adding + dataplane logic to the binary crate. +- Keep persistent config access behind `UserConfigStore`; do not push Redis + details into routing code. +- Do not change the backend prefix naming contract without updating merge + logic, split logic, and tests. +- This project is still early development with no external users; prefer the + right architecture over preserving unstable APIs or compatibility surfaces. +- When behavior on the hot path changes, update the matching book page in the + same change. ## Logging @@ -152,83 +51,3 @@ Keep persistent config access behind `UserConfigStore`. Do not push Redis detail - Keep the method/event prefix stable and reuse the same field names/order for related events. - Keep warning logs for unexpected conditions that likely need operator attention. Expected user/config misses should be debug or info unless they indicate a platform problem. - Do not log tokens, authorization headers, secrets, Redis key/value bytes, full `UserConfig`, or backend credentials. - -## Backend Sessions - -Initialization fans out: - -1. Client calls `initialize`. -2. `McpService::initialize` validates virtual host and downstream session id. -3. It creates one RMCP client transport per backend URL. -4. It forwards the initialize request to each backend. -5. It stores backend services under `(backend_name, downstream_session_id)`. -6. It returns merged gateway capabilities downstream. - -Later calls reuse backend services from the shared transport map. `SessionManager::borrow_transports` temporarily removes services from the map; callers must return them with `return_transports` or deliberately remove them with `cleanup_backends`. - -Load-balanced deployments are unresolved. Known options are sticky routing by `Mcp-session-id`, remote session mapping through Redis/external cache, or active-hot-standby where clients reinitialize after failover. Do not assume any node can serve any stateful MCP session unless session state has moved out of process. - -## MCP Routing Semantics - -Backends are namespaced by prefix. - -- Backend tool `increment` from backend `gateway-one` becomes `gateway-one-increment`. -- `call_tool` splits `{backend_name}-{tool_name}` and routes only to that backend. -- Resources follow the same prefix/split model. -- Listed tools/resources are merged and sorted before returning. - -Do not change this naming contract without updating merge logic, split logic, and tests. - -Tracked MCP gaps: - -- `list_tools`, `list_resources`, and `list_prompts` pagination should gather all backend pages before returning merged output. -- SSE responses should stream downstream as backend chunks arrive. -- Stateless MCP calls are a target for parity and future load-balancing. -- Filtering must use principal plus runtime config, not hard-coded backend behavior. - -## Transports - -Keep three transport concerns separate: - -- Downstream listener transport: how clients reach the gateway; implemented in `transports/tcp.rs` and `transports/tls.rs`. -- Upstream backend transport: how the gateway reaches backend MCP servers; implemented through `reqwest::Client` plus RMCP `StreamableHttpClientTransport`. -- Config-store transport: how runtime config is loaded; currently Redis. - -Transport security is moving from static process config toward runtime config: - -- downstream TLS remains listener-level gateway config -- upstream TLS/mTLS should be selectable per backend -- backend authorization headers may come from runtime config -- PEM certificate material may be stored or referenced through Redis-managed config - -## Plugins And Policy - -Plugins may inspect or mutate request/response bodies, so they affect architecture more than simple header filters. - -Likely hook points: - -- after auth/config lookup, before backend selection -- before forwarding upstream -- after receiving backend response -- before returning merged/listed results downstream - -When adding plugins, define behavior for streaming/SSE, failures, timeouts, backpressure, resource ownership, and OpenTelemetry attribution. Plugin execution must not block unrelated sessions or poison shared gateway state. - -## Module Boundaries - -- Keep config validation in `common.rs`. -- Keep request extension extraction in `layers/`. -- Keep MCP fanout/routing in `gateway/`. -- Keep downstream listener logic in `transports/`. -- Keep shared config shapes in `contextforge-gateway-rs-apis`. -- Keep protocol-neutral concerns separate from MCP-specific logic so A2A/LLM support can reuse auth, TLS, config, telemetry, plugin execution, and session strategy. - -## Invariants - -- JWT subject selects the user config. -- Path virtual host id selects one `VirtualHost` inside that user's config. -- Backend name is part of the public tool/resource namespace. -- Backend service ownership is temporarily moved out of the shared map during calls. -- Redis encoding is MessagePack. -- End-to-end behavior is defined by merged MCP semantics, not by leaking backend identity directly. -- This dataplane consumes control-plane config and enforces it; it does not own UI, OpenAPI management APIs, durable observability storage, or customer IAM. diff --git a/README.md b/README.md index d7e8498d..ade2d628 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,9 @@ # ContextForge Dataplane +Architecture, configuration, and operations documentation lives in +[The ContextForge Gateway Book](docs/book/src/SUMMARY.md) under `docs/book`. +Build it locally with `mdbook serve docs/book`; see +[docs/book/README.md](docs/book/README.md) for details. ## Running 1. Start Redis and gateways @@ -34,18 +38,30 @@ curl --request POST \ --header 'authorization: Bearer {{token}}' \ --header 'content-type: application/json' \ --data '{ - "virtualHosts": { - "c0ffee00f001f00lf00ldeadbeefdead": { - "backends": { - "gateway-one": { - "url": "http://127.0.0.1:5555/mcp" - }, - "gateway-two": { - "url": "http://127.0.0.1:5556/mcp" - } + "virtual_hosts": { + "c0ffee00f001f00lf00ldeadbeefdead": { + "backends": { + "gateway-one": { + "name": "gateway-one", + "url": "http://127.0.0.1:5555/mcp", + "transport": "STREAMABLEHTTP", + "passthrough_headers": [], + "allowed_tool_names": [], + "allowed_resource_names": [], + "allowed_prompt_names": [] + }, + "gateway-two": { + "name": "gateway-two", + "url": "http://127.0.0.1:5556/mcp", + "transport": "STREAMABLEHTTP", + "passthrough_headers": [], + "allowed_tool_names": [], + "allowed_resource_names": [], + "allowed_prompt_names": [] } } } + } }' ``` @@ -166,10 +182,18 @@ curl --silent --show-error --request POST \ --header "authorization: Bearer ${TOKEN}" \ --header 'content-type: application/json' \ --data '{ - "virtualHosts": { + "virtual_hosts": { "c0ffee00f001f00lf00ldeadbeefdead": { "backends": { - "gateway-one": { "url": "http://127.0.0.1:5555/mcp" } + "gateway-one": { + "name": "gateway-one", + "url": "http://127.0.0.1:5555/mcp", + "transport": "STREAMABLEHTTP", + "passthrough_headers": [], + "allowed_tool_names": [], + "allowed_resource_names": [], + "allowed_prompt_names": [] + } } } } @@ -249,139 +273,14 @@ With `--runtime-plugins-enabled true`, the response content should include the b [cpex:payload-marker] ``` -## Tracing & Metrics (Langfuse + OTel Collector + Prometheus) - -Issue [#4721](https://github.com/IBM/mcp-context-forge/issues/4721) adds OTLP -**traces** and **metrics** to the Rust dataplane. A local verification stack -ships under `docker/` so the same release binary can be exercised end-to-end -without any external services. - -The stack consists of three overlays composed on top of `docker-compose-local.yaml`: - -| Component | Role | UI / endpoint | -| --------------- | ------------------------------------------------------- | ---------------------------------------------- | -| Langfuse | Trace backend (OTLP/HTTP receiver, span viewer) | http://localhost:3100 (`admin@example.com` / `admin`) | -| OTel Collector | Receives OTLP from the gateway, fans out traces + metrics | OTLP/HTTP `:4318`, Prometheus exposition `:8889`, stdout via `docker logs` | -| Prometheus | Scrapes the collector's `/metrics` for browsable PromQL | http://localhost:9090 | - -### 1. Bring up the verification stack - -```bash -docker compose \ - -f docker/docker-compose-local.yaml \ - -f docker/docker-compose-langfuse.yaml \ - -f docker/docker-compose-otel-collector.yaml \ - up -d -``` - -Wait for all containers to become healthy: - -```bash -docker compose \ - -f docker/docker-compose-local.yaml \ - -f docker/docker-compose-langfuse.yaml \ - -f docker/docker-compose-otel-collector.yaml \ - ps -``` - -### 2. Run the gateway with traces and metrics enabled - -```bash -RUST_TRACE_LOG=debug \ -cargo run --release --bin contextforge-gateway-rs -- \ - --address 0.0.0.0:8001 \ - --redis-port 6379 --redis-address 127.0.0.1 --redis-mode=plain-text \ - --token-verification-public-key assets/jwt.key.pub \ - --number-of-cpus 4 \ - --upstream-connection-mode=plain-text-or-tls \ - --enable-open-telemetry true \ - --enable-otel-metrics true \ - --otlp-protocol http-protobuf \ - --otlp-endpoint http://127.0.0.1:3100/api/public/otel/v1/traces \ - --otlp-headers "Authorization=Basic cGstbGYtY29udGV4dGZvcmdlOnNrLWxmLWNvbnRleHRmb3JnZQ==" \ - --otlp-metrics-endpoint http://127.0.0.1:4318/v1/metrics \ - --otlp-service-name contextforge-gateway-rs -``` - -Relevant flags (all also configurable via environment variables — see `--help`): - -| Flag | Env var | Purpose | -| ----------------------------- | -------------------------------------------------- | ------------------------------------------------------------- | -| `--enable-open-telemetry` | `CONTEXTFORGE_GATEWAY_RS_ENABLE_OPEN_TELEMETRY` | Turn on the OTel tracer pipeline. | -| `--otlp-endpoint` | `CONTEXTFORGE_GATEWAY_RS_OTEL_EXPORTER_OTLP_ENDPOINT` | Trace destination (Langfuse OTLP/HTTP URL here). | -| `--otlp-headers` | `CONTEXTFORGE_GATEWAY_RS_OTEL_EXPORTER_OTLP_HEADERS` | Auth header for Langfuse (Basic auth, base64 of `pk:sk`). | -| `--enable-otel-metrics` | `CONTEXTFORGE_GATEWAY_RS_ENABLE_OTEL_METRICS` | Turn on the OTel meter pipeline (added in #4721). | -| `--otlp-metrics-endpoint` | `CONTEXTFORGE_GATEWAY_RS_OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | Metrics destination (Collector OTLP/HTTP `/v1/metrics`). | -| `--otlp-service-name` | `CONTEXTFORGE_GATEWAY_RS_OTEL_SERVICE_NAME` | `service.name` resource attribute on every span and metric. | - -> `RUST_TRACE_LOG=debug` is required: the `tower_http::TraceLayer` emits -> `DEBUG`-level spans, and the default filter (`info`) would drop them before -> they ever reach the OTLP exporter — no spans would land in Langfuse. - -### 3. Generate traffic - -```bash -for i in {1..10}; do - curl -s -o /dev/null -w "%{http_code}\n" \ - http://127.0.0.1:8001/contextforge-rs/admin/tokens/admin@example.com -done -``` - -A `404` response is expected without configured users; the request is still -traced and counted as a metric sample. - -### 4. Inspect the data - -* **Langfuse — traces:** open http://localhost:3100, log in, project - `contextforge`. Each curl produces one span (HTTP method, route, status, - latency). -* **Prometheus — metrics:** open http://localhost:9090. - * `Status → Targets` should show `otel-collector:8889` as **UP**. - * Try these queries in the `Graph` tab: - * `http_server_request_duration_count` — request count, broken down by - `http_request_method`, `http_response_status_code`, and `service_name`. - * `histogram_quantile(0.95, sum by (le) (rate(http_server_request_duration_bucket[1m])))` — p95 latency. - * `http_server_active_requests` — gauge of in-flight requests. - * `http_server_request_body_size_sum` / `http_server_response_body_size_sum` — payload throughput. -* **Collector stdout:** `docker logs otel-collector --tail 200` for raw OTLP - dumps (both traces and metrics, via the `logging` exporter). - -Metrics are exported by the gateway every 30 s (one `PeriodicReader` tick), so -allow ~35 s after the first request before the first data point appears in -Prometheus. - -### Architecture - -``` -ContextForge Gateway (release binary, :8001) - │ - │ OTLP/HTTP (protobuf) - │ - ├──► :3100 ── Langfuse ──► trace UI - │ - └──► :4318 ── OTel Collector - │ - ├──► stdout (logging exporter, docker logs) - │ - └──► :8889 ── Prometheus ──► PromQL UI :9090 -``` - -### Out of scope (tracked separately) - -* W3C trace-context propagation across gateway hops — issue - [#4723](https://github.com/IBM/mcp-context-forge/issues/4723). -* MCP-semantic spans (tool names, JSON-RPC method attributes) — issue - [#4722](https://github.com/IBM/mcp-context-forge/issues/4722). - -### Tear down +## Tracing & Metrics -```bash -docker compose \ - -f docker/docker-compose-local.yaml \ - -f docker/docker-compose-langfuse.yaml \ - -f docker/docker-compose-otel-collector.yaml \ - down -``` +The gateway exports OTLP traces and metrics +(issue [#4721](https://github.com/IBM/mcp-context-forge/issues/4721)), and a +local verification stack (Langfuse + OTel Collector + Prometheus) ships as +compose overlays under `docker/`. The full walkthrough — flags, stack setup, +starter PromQL queries, and a debugging checklist — lives in the book: +[Telemetry And Diagnostics](docs/book/src/telemetry-and-diagnostics.md). ## Performance Tests diff --git a/docs/book/README.md b/docs/book/README.md new file mode 100644 index 00000000..ebafcbdf --- /dev/null +++ b/docs/book/README.md @@ -0,0 +1,109 @@ +# Developing The ContextForge Gateway Book + +This directory contains the mdBook source for The ContextForge Gateway Book. +The rendered book also documents its own publishing path in +[Publishing This Book](src/publishing-this-book.md); keep the two in sync when +the workflow or mdBook version changes. + +## Layout + +```text +docs/book/ + book.toml mdBook configuration + README.md contributor notes for this book + src/ + SUMMARY.md chapter order and sidebar structure + *.md rendered book chapters + book/ generated HTML output, ignored by git +``` + +Keep book source in `src/`. Do not edit generated files under `docs/book/book/`. + +## Install mdBook + +The GitHub Pages workflow installs `mdbook v0.5.3`, so local development should +use the same version: + +```bash +cargo install mdbook --version 0.5.3 --locked +``` + +Check the installed version: + +```bash +mdbook --version +``` + +## Render Locally + +Build the static HTML: + +```bash +mdbook build docs/book +``` + +The output is written to: + +```text +docs/book/book/ +``` + +Serve the book with live rebuilds: + +```bash +mdbook serve docs/book --hostname 127.0.0.1 --port 3000 +``` + +Then open: + +```text +http://127.0.0.1:3000 +``` + +Use `--open` if you want mdBook to open the browser: + +```bash +mdbook serve docs/book --hostname 127.0.0.1 --port 3000 --open +``` + +## Validate Changes + +Run these before pushing book changes: + +```bash +mdbook build docs/book +mdbook test docs/book +git diff --check +``` + +`mdbook test` runs Rust code blocks as tests. For prose-only pages, it still +checks that mdBook can parse and walk every chapter in `SUMMARY.md`. + +## Add Or Rename A Chapter + +1. Add the Markdown file under `docs/book/src/`. +2. Add it to `docs/book/src/SUMMARY.md` in the intended reading order. +3. Run `mdbook build docs/book`. +4. Run `mdbook test docs/book`. + +The chapter order in `SUMMARY.md` is the reader's numbered path through the +book. Keep that order intentional. + +## Draft Chapters + +Use this marker for pages that are intentionally present but not implemented: + +```markdown +> Status: draft. To be implemented. +``` + +Follow it with a `## To implement` section and concrete bullets. That keeps the +book navigable while making unfinished work obvious. + +## Publishing + +The workflow at `.github/workflows/pages.yml` builds the book on pull requests +that touch book files and deploys on pushes to `main`. + +Publishing expects GitHub Pages to use `GitHub Actions` as the repository's +Pages source. The workflow uploads `docs/book/book` as the Pages artifact. diff --git a/docs/book/book.toml b/docs/book/book.toml new file mode 100644 index 00000000..2884cafd --- /dev/null +++ b/docs/book/book.toml @@ -0,0 +1,10 @@ +[book] +authors = ["ContextForge Gateway maintainers"] +language = "en" +src = "src" +title = "The ContextForge Gateway Book" +description = "Architecture, configuration, and operations guide for contextforge-gateway-rs, the Rust MCP dataplane that presents many backend MCP servers as one gateway endpoint." + +[output.html] +git-repository-url = "https://github.com/contextforge-gateway-rs/contextforge-gateway-rs" +edit-url-template = "https://github.com/contextforge-gateway-rs/contextforge-gateway-rs/edit/main/docs/book/{path}" diff --git a/docs/book/src/SUMMARY.md b/docs/book/src/SUMMARY.md new file mode 100644 index 00000000..5b42dcad --- /dev/null +++ b/docs/book/src/SUMMARY.md @@ -0,0 +1,30 @@ +# The ContextForge Gateway Book + +- [🌉 What is ContextForge Gateway?](what-is-contextforge-gateway.md) +- [🚀 Getting Started](usage.md) + - [Run the Gateway Locally](running-the-gateway.md) + - [Configuration Reference](gateway-options.md) +- [🏗️ Architecture](architecture.md) + - [System Shape](system-shape.md) + - [Request Flow](request-flow.md) + - [Concurrency And Runtime Model](concurrency-and-runtime.md) + - [Authentication And User Config Lookup](authentication-and-user-config.md) + - [Security Model And Trust Boundaries](security-model.md) + - [Runtime Configuration](runtime-configuration.md) + - [Control-Plane Integration](control-plane-integration.md) + - [Backend Connections And Transports](backend-connections-and-transports.md) + - [Session Ownership](session-ownership.md) + - [Architectural Choices](architectural-choices.md) +- [🔌 MCP Behavior](mcp-behavior.md) + - [MCP Method Reference](mcp-method-reference.md) + - [MCP Routing Semantics](mcp-routing-semantics.md) +- [🧭 Operations](operations.md) + - [Plugins And Policy](plugins-and-policy.md) + - [Telemetry And Diagnostics](telemetry-and-diagnostics.md) + - [Failure Modes](failure-modes.md) + - [Testing](testing.md) + - [Performance](performance.md) + - [Deployment Notes](deployment-notes.md) +- [🛠️ Project](project.md) + - [Contributing To The Gateway](contributing.md) + - [Publishing This Book](publishing-this-book.md) diff --git a/docs/book/src/architectural-choices.md b/docs/book/src/architectural-choices.md new file mode 100644 index 00000000..608d0ecb --- /dev/null +++ b/docs/book/src/architectural-choices.md @@ -0,0 +1,166 @@ +# Architectural Choices + +> 🧱 **Design rule:** these choices are not permanent, but changing one should +> be a deliberate architecture decision with code, tests, and migration notes. + +![Architectural choices](assets/architectural-choices.svg) + +This page records the choices that should stay visible as the gateway evolves. +They describe the shape of the current Rust dataplane, not just preferences. + +## Choice Matrix + +| Choice | Current decision | Why it matters | +| --- | --- | --- | +| Dataplane, not control plane | This repo consumes runtime config and handles traffic. It does not own UI, IAM lifecycle, management APIs, or durable observability storage. | Keeps the hot path small and prevents product workflows from leaking into request routing. | +| Config access is abstracted | MCP routing depends on `UserConfig`, `VirtualHost`, and `UserConfigStore`, not Redis commands. | Keeps future xDS/gRPC or another config stream possible. | +| Backend names are public | The backend map key is part of tool/resource/prompt names. | Backend renames are client-visible behavior changes. | +| Sessions are local today | Backend RMCP services live in `BackendTransports` inside one process. | Load-balanced deployments need sticky routing or a new session ownership design. | +| Merged MCP semantics define the contract | The client sees one gateway MCP server with namespaced backend objects. | Backend topology should not become a hard client dependency beyond the namespace contract. | +| Plugin boundaries stay explicit | Tool pre/post hooks are integrated at known points around backend invocation. | Payload mutation needs clear failure, timeout, cancellation, and telemetry behavior. | + +## Dataplane, Not Control Plane + +The gateway should enforce decisions already made elsewhere: + +```text +control plane authors config and policy + -> Redis/config transport exposes runtime data + -> Rust dataplane enforces it on MCP traffic +``` + +New features should start with one question: + +```text +Is this hot-path enforcement, or is this a management workflow? +``` + +If it is a workflow, it probably belongs outside this repo. The Rust gateway +should enforce the result of management decisions, not become the place where +those decisions are authored. + +## Config Access Is Abstracted + +Redis is the current storage and transport adapter. It is not the routing +model. The routing model is: + +```text +UserConfig + -> VirtualHost + -> BackendMCPGateway +``` + +That is why request code should stay behind `UserConfigStore`. Redis key +encoding, MessagePack, cache expiry, and retry settings belong in the adapter, +not in MCP method handling. + +## Backend Names Are Public + +Backend names are visible in the MCP namespace: + +```text +backend map key: gateway-one +backend tool: increment +gateway tool: gateway-one-increment +``` + +The map key, not `BackendMCPGateway.name`, is the namespace used by current +routing. Any future aliasing, filtering, or prettier naming scheme needs a +migration plan because clients may already refer to these prefixed names. + +## Session State Is Local Today + +Backend services are not just data. They are live RMCP running services stored +under: + +```text +principal + backend_name + downstream_session_id +``` + +That makes the current state model fast and direct, but not horizontally +portable. Do not design request handling as if every node can serve every +stateful MCP session until backend service ownership has an external owner or +can be rebuilt safely. + +## Merged MCP Semantics Define The Contract + +The downstream client should reason about one MCP server: + +```text +client + -> ContextForge Gateway + -> merged tools/resources/prompts +``` + +Backend identity appears through namespaced objects, but clients should not +need to know transport details, Redis storage, fanout mechanics, or plugin +runtime internals. + +This choice leaves room for filtering, policy, and route changes without +turning every backend topology change into a client integration change. + +## Plugin Boundaries Stay Explicit + +Plugins can inspect or mutate payloads. That is more powerful than a header +filter, so the hook points need to stay obvious. + +Current CPEX support is intentionally narrow: + +| Hook | Current boundary | +| --- | --- | +| `TOOL_PRE_INVOKE` | Runs before `call_tool` forwards to the selected backend. | +| `TOOL_POST_INVOKE` | Runs after `call_tool` receives a backend result and on backend progress events. | + +Avoid adding ad hoc plugin calls in the middle of routing code. If a new hook +is needed, define its ownership, failure behavior, timeout behavior, +cancellation behavior, streaming behavior, and telemetry attribution. + +## Transport Security Is Split + +Downstream TLS is listener-level process config. Upstream backend security is +also process config today, but some of it is naturally backend-specific. + +Keep this split visible: + +| Concern | Stable owner | +| --- | --- | +| Gateway listener certificate | Process config. | +| JWT verification keys | Process config. | +| Backend URL, auth headers, pass-through policy, allowed objects | Runtime user config. | +| Backend-specific trust and client identity | Likely runtime config or referenced secret material over time. | + +The gateway should not bury transport security decisions inside MCP method +handlers. They should remain either startup assembly or explicit backend +transport construction. + +## MCP-First, Not MCP-Only + +The current code implements MCP behavior, but the shell is broader: + +```text +auth +config lookup +transport setup +plugin runtime +telemetry +session strategy +``` + +Keep protocol-neutral concerns reusable. Future A2A or model-provider routing +should be able to reuse the gateway shell without copying the MCP routing +stack. + +## When A Choice Changes + +Changing one of these choices should update more than one file. + +| Change | Expected follow-through | +| --- | --- | +| Backend namespace changes | Update merge logic, split logic, tests, docs, and migration notes. | +| Session state moves external | Update `SessionManager`, cleanup behavior, load-balancing docs, and failure-mode tests. | +| Config transport changes | Keep `UserConfigStore` as the boundary and update adapter tests. | +| Plugin hook surface expands | Document ordering, failure behavior, cancellation, streaming, and telemetry. | +| New protocol joins the gateway | Keep shared shell code protocol-neutral and isolate protocol-specific routing. | + +These pages are part of that safety net: they make architecture drift visible +before it becomes accidental API behavior. diff --git a/docs/book/src/architecture.md b/docs/book/src/architecture.md new file mode 100644 index 00000000..7511f8a0 --- /dev/null +++ b/docs/book/src/architecture.md @@ -0,0 +1,25 @@ +# Architecture + +This section explains how the gateway is put together and why the main +boundaries exist. + +> 🧭 **Read this when changing the hot path.** The architecture pages keep +> request handling, config lookup, backend sessions, transports, and +> control-plane boundaries explicit. + +The pages are ordered for a first read: start at the top for the big picture, +then work down into each boundary. If you are changing one area, jump straight +to its page. + +| Page | What it covers | +| --- | --- | +| 🧭 [System Shape](system-shape.md) | The gateway's role in ContextForge, its crate layout, and the line between dataplane and control plane. | +| 🔀 [Request Flow](request-flow.md) | The ordered path from downstream HTTP request to backend MCP call and merged response. | +| 🧵 [Concurrency And Runtime Model](concurrency-and-runtime.md) | Executor shapes, shared state and locks, fanout, cancellation, and the allocator. | +| 🔐 [Authentication And User Config Lookup](authentication-and-user-config.md) | How JWT claims, Redis-backed user config, and virtual host selection combine before routing. | +| 🔒 [Security Model And Trust Boundaries](security-model.md) | What the gateway trusts, what compromise of each boundary means, and transport security posture. | +| 🗂️ [Runtime Configuration](runtime-configuration.md) | The current `UserConfig` model, MessagePack Redis persistence, cache behavior, and expected growth. | +| 🤝 [Control-Plane Integration](control-plane-integration.md) | The current, still-provisional integration surface: Redis keys, schemas, token shape, and route parity. | +| 🔌 [Backend Connections And Transports](backend-connections-and-transports.md) | Downstream listeners, upstream RMCP transports, config-store transport, and TLS direction. | +| 🧵 [Session Ownership](session-ownership.md) | How backend services are keyed, shared, cleaned up, and constrained by local process ownership. | +| 🧱 [Architectural Choices](architectural-choices.md) | The main tradeoffs behind dataplane scope, namespacing, config boundaries, and future protocols. | diff --git a/docs/book/src/assets/architectural-choices.svg b/docs/book/src/assets/architectural-choices.svg new file mode 100644 index 00000000..ff098acf --- /dev/null +++ b/docs/book/src/assets/architectural-choices.svg @@ -0,0 +1,83 @@ + + Architectural choices + The gateway keeps control plane concerns out, abstracts config access, treats backend names as public namespace, keeps sessions local today, exposes merged MCP semantics, and keeps plugin boundaries explicit. + + + + + + + + + Rust gateway dataplane + hot-path MCP enforcement + auth, config lookup, routing, hooks, telemetry + + + + + Not control plane + consume config, do not author it + + + Config behind traits + Redis is adapter, not model + + + Backend names public + map key becomes MCP namespace + + + + + Merged MCP contract + one gateway server view + + + Sessions local today + sticky routing or redesign needed + + + Explicit plugin hooks + known ordering and failure behavior + + + + + + + + + + + + + + + + + + + + + + + + + Changing a choice requires code, tests, docs, and migration notes. + + diff --git a/docs/book/src/assets/auth-user-config.svg b/docs/book/src/assets/auth-user-config.svg new file mode 100644 index 00000000..ce67cd20 --- /dev/null +++ b/docs/book/src/assets/auth-user-config.svg @@ -0,0 +1,122 @@ + + Authentication and user config lookup + The gateway extracts the virtual host id, validates a bearer JWT into ContextForgeClaims, reads the MCP session id if present, loads UserConfig by JWT subject through the cache and Redis, rejects unknown virtual hosts with 404, and exposes typed extensions to MCP validators. + + + + + + + + + HTTP request + path + bearer JWT + optional Mcp-session-id + + + + + virtual_host_id_layer + insert VirtualHostId + + + claims_layer + validate issuer, audience, exp + insert ContextForgeClaims + + + session_id_layer + insert SessionId if present + + + + + user_config_store_layer + claims.sub -> User::new + load and insert UserConfig + + + + + LRU cache + 50k entries, 60s default + keyed by claims.sub string + + + Redis + MessagePack key/value + User -> UserConfig + + + + + MCP validators + VirtualHostId + UserConfig + SessionId when present + + + + + + + + + + + + + + claims + + + + + + + + + + + cache hit + + + + + + cache miss + + + + + + + + 401 auth + + + + 400/500 config + + + diff --git a/docs/book/src/assets/backend-transports.svg b/docs/book/src/assets/backend-transports.svg new file mode 100644 index 00000000..6964793d --- /dev/null +++ b/docs/book/src/assets/backend-transports.svg @@ -0,0 +1,99 @@ + + Backend connections and transports + The gateway separates downstream TCP or TLS listener transport, upstream reqwest and RMCP streamable HTTP backend transport, and Redis config-store transport. + + + + + + + + + MCP client + streamable HTTP + front door or direct caller + + + + + Gateway dataplane + + + downstream listeners + TCP and optional Rustls TLS + + + Axum + RMCP service + middleware, validators, routing + + + upstream client boundary + reqwest + streamable HTTP + + + + + Redis config store + plain, TLS, or mTLS + UserConfig and plugin config + + + Backend MCP servers + streamable HTTP today + HTTPS-only by default + mTLS supported by process config + + + + + + TCP/TLS + + + + config lookup + + + + UserConfig + + + + reqwest client + + + + MCP response + + + + + Boundary rule: + listener setup, backend transport creation, and Redis config access should stay in separate modules. + + diff --git a/docs/book/src/assets/gateway-overview.svg b/docs/book/src/assets/gateway-overview.svg new file mode 100644 index 00000000..b7c523c1 --- /dev/null +++ b/docs/book/src/assets/gateway-overview.svg @@ -0,0 +1,88 @@ + + ContextForge Gateway request and response path + An MCP client sends streamable HTTP traffic to the ContextForge Gateway. The gateway validates the caller, loads runtime configuration, calls backend MCP servers, receives backend responses, merges them, and returns one MCP response or stream to the client. + + + + + + + + + MCP client + streamable HTTP + JWT + session id + + + + + + MCP request + + + + merged response + + + + + ContextForge Gateway + + + validate caller + + + load runtime config + + + route and merge MCP methods + + + + + Redis runtime config by JWT subject + + + + + + + + backend calls + + + + backend responses + + + + + Backend MCP servers + + + gateway-one + + + gateway-two + + + more backends + + + + + one logical MCP server downstream + + diff --git a/docs/book/src/assets/request-flow.svg b/docs/book/src/assets/request-flow.svg new file mode 100644 index 00000000..71c0bc32 --- /dev/null +++ b/docs/book/src/assets/request-flow.svg @@ -0,0 +1,230 @@ + + ContextForge Gateway request flow + A normal MCP HTTP request enters the TCP or TLS listener, passes through metrics, tracing, the contextforge nested router, CORS, virtual host extraction, claims validation, session extraction, user config lookup, a virtual host config check, and RMCP. RMCP then follows either initialize handling or authorized MCP method handling, calls backend MCP servers as needed, and returns the response through the same stack. + + + + + + + + + MCP client + streamable HTTP + + + TCP/TLS + listener transport + + + Metrics + HttpMetricsLayer + + + Tracing + TraceLayer + + + Nested router + /contextforge-rs + + + + + + + + + + + + + + Inner Axum request order + normal MCP request after /contextforge-rs nesting + + + CORS layer + may answer preflight before MCP handling + + + virtual_host_id_layer + extracts /servers/{virtual_host_id}/mcp + inserts VirtualHostId + + + claims_layer + validates Authorization: Bearer token + inserts ContextForgeClaims + + + session_id_layer + reads Mcp-session-id when present + inserts SessionId for authorized calls + DELETE cleanup happens on successful response + + + user_config_store_layer + loads UserConfig for claims.sub + inserts UserConfig; unknown virtual host gets 404 + + + + + + + + + + + + 400 + + 401 + + 400/404/500 + + + + + + request enters inner router + + + + + RMCP service + StreamableHttpService + creates or reuses McpService + handler reads typed extensions + + + + + + + + initialize path + + + InitializeCallValidator + DownstreamSessionId + UserConfig + VirtualHostId + claims + + + resolve selected VirtualHost + + + read local user session mapping + + + join_all over configured backends + StreamableHttpClientTransport + GatewayBackendClient::serve + + + set session mapping, then store BackendTransports + + + + + authorized MCP calls + + + AuthorizedCallValidator + SessionId + UserConfig + VirtualHostId + claims + + + list_tools / list_resources / list_prompts + borrow transports -> fan_out_list -> namespace + sort + + + call_tool + split prefix -> resolve backend -> hooks + backend call -> response + + + read_resource / get_prompt / complete + split prefix -> resolve backend -> strip prefix -> call + + ping, subscribe, unsubscribe are local today + + + + + + initialize + + + + post-init calls + + + + + Backend + MCP + servers + initialize + list/call/read/get + progress events + + + + backend init + + + + capabilities + + + + routed call + + + + result/progress + + + + + HTTP response unwinds stack + successful DELETE removes local session and backend transports + + + + + + + + + response to client + + diff --git a/docs/book/src/assets/runtime-config.svg b/docs/book/src/assets/runtime-config.svg new file mode 100644 index 00000000..c22d4a23 --- /dev/null +++ b/docs/book/src/assets/runtime-config.svg @@ -0,0 +1,115 @@ + + Runtime configuration surfaces + Process configuration is loaded at startup. UserConfig is loaded per request by JWT subject and virtual host id. RuntimePluginConfigDocument is loaded from Redis when CPEX plugins are enabled and can be reloaded by the watcher. + + + + + + + + + Process Config + CLI + env at startup + listeners, JWT keys, Redis, TLS + telemetry, runtime shape, plugins + + + + + UserConfig + Redis MessagePack + key: User::new(claims.sub) + value: UserConfig + + + + + Plugin Config + RuntimePluginConfigDocument + JSON or MessagePack + CPEX pre/post tool hooks + + + + + Gateway dataplane + + + startup assembly + + + request middleware + claims.sub -> UserConfig + + + MCP validators + VirtualHostId -> VirtualHost + + + optional CPEX runtime + + + + + Selected VirtualHost + backend map key is namespace + backend URL builds upstream transport + + + CPEX Runtime + loaded at startup + watcher reloads every 10 minutes + + + + + + startup + + + + per request + + + + select route + + + + plugin document + + + + active hooks + + + + reload swaps runtime state + + diff --git a/docs/book/src/assets/session-ownership.svg b/docs/book/src/assets/session-ownership.svg new file mode 100644 index 00000000..67783238 --- /dev/null +++ b/docs/book/src/assets/session-ownership.svg @@ -0,0 +1,106 @@ + + Session ownership + Initialize creates local backend MCP running services keyed by principal, backend name, and downstream session id. Later calls use Mcp-session-id to find them, and DELETE removes local state. + + + + + + + + + MCP client + initialize + then Mcp-session-id + + + + + Gateway process + + + RMCP LocalSessionManager + creates downstream session id + + + SessionManager + principal + session + virtual host + + + BackendTransports + principal + backend + session + Arc<RunningService> + + + + + LocalUserSessionStore + local LRU mapping + Redis store exists, not wired by default + + + Backend MCP services + one running service per backend + local process ownership + + + + + DELETE cleanup removes local entries + + + + + + initialize + + + + session mapping + + + + create services + + + + store handles + + + + later calls + + + + successful DELETE + + diff --git a/docs/book/src/assets/system-shape.svg b/docs/book/src/assets/system-shape.svg new file mode 100644 index 00000000..8daebed6 --- /dev/null +++ b/docs/book/src/assets/system-shape.svg @@ -0,0 +1,105 @@ + + ContextForge Gateway system shape + The ContextForge control plane writes runtime config into Redis. MCP clients call the Rust gateway dataplane. The gateway loads config, validates identity, calls backend MCP servers, receives backend responses, merges them, and returns one downstream response. + + + + + + + + + Control plane + management, UI, IAM + config and policy authoring + + + + + MCP client + streamable HTTP + JWT + MCP session + + + + + Rust gateway dataplane + one logical MCP server downstream + + + Axum listener stack + + + auth + request context + + + UserConfig lookup + + + MCP fanout + merge + + + hooks + telemetry + + + + + Redis config + UserConfig by JWT subject + plugin runtime config + + + + + Backend MCP + servers + tools, resources, prompts + per-backend sessions + + + + + + writes runtime config + + + + loads config + + + + + + MCP request + + + + merged response + + + + + + backend calls + + + + backend responses + + diff --git a/docs/book/src/authentication-and-user-config.md b/docs/book/src/authentication-and-user-config.md new file mode 100644 index 00000000..13b42a78 --- /dev/null +++ b/docs/book/src/authentication-and-user-config.md @@ -0,0 +1,142 @@ +# Authentication And User Config Lookup + +> 🔐 **Boundary:** authentication proves who is calling. User config lookup +> decides which virtual hosts and backends that caller can reach. + +![Authentication and user config lookup](assets/auth-user-config.svg) + +This page follows the identity boundary in the request path. The gateway does +not let an MCP method choose arbitrary backend URLs. It validates the bearer +token, loads the caller's `UserConfig`, and only then lets MCP validators select +a virtual host from that config. + +## Request Order + +Authentication and config lookup happen before `McpService` handles the MCP +method: + +| Step | Code | Output | +| --- | --- | --- | +| Path context | `virtual_host_id_layer` | `VirtualHostId` extension. | +| JWT validation | `claims_layer` | `ContextForgeClaims` extension. | +| Session header | `session_id_layer` | Optional `SessionId` extension. | +| Config lookup | `user_config_store_layer` | `UserConfig` extension. | +| Virtual host check | `virtual_host_config_layer` | `404` when the path's virtual host id is not in the loaded config. | +| MCP validation | `InitializeCallValidator` or `AuthorizedCallValidator` | Selected `VirtualHost`, session id, and claims. | + +The order matters: `user_config_store_layer` needs `ContextForgeClaims`, and MCP +validators need both `UserConfig` and `VirtualHostId`. + +## Token Validation + +`claims_layer` reads `Authorization: Bearer ...` and decodes the JWT with the +algorithm declared in the JWT header. + +| Token property | Current behavior | +| --- | --- | +| Algorithm | Accepts RS256/RS384/RS512 when an RSA public key is configured, or HS256/HS384/HS512 when a shared secret is configured. | +| Issuer | Must match `mcpgateway`. | +| Audience | Must match `mcpgateway-api`. | +| Expiration | `exp` is validated. | +| Unsupported algorithm | Rejected before claims are inserted. | + +The decoded value is stored as `ContextForgeClaims`. The fields currently +important to routing are: + +| Claim | Routing role | +| --- | --- | +| `sub` | Becomes the user config key and the principal for backend session lookup. | +| `iss`, `aud`, `exp` | Authentication checks only. | +| `jti`, `token_use`, `iat`, `teams`, `user`, `scopes` | Carried in claims for future policy use; not currently used by MCP routing. `token_use`, `iat`, `teams`, and `scopes` are optional, as is `user.full_name`, so tokens without those fields still validate. | + +A concrete decoded payload for the local `admin@example.com` subject looks like +this (timestamps shown as example Unix seconds). Of everything here, MCP routing +depends only on `sub` today. The optional fields are included for illustration: + +```json +{ + "iss": "mcpgateway", + "aud": "mcpgateway-api", + "sub": "admin@example.com", + "exp": 1717180800, + "iat": 1717177200, + "jti": "example-token", + "token_use": "api", + "teams": ["team_awesome"], + "user": { + "email": "admin@example.com", + "full_name": "API Token User", + "is_admin": true, + "auth_provider": "api_token" + }, + "scopes": { + "server_id": "my_id", + "permissions": ["tools.read", "servers.use"], + "ip_restrictions": ["192.169.1.0/24"], + "time_restrictions": null + } +} +``` + +## User Config Key + +`user_config_store_layer` turns the subject into a typed key: + +```text +ContextForgeClaims.sub + -> User::new(subject) + -> UserConfigStore::get_config(&user) +``` + +The Redis adapter serializes that `User` key with MessagePack. The key includes +both the key type and the subject, so user config data is not just stored under +the raw subject string. + +## Cache And Redis Lookup + +`RedisUserConfigStore` checks an in-process LRU cache before going to Redis: + +| Stage | Behavior | +| --- | --- | +| LRU hit | Clone the decoded `UserConfig` from the cache. | +| LRU miss | MessagePack-encode `User::new(subject)`, `GET` that Redis key, decode the MessagePack `UserConfig`, then cache it. | +| Cache size | 50,000 entries. | +| Cache expiry | `--user-config-cache-expiry-seconds`, default 60 seconds. `0` disables the cache and reads Redis on every request. | +| Redis retry setting | Connection manager is configured with 1,000 retries. | + +The cache is an implementation detail of `RedisUserConfigStore`. Routing code +depends on `UserConfigStore`, not Redis commands. + +## Failure Behavior + +Failures before RMCP method handling are HTTP responses: + +| Failure | Response | +| --- | --- | +| Missing `Authorization` header | `401 Unauthorized`. | +| Header does not start with `Bearer ` | `401 Unauthorized`. | +| JWT header or body cannot be decoded | `401 Unauthorized`. | +| JWT uses an unsupported algorithm | `401 Unauthorized`. | +| Required decoder key or secret is not configured | `401 Unauthorized`. | +| No user config exists for `claims.sub` | `400 Bad Request`. | +| Redis/config store error other than missing data | `500 Internal Server Error`. | +| `user_config_store_layer` runs without claims | `400 Bad Request`. | +| Virtual host id not present in the caller's config | `404 Not Found` with body `{"detail":"Server not found"}`. | + +A valid user config can still fail a request: `virtual_host_config_layer` +returns `404` before MCP method handling when the config does not contain the +path's `VirtualHostId`. + +## What This Boundary Does Not Do + +Authentication and user config lookup do not route to a backend by themselves. +They only establish: + +```text +caller identity + + caller UserConfig + + requested VirtualHostId +``` + +`McpService` still has to validate the MCP call, resolve the virtual host, and +choose either the initialize path, routed backend path, or local method path. diff --git a/docs/book/src/backend-connections-and-transports.md b/docs/book/src/backend-connections-and-transports.md new file mode 100644 index 00000000..8f30bd98 --- /dev/null +++ b/docs/book/src/backend-connections-and-transports.md @@ -0,0 +1,130 @@ +# Backend Connections And Transports + +> 🚚 **Transport boundary:** downstream listener traffic, upstream backend +> traffic, and config-store traffic are separate concerns. Keep them separate +> even when they all use TCP underneath. + +![Backend connections and transports](assets/backend-transports.svg) + +The gateway has three transport classes on the hot path. They are built in +different modules, configured from different fields, and serve different +architecture roles. + +## Transport Classes + +| Transport class | Current implementation | Main owner | Purpose | +| --- | --- | --- | --- | +| Downstream listener | Axum/Hyper over TCP and optional Rustls TLS. | `transports/` and `Gateway::run_gateway`. | Accept MCP streamable HTTP traffic from clients or the front door. | +| Upstream backend | Shared `reqwest::Client` plus RMCP `StreamableHttpClientTransport`. | `common.rs` and `gateway/mcp_gateway.rs`. | Open MCP client sessions to configured backend MCP servers. | +| Config store | Redis plain, TLS, or mTLS connection manager. | `common.rs` and `user_config_store/`. | Load `UserConfig` and plugin runtime config from control-plane authored storage. | + +The current MCP dataplane only uses streamable HTTP for backend MCP traffic. +`BackendMCPGateway.transport` already has `STREAMABLEHTTP`, `SSE`, and `STDIO`, +but upstream routing does not branch on that field yet. + +## Downstream Listeners + +`Gateway::run_gateway` builds one Axum router and can expose it through TCP, +TLS, or both. + +| Listener | Config fields | Behavior | +| --- | --- | --- | +| TCP | `address` | Binds a Tokio `TcpSocket`, sets reuse options and keepalive, listens with backlog `1024`, and serves Axum with graceful shutdown on `ctrl_c`. | +| TLS | `tls_address`, `server_certificate`, `server_private_key` | Builds a Rustls server config, accepts TLS by hand, then serves the same Axum router through Hyper. | + +TLS listener setup has two important constraints: + +| Constraint | Why | +| --- | --- | +| `tls_address` requires both certificate and private key. | The listener cannot build a Rustls server config without both. | +| `tls_address` cannot equal `address`. | TCP and TLS cannot bind the same socket in this process. | + +The downstream TLS listener currently uses `with_no_client_auth()`. Client +identity is established by the gateway's bearer JWT layer, not by downstream +mTLS. + +## Upstream Backend Client + +The upstream HTTP client is built once at gateway startup: + +```text +Config + -> reqwest::Client::try_from(&config) + -> clone per backend initialize task + -> StreamableHttpClientTransport::with_client(...) +``` + +The process-level upstream mode controls whether backend URLs may use plain +HTTP, HTTPS, or HTTPS with client identity: + +| Mode | `reqwest` behavior | +| --- | --- | +| unset | `https_only(true)`. Same as `TlsOnly`. | +| `TlsOnly` | HTTPS backends only. | +| `PlainTextOrTls` | HTTP or HTTPS backends. | +| `PlainTextOrMTls` | HTTP or HTTPS backends, with a client identity configured for TLS handshakes. | +| `MtlsOnly` | HTTPS backends only, with a client identity configured for TLS handshakes. | + +If `upstream_trust_bundle` is configured, the PEM bundle is merged into the +client's TLS trust roots. For mTLS modes, the upstream certificate and private +key are read from disk and combined into a `reqwest::Identity`. + +## Backend MCP Transport + +During `initialize`, the selected virtual host fans out to every configured +backend: + +```text +VirtualHost.backends + -> for each backend URL + -> build StreamableHttpClientTransportConfig + -> serve GatewayBackendClient over StreamableHttpClientTransport + -> store running service in BackendTransports +``` + +For HTTPS backend URLs, the gateway also sets a custom `Host` header from the +backend URL host and optional port. HTTP backend URLs do not get this custom +header in the current code. + +Backend connection failures are not fatal to the whole initialize call. The +gateway stores the backend entry with no running service, so list calls can +continue with available backends and routed calls to that backend can fail +locally. + +## Config-Store Transport + +Redis is the current config-store transport. It is used for user config and, +when runtime plugins are enabled, plugin runtime config. + +| Redis mode | Connection behavior | +| --- | --- | +| `PlainText` | Connects to `host:port` over TCP. | +| `Tls` | Connects with `rediss://host:port` and a required trust bundle. | +| `Mtls` | Connects with `rediss://host:port`, required trust bundle, required client certificate, and required client key. | + +The Redis user config adapter stores: + +```text +MessagePack(User::new(claims.sub)) -> MessagePack(UserConfig) +``` + +The Redis connection manager is configured with `1,000` retries. The adapter +keeps an in-process LRU cache in front of Redis, but routing code should only +depend on the `UserConfigStore` trait. + +## What Should Move To Runtime Config + +Transport security is mostly process config today. That keeps startup simple, +but it is not the final shape for every backend-specific decision. + +| Setting | Today | Better long-term owner | +| --- | --- | --- | +| Downstream TLS certificate | Process config. | Process config. It belongs to the gateway listener. | +| Upstream trust bundle and mTLS identity | Process config. | Runtime config per backend or referenced secret material. | +| Backend auth headers | Not applied from `UserConfig` yet. | Runtime config per backend. | +| Backend transport type | Model field exists, not routed yet. | Runtime config per backend. | +| Header pass-through policy | Model field exists, not enforced yet. | Runtime config per backend or route policy. | + +The boundary to preserve is simple: listener code should not know Redis schema, +MCP routing code should not know Redis command details, and backend transport +creation should stay behind a small, explicit upstream boundary. diff --git a/docs/book/src/concurrency-and-runtime.md b/docs/book/src/concurrency-and-runtime.md new file mode 100644 index 00000000..31d9ba6e --- /dev/null +++ b/docs/book/src/concurrency-and-runtime.md @@ -0,0 +1,64 @@ +# Concurrency And Runtime Model + +> 🧵 **Execution lens:** the gateway is async Rust on Tokio with jemalloc as +> the global allocator. This page explains the two executor shapes, what +> state is shared under which locks, and where work fans out. + +## Executor Shapes + +`--single-runtime` selects between two models: + +| Mode | Shape | When | +| --- | --- | --- | +| `true` (default) | One multi-thread Tokio runtime with `--number-of-cpus` worker threads (default: host CPU count). All connections share one runtime and one set of gateway state. | The default for all stateful MCP traffic. | +| `false` | One OS thread per CPU, each running its own current-thread Tokio runtime, each executing the full gateway stack. Listeners bind with `SO_REUSEPORT`, so the kernel spreads incoming connections across the per-thread listeners. | A shared-nothing, per-core experiment shape for throughput work. | + +In multi-runtime mode, the first thread initializes the optional CPEX plugin +runtime before the others start; the current-thread builders are tuned with a +global queue interval of `1024` and `4` I/O events per tick. + +> ⚠️ **Multi-runtime consequence:** each runtime thread builds its own +> `BackendTransports` map and user-session store inside `run_gateway`. +> Backend session state is therefore per-runtime-thread, and `SO_REUSEPORT` +> gives no connection affinity — later requests in a streamable HTTP session +> arrive on new connections and can land on a thread that does not own the +> session. Treat single-runtime mode as the only mode that supports stateful +> MCP sessions today; this is the in-process version of the +> [load-balancing constraint](session-ownership.md#load-balancing-consequence). + +## Shared State And Locks + +| State | Lock | Contention profile | +| --- | --- | --- | +| `BackendTransports` map | `Arc>>` | Locked briefly on initialize insert, per-call borrow, and cleanup. Borrowing clones `Arc` handles so the lock is not held across backend calls. | +| Subscription set | `Arc>>` | Local `subscribe`/`unsubscribe` only. | +| User config LRU cache | `Arc>` inside `RedisUserConfigStore` | One lock per config lookup on the hot path; misses add a Redis round trip. | +| User session LRU cache | Same pattern in `LocalUserSessionStore` | Initialize and delete paths. | +| JWT decoders, upstream `reqwest::Client`, process `Config` | No lock — immutable after startup, shared by `Arc`/clone. | None. | + +The design rule: locks guard maps of handles, not I/O. Backend calls, Redis +reads, and plugin hooks all run outside any gateway lock. + +## Fanout And Cancellation + +- `initialize` opens one backend transport per configured backend + concurrently (`futures::future::join_all`); a failed backend degrades that + backend only. +- List methods fan out to all connected backends concurrently and merge. +- Targeted calls resolve exactly one backend service handle. +- `call_tool` watches the downstream cancellation token and forwards a cancel + to the backend if the client gives up first; backend progress notifications + are forwarded downstream while the call is in flight. + +## Listener Behavior + +The TCP listener binds with `reuseaddr`, `reuseport`, and keepalive, listens +with a backlog of `1024`, and serves Axum with graceful shutdown on `ctrl_c`. +The TLS listener accepts by hand through Rustls and serves the same router +via Hyper. + +## Allocator + +The binary sets `tikv_jemallocator` as the global allocator, which holds up +better than the system allocator under the many small, short-lived +allocations of per-request JSON and header processing. diff --git a/docs/book/src/contributing.md b/docs/book/src/contributing.md new file mode 100644 index 00000000..7a124e1b --- /dev/null +++ b/docs/book/src/contributing.md @@ -0,0 +1,73 @@ +# Contributing To The Gateway + +> 🛠️ **Contribution rule:** put behavior in the crate that owns it, keep the +> client-visible contracts stable, and update the matching book page in the +> same change. + +## Where Changes Belong + +| Change | Home | +| --- | --- | +| Dataplane behavior: routing, middleware, sessions, transports | `contextforge-gateway-rs-lib` — almost everything goes here. | +| Process shell: CLI flags, logging, runtime shape, exporters | `contextforge-gateway-rs` (the binary crate). Do not add dataplane logic here. | +| Shared config shapes (`UserConfig`, `User`, plugin config document) | `contextforge-gateway-rs-apis`. Regenerate the JSON schemas after any change: `cargo run -p contextforge-gateway-rs-apis` (see [Control-Plane Integration](control-plane-integration.md)). | +| Plugin integration | `contextforge-gateway-rs-cpex`. | +| Load generation | `contextforge-load-test`. | + +Inside the library crate, keep the module boundaries from +[System Shape](system-shape.md#module-boundaries): config validation in +`common.rs`, extension extraction in `layers/`, MCP behavior in `gateway/`, +listeners in `transports/`, Redis details behind `UserConfigStore`. + +## Changing MCP Routing + +The backend prefix namespace is a client-visible contract +([MCP Routing Semantics](mcp-routing-semantics.md)). Any change to it must +update the merge logic, the split logic, and the tests in the same PR — and +the [Control-Plane Integration](control-plane-integration.md) if the +client-facing surface moves. + +The project is still early, with no external users: prefer the right +architecture over preserving unstable APIs or compatibility surfaces. + +## Adding Plugin Hooks + +New hook points need defined behavior for failure, timeout, cancellation, +streaming, and telemetry attribution before they land on the hot path — see +[Plugins And Policy](plugins-and-policy.md). Avoid ad hoc plugin calls in the +middle of routing code. + +## Validation + +The pre-commit hooks run these local gates: + +```bash +cargo fmt --all --check +cargo clippy --locked --workspace --all-targets -- -D warnings +cargo deny check advisories licenses +cargo nextest run --locked --workspace +cargo build --locked --workspace +cargo bench --locked --workspace --no-run +``` + +CI runs the same gates and additionally runs +`cargo shear --check-test-targets --deny-warnings --locked`. + +Expectations by change type: + +| Change type | Minimum validation | +| --- | --- | +| Docs only | `mdbook build docs/book` and `mdbook test docs/book`. | +| Routing or session behavior | New or updated integration tests under `crates/contextforge-gateway-rs-lib/tests/` against the mock backends. | +| Config shape | Schema regeneration plus a control-plane compatibility check. | +| Plugin behavior | `gateway_plugins.rs` coverage for the new hook path. | +| Performance-sensitive paths | A [load-test run](performance.md) before and after. | + +For end-to-end confidence against the real control plane, run the +[cf-integration lanes](testing.md#full-stack-integration-harness). + +## Keep The Book True + +Every page in this book states verifiable behavior. When a change makes a +page wrong — a flag, a status code, a lock, a boundary — fix the page in the +same PR. Stale architecture docs are worse than none. diff --git a/docs/book/src/control-plane-integration.md b/docs/book/src/control-plane-integration.md new file mode 100644 index 00000000..154b3dfe --- /dev/null +++ b/docs/book/src/control-plane-integration.md @@ -0,0 +1,73 @@ +# Control-Plane Integration + +> 🤝 **Provisional:** no formal contract with the control plane has been +> stipulated yet. This page is a snapshot of the current de facto integration +> surface with +> [IBM/mcp-context-forge](https://github.com/IBM/mcp-context-forge) as +> implemented today. Any row may change while the project is early; when a +> proper contract is agreed, this page should track it. + +## Current Integration Surface + +These are the values both sides currently rely on: + +| Agreement | Value today | +| --- | --- | +| Client-facing route | `/servers/{virtual_host_id}/mcp` behaves like the legacy ContextForge MCP endpoint. The front door rewrites it to `/contextforge-rs/servers/{virtual_host_id}/mcp` on the dataplane. | +| Unknown virtual host | `404` with body `{"detail":"Server not found"}`, matching the control-plane response shape. | +| Token issuer and audience | `iss = mcpgateway`, `aud = mcpgateway-api` — the values the control plane mints. | +| Claims shape | `sub`, `jti`, `iss`, `aud`, `exp`, and `user` are required. `token_use`, `iat`, `teams`, and `scopes` are optional, as is `user.full_name`. The dataplane routes on `sub` only. | +| User config key | MessagePack-encoded `User::new(jwt_subject)` (key type plus subject). | +| User config value | MessagePack-encoded `UserConfig`; the JSON schema is generated into `schemas/user_config.json`. | +| Plugin config key | `ContextForgeGatewayRuntimePluginConfig`, JSON or MessagePack, `version: 1` with a `cpex` section. | + +Changing any of these is a cross-repo change: the dataplane, the control-plane +publisher, and the integration harness all need updating together. + +## Config Publishing + +The control plane owns durable config and publishes runtime snapshots to +Redis. With `DATAPLANE_PUBLISHER=true`, it rewrites the dataplane's +`UserConfig` keys on an interval — every 60 seconds by default, configurable +in newer control-plane images. + +Config staleness on the dataplane is bounded by two knobs: + +```text +worst-case staleness = publisher interval + user config cache expiry +``` + +The dataplane's in-process cache defaults to 60 seconds +(`--user-config-cache-expiry-seconds`; `0` disables it and reads Redis on +every request). Functional test setups shorten the publisher interval and +disable the cache; production keeps both at 60. + +## Schema Generation + +`contextforge-gateway-rs-apis` is the single source of truth for the shared +config shapes. It generates the JSON Schemas the control plane can validate +against: + +```bash +cargo run -p contextforge-gateway-rs-apis +``` + +This writes `schemas/user.json` and `schemas/user_config.json`. Regenerate and +commit them whenever `UserConfig`, `VirtualHost`, `BackendMCPGateway`, or the +`User` key type changes. + +## Front-Door Split + +Only MCP dataplane traffic comes to this process. The repository's reference +`docker/nginx.conf` proxies `location ^~ /contextforge-rs` to the gateway; +all UI, management API, and other ContextForge traffic stays on the existing +control-plane paths. + +## Verifying The Integration + +The [`cf-integration`](https://github.com/contextforge-gateway-rs/cf-integration) +harness tests exactly this surface: it runs the stock upstream control-plane +stack with the nginx split and the dataplane publisher enabled, then drives +probe, live-test, and load lanes through the public route. When the +integration surface changes, its lanes are what prove both sides still agree. +See [Testing](testing.md#full-stack-integration-harness) for the commands. diff --git a/docs/book/src/deployment-notes.md b/docs/book/src/deployment-notes.md new file mode 100644 index 00000000..4293f7f8 --- /dev/null +++ b/docs/book/src/deployment-notes.md @@ -0,0 +1,89 @@ +# Deployment Notes + +> 🏗️ **Deployment lens:** the gateway is one stateless-config, stateful-session +> process behind a front door. Everything here follows from that: route only +> MCP traffic to it, keep Redis close and trusted, and give stateful sessions +> affinity. + +## Front-Door Routing + +The reference `docker/nginx.conf` shows the intended split: + +- `location ^~ /contextforge-rs` proxies to the gateway upstream. +- Everything else — UI, management APIs, other ContextForge traffic — stays on + the existing control-plane paths. +- The reference listener uses `backlog=4096 reuseport` and configures upstream + retries (`error timeout http_502/503/504`, 2 tries, 10 s window). For MCP + `POST` bodies this effectively retries only connection-stage failures: + nginx does not re-send non-idempotent requests once they reached an + upstream, and MCP calls are not idempotent. + +There is no production health endpoint today: `/health` is a `with_tools` +bootstrap helper served at `/contextforge-rs/health`, and production builds +compile it out. Use TCP-level checks or the exported metrics for liveness +until a real health endpoint exists. (The reference nginx config's +`location = /health` predates this and does not match the gateway's route.) + +The [`cf-integration`](https://github.com/contextforge-gateway-rs/cf-integration) +harness runs the same split with the stock upstream control-plane stack and +rewrites public `/servers/{id}/mcp` to `/contextforge-rs/servers/{id}/mcp`. + +## TLS Choices + +| Leg | Options | +| --- | --- | +| Front door to gateway | Plain HTTP on a trusted private network (the common shape behind nginx), or terminate TLS at the gateway with `--tls-address` plus certificate and key. Both listeners can run at once on different sockets. | +| Gateway to Redis | `--redis-mode` plain, TLS, or mTLS. Use TLS/mTLS across trust zones — Redis is the config trust boundary (see [Security Model](security-model.md)). | +| Gateway to backends | HTTPS-only by default; opt into plain HTTP or mTLS with `--upstream-connection-mode`. | + +## Session Affinity And Failover + +Backend MCP sessions are local process state +([Session Ownership](session-ownership.md)): + +- More than one replica requires sticky routing by `Mcp-session-id`; the + reference nginx config does not provide this, so today's safe shapes are a + single replica or a front door that adds stickiness. +- On restart or failover, sessions are gone; clients must re-run + `initialize`. Design clients to treat a session-not-found error as + "reinitialize", not "retry". +- A Redis-backed user session store exists in code, but live backend services + would still be process-local; a remote session story is future work. +- Inside one host, the same constraint applies to `--single-runtime false`; + see [Concurrency And Runtime Model](concurrency-and-runtime.md). + +## Redis Availability + +Redis is required at startup and on every uncached config lookup. The +connection manager retries heavily (1,000 retries) rather than failing fast, +and the in-process cache (default 60 s) rides out short blips for warm +subjects. A cold subject during a Redis outage fails at config lookup; the +current Redis adapter reports failed `GET` calls as missing data, so the layer +returns `400` until Redis returns. + +## Images And Sizing + +- CI builds `docker/Dockerfile` (a `rust:1.96.1` builder stage) on every push + to `main` and pushes `ghcr.io//contextforge-gateway-rs:`, + where the tag is the Cargo package version. There is no `latest` tag — + pin the version. +- The reference Compose stack runs the gateway with raised limits worth + copying to real deployments: `nofile` 65535 and TCP tuning + (`tcp_fin_timeout=15`, widened local port range) for high connection churn. +- Size CPU with `--number-of-cpus` (defaults to host CPU count) and keep the + default single multi-thread runtime for stateful traffic. Memory scales + with active sessions (live backend services) and the config caches (up to + 50,000 entries each). + +## Deployment Checklist + +1. Front door routes only `/contextforge-rs` here. +2. JWT verification key or secret in place and rotated with the control + plane's signing key. +3. Redis reachable, TLS/mTLS across trust zones, write access restricted to + the control plane; `DATAPLANE_PUBLISHER` enabled on the control plane. +4. Upstream connection mode matches the backend URL schemes. +5. One replica per `Mcp-session-id` (single replica or sticky routing). +6. `with_tools` disabled in the production build. +7. Telemetry export pointed at the collector; see + [Telemetry And Diagnostics](telemetry-and-diagnostics.md). diff --git a/docs/book/src/failure-modes.md b/docs/book/src/failure-modes.md new file mode 100644 index 00000000..4b46172e --- /dev/null +++ b/docs/book/src/failure-modes.md @@ -0,0 +1,68 @@ +# Failure Modes + +> 🚧 **Boundary rule:** every failure should come from the layer that owns the +> missing fact. Identity and config failures are HTTP responses before MCP +> handling; routing and backend failures are JSON-RPC errors. + +## HTTP Layer Failures + +These happen in middleware, before any MCP method runs: + +| Failure | Response | Owning layer | +| --- | --- | --- | +| Inner path does not match `/servers/{virtual_host_id}/mcp` | `400 Bad Request` | `virtual_host_id_layer` | +| Missing `Authorization` header or non-`Bearer` scheme | `401 Unauthorized` | `claims_layer` | +| JWT cannot be decoded, uses an unsupported algorithm, or no matching decoder key/secret is configured | `401 Unauthorized` | `claims_layer` | +| Expired token, wrong issuer, or wrong audience | `401 Unauthorized` | `claims_layer` | +| No user config exists for `claims.sub`, or claims are absent | `400 Bad Request` | `user_config_store_layer` | +| Config store error other than missing data | `500 Internal Server Error` | `user_config_store_layer` | +| Virtual host id not present in the caller's config | `404 Not Found` with `{"detail":"Server not found"}` | `virtual_host_config_layer` | + +## MCP Validation Failures + +`InitializeCallValidator` and `AuthorizedCallValidator` re-check the request +context before method handling. These are defense-in-depth errors: in a +healthy stack the middleware has already established the context. + +| Failure | JSON-RPC error | +| --- | --- | +| Missing session id, user config, virtual host id, or claims extension | Internal error (`Routing problem...`). | +| Virtual host absent from the user config | `RESOURCE_NOT_FOUND` with message `No configuration`. Normally unreachable because `virtual_host_config_layer` already returned `404`. | + +## Routing Failures + +| Failure | Behavior | +| --- | --- | +| Prefixed name or completion reference does not start with a configured backend name plus `-` | Internal error (`wrong tool name` / `wrong resource name` / `wrong prompt name` / `wrong completion reference`). | +| No backend entry matches the split name | Internal error (`got no responses from backends`). | +| Backend entry exists but has no running service | Internal error. Happens when that backend failed during `initialize`. | +| More than one backend entry matches | `INVALID_REQUEST`, and the session's backend entries are removed via `cleanup_backends`. | + +## Backend Session Failures + +| Situation | Behavior | +| --- | --- | +| Backend unreachable during `initialize` | The backend is stored with no running service. `initialize` still succeeds with the remaining backends. | +| Backend unreachable during a routed call | The call returns an internal error; other backends are unaffected. | +| Gateway process restart | All backend session state is lost because it is local process state. Clients must re-run `initialize`. | +| Request lands on a gateway node that does not own the session | List calls return empty results and routed calls fail, because `BackendTransports` has no entries there. Stateful sessions need sticky routing; see [Session Ownership](session-ownership.md). | + +## Plugin Failures + +| Failure | Behavior | +| --- | --- | +| Plugin denies a tool call or response | The denial becomes an MCP error to the caller. | +| Soft plugin error | Logged; the call proceeds. | +| Invalid plugin config document (wrong version, missing `cpex` config, unsupported features) | Rejected at load. On an invalid reload, the runtime is marked failed and plugin calls return an internal MCP error until a valid config is applied. | + +## Config Store Failures + +| Failure | Behavior | +| --- | --- | +| Redis connection loss | The connection manager retries (configured with 1,000 retries). | +| User config missing | `400 Bad Request` from `user_config_store_layer`. | +| Redis `GET` returns an error | Currently reported by the Redis adapter as missing data, so the layer returns `400 Bad Request`. | +| User config undecodable, key encoding failure, or other non-missing store errors | `500 Internal Server Error` from `user_config_store_layer`. | + +For a symptom-first version of this table, see the troubleshooting section in +[Run the Gateway Locally](running-the-gateway.md#troubleshooting). diff --git a/docs/book/src/gateway-options.md b/docs/book/src/gateway-options.md new file mode 100644 index 00000000..86b5ddca --- /dev/null +++ b/docs/book/src/gateway-options.md @@ -0,0 +1,226 @@ +# Configuration Reference + +This page is the reference for every gateway setting. The binary parses its +configuration with `clap`, so each option has both a CLI flag and an environment +variable, and both feed the same `Config` struct. When a setting is supplied as +a flag and an environment variable at the same time, the command-line flag wins. + +For the always-current list, ask the binary directly: + +```bash +cargo run -p contextforge-gateway-rs --bin contextforge-gateway-rs -- --help +``` + +The sections below group the options by concern: listeners, JWT, Redis, upstream +transport, runtime, telemetry, and logging. + +## Minimum Useful Configuration + +At startup, the CLI requires Redis location and Redis connection mode: + +```text +--redis-address +--redis-port +--redis-mode +``` + +To serve useful traffic, the process also needs: + +| Need | Typical flag | +| --- | --- | +| At least one downstream listener | `--address 127.0.0.1:8001` or `--tls-address 0.0.0.0:8443` | +| JWT verification material | `--token-verification-public-key` or `--token-verification-secret` | +| A compatible upstream client mode | `--upstream-connection-mode plain-text-or-tls` for local HTTP backends | +| Runtime user config in Redis | Written by the control plane or by local bootstrap helpers | + +If no token verification key or secret is configured, authenticated MCP requests +cannot pass the claims layer. + +## Listener Options + +| Flag | Env var | Required | Meaning | +| --- | --- | --- | --- | +| `--address ` | `CONTEXTFORGE_GATEWAY_RS_ADDRESS` | No | Plain HTTP listener. Omit it when serving only TLS. | +| `--tls-address ` | `CONTEXTFORGE_GATEWAY_RS_TLS_ADDRESS` | No | TLS listener address. Requires certificate and private key. | +| `--server-certificate ` | `CONTEXTFORGE_GATEWAY_RS_TLS_SERVER_CERTIFICATE` | With `--tls-address` | PEM certificate chain for downstream TLS. | +| `--server-private-key ` | `CONTEXTFORGE_GATEWAY_RS_TLS_SERVER_PRIVATE_KEY` | With `--tls-address` | PEM private key for downstream TLS. | + +`--address` and `--tls-address` may both be configured, but they must not use +the same socket address. + +## JWT Options + +| Flag | Env var | Required | Meaning | +| --- | --- | --- | --- | +| `--token-verification-public-key ` | `CONTEXTFORGE_GATEWAY_RS_TOKEN_VERIFICATION_PUBLIC_KEY` | For RSA tokens | RSA public key used for `RS256`, `RS384`, or `RS512` tokens. | +| `--token-verification-secret ` | `CONTEXTFORGE_GATEWAY_RS_TOKEN_SECRET` | For HMAC tokens | Shared secret used for `HS256`, `HS384`, or `HS512` tokens. | +| `--token-verification-private-key ` | `CONTEXTFORGE_GATEWAY_RS_TOKEN_VERIFICATION_PRIVATE_KEY` | Local tools only | RSA private key used by the optional local token helper. Present only when `contextforge-gateway-rs-lib/with_tools` is enabled. | + +The claims layer validates issuer, audience, and expiration: + +| Claim | Expected value | +| --- | --- | +| `iss` | `mcpgateway` | +| `aud` | `mcpgateway-api` | +| `exp` | Present and not expired | + +The JWT `sub` claim selects the Redis user config key. The path virtual host id +then selects one virtual host inside that config. + +## Redis Options + +| Flag | Env var | Required | Meaning | +| --- | --- | --- | --- | +| `--redis-address ` | `CONTEXTFORGE_GATEWAY_RS_REDIS_HOSTNAME` | Yes | Redis host name or IP. | +| `--redis-port ` | `CONTEXTFORGE_GATEWAY_RS_REDIS_PORT` | Yes | Redis port. | +| `--redis-mode ` | `CONTEXTFORGE_GATEWAY_RS_REDIS_CONNECTION_MODE` | Yes | Redis connection mode: `plain-text`, `tls`, or `mtls`. | +| `--redis-tls-trust-bundle ` | `CONTEXTFORGE_GATEWAY_RS_REDIS_TLS_REDIS_TRUST_BUNDLE` | TLS and mTLS | PEM trust bundle for Redis TLS. | +| `--redis-tls-client-certificate ` | `CONTEXTFORGE_GATEWAY_RS_REDIS_TLS_REDIS_CLIENT_CERTIFICATE` | mTLS | PEM client certificate for Redis mTLS. | +| `--redis-tls-client-private-key ` | `CONTEXTFORGE_GATEWAY_RS_REDIS_TLS_REDIS_CLIENT_PRIVATE_KEY` | mTLS | PEM client private key for Redis mTLS. | +| `--user-config-cache-expiry-seconds ` | `CONTEXTFORGE_GATEWAY_RS_USER_CONFIG_CACHE_EXPIRY_SECONDS` | No, default `60` | Expiry for the in-process user config cache in front of Redis. `0` disables caching and reads Redis on every request. | + +Local Compose exposes plain Redis on `127.0.0.1:6379`, so local runs normally +use: + +```bash +--redis-address 127.0.0.1 \ +--redis-port 6379 \ +--redis-mode plain-text +``` + +Runtime config values are MessagePack encoded. Redis is the current transport +for config, not the routing model itself. + +## Upstream MCP Transport Options + +| Flag | Env var | Required | Meaning | +| --- | --- | --- | --- | +| `--upstream-connection-mode ` | `CONTEXTFORGE_GATEWAY_RS_UPSTREAM_CONNECTION_MODE` | No | Controls whether backend MCP URLs may be HTTP, HTTPS, or mTLS. | +| `--upstream-trust-bundle ` | `CONTEXTFORGE_GATEWAY_RS_TLS_UPSTREAM_TRUST_BUNDLE` | No | Additional PEM trust bundle for HTTPS upstreams. | +| `--upstream-certificate ` | `CONTEXTFORGE_GATEWAY_RS_TLS_UPSTREAM_CERTIFICATE` | mTLS modes | PEM client certificate for upstream mTLS. | +| `--upstream-private-key ` | `CONTEXTFORGE_GATEWAY_RS_TLS_UPSTREAM_PRIVATE_KEY` | mTLS modes | PEM client private key for upstream mTLS. | + +Connection modes: + +| Mode | Behavior | +| --- | --- | +| omitted or `tls-only` | HTTPS upstreams only. This is the safe default. | +| `plain-text-or-tls` | Allows HTTP and HTTPS upstream URLs. Use this for the local Compose backends. | +| `plain-text-or-m-tls` | Allows HTTP and HTTPS with client identity configured. | +| `mtls-only` | Requires HTTPS and uses the configured client certificate and key. | + +If an upstream backend URL is `http://...` and the mode is omitted, calls fail +before reaching that backend because the reqwest client is HTTPS-only. + +## Runtime Options + +| Flag | Env var | Default | Meaning | +| --- | --- | --- | --- | +| `--number-of-cpus ` | `CONTEXTFORGE_GATEWAY_RS_GATEWAY_CPUS` | Host CPU count | Worker thread count for the Tokio runtime shape. | +| `--single-runtime ` | `CONTEXTFORGE_GATEWAY_RS_SINGLE_RUNTIME` | `true` | `true` uses one multi-thread runtime. `false` starts multiple current-thread runtimes. | +| `--runtime-plugins-enabled ` | `CONTEXTFORGE_GATEWAY_RS_RUNTIME_PLUGINS_ENABLED` | `false` | Enables CPEX runtime plugin execution and Redis plugin config loading. | + +When runtime plugins are enabled, plugin config is read from Redis key +`ContextForgeGatewayRuntimePluginConfig`. That key is a control-plane trust +boundary because it decides which registered hooks run. + +## Telemetry Options + +| Flag | Env var | Default | Meaning | +| --- | --- | --- | --- | +| `--enable-open-telemetry ` | `CONTEXTFORGE_GATEWAY_RS_ENABLE_OPEN_TELEMETRY` | `false` | Enables trace export. | +| `--enable-otel-metrics ` | `CONTEXTFORGE_GATEWAY_RS_ENABLE_OTEL_METRICS` | `false` | Enables HTTP server metric export. | +| `--otlp-protocol ` | `CONTEXTFORGE_GATEWAY_RS_OTEL_EXPORTER_OTLP_PROTOCOL` | `grpc` | `grpc` or `http-protobuf`. | +| `--otlp-endpoint ` | `CONTEXTFORGE_GATEWAY_RS_OTEL_EXPORTER_OTLP_ENDPOINT` | Protocol-specific | Trace export endpoint. | +| `--otlp-metrics-endpoint ` | `CONTEXTFORGE_GATEWAY_RS_OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | Protocol-specific | Metrics export endpoint. | +| `--otlp-headers ` | `CONTEXTFORGE_GATEWAY_RS_OTEL_EXPORTER_OTLP_HEADERS` | none | Comma-separated `key=value` headers for OTLP export. | +| `--otlp-service-name ` | `CONTEXTFORGE_GATEWAY_RS_OTEL_SERVICE_NAME` | `CONTEXTFORGE-GATEWAY-RS` | OpenTelemetry `service.name`. | + +Default endpoints: + +| Protocol | Traces | Metrics | +| --- | --- | --- | +| `grpc` | `http://127.0.0.1:4317` | `http://127.0.0.1:4317` | +| `http-protobuf` | `http://127.0.0.1:4318/v1/traces` | `http://127.0.0.1:4318/v1/metrics` | + +`RUST_TRACE_LOG` controls which spans reach the OTLP trace layer. The HTTP +trace layer emits debug-level spans, so local trace verification usually needs: + +```bash +RUST_TRACE_LOG=debug +``` + +## Logging Options + +| Flag or env var | Default | Meaning | +| --- | --- | --- | +| `--log-name` / `CONTEXTFORGE_GATEWAY_LOG_NAME` | `contextforge-gateway-rs.log` | File log name in the current working directory. | +| `--log-rotation` / `CONTEXTFORGE_GATEWAY_LOG_ROTATION` | `hourly` | Rotation mode: `minutely`, `hourly`, `daily`, or `never`. | +| `RUST_LOG` | `debug` | Console event filter. | +| `RUST_FILE_LOG` | `debug` | File event filter. | +| `RUST_TRACE_LOG` | `info` | OpenTelemetry span filter. | + +## Common Flag Sets + +### Local HTTP Gateway And Plain Redis + +```bash +--address 127.0.0.1:8001 \ +--redis-address 127.0.0.1 \ +--redis-port 6379 \ +--redis-mode plain-text \ +--token-verification-public-key assets/jwt.key.pub \ +--upstream-connection-mode plain-text-or-tls +``` + +### Downstream TLS Listener + +```bash +--tls-address 0.0.0.0:8443 \ +--server-certificate assets/contextforgeCA/contextforge-server.cert.pem \ +--server-private-key assets/contextforgeCA/contextforge-server.key.pem +``` + +You may combine this with `--address` to expose both HTTP and HTTPS listeners. + +### HMAC Token Verification + +```bash +--token-verification-secret "${CONTEXTFORGE_GATEWAY_RS_TOKEN_SECRET}" +``` + +Use this only when downstream JWTs are signed with an `HS*` algorithm. RSA +tokens need `--token-verification-public-key`. + +### Redis TLS + +```bash +--redis-address 127.0.0.1 \ +--redis-port 16379 \ +--redis-mode tls \ +--redis-tls-trust-bundle assets/contextforgeCA/contextforge.intermediate.ca-chain.cert.pem +``` + +### Upstream mTLS + +```bash +--upstream-connection-mode mtls-only \ +--upstream-trust-bundle assets/contextforgeCA/contextforge.intermediate.ca-chain.cert.pem \ +--upstream-certificate assets/contextforgeCA/contextforge-client.cert.pem \ +--upstream-private-key assets/contextforgeCA/contextforge-client.key.pem +``` + +The certificate and key paths must point to PEM files accepted by reqwest. + +## Startup Validation + +The gateway fails fast for these invalid combinations: + +| Invalid combination | Reason | +| --- | --- | +| `--tls-address` without server cert or key | Downstream TLS cannot be configured partially. | +| Same socket for `--address` and `--tls-address` | The process cannot bind both listeners to the same address. | +| `--redis-mode tls` without trust bundle | Redis TLS needs a root certificate bundle. | +| `--redis-mode mtls` without trust bundle, client cert, or client key | Redis mTLS needs all three pieces. | +| mTLS upstream mode without upstream cert and key | The reqwest identity cannot be built. | +| Plain HTTP backend with default upstream mode | Default upstream mode is HTTPS-only. | diff --git a/docs/book/src/mcp-behavior.md b/docs/book/src/mcp-behavior.md new file mode 100644 index 00000000..fd3d0488 --- /dev/null +++ b/docs/book/src/mcp-behavior.md @@ -0,0 +1,13 @@ +# MCP Behavior + +This section describes what the gateway exposes as an MCP server and how it +maps downstream MCP calls onto configured backend MCP servers. + +> 📋 **Use this section for protocol behavior.** It covers the public MCP +> surface, backend fanout, namespacing, targeted routing, pagination, and +> streaming gaps. + +| Page | What it covers | +| --- | --- | +| 📋 [MCP Method Reference](mcp-method-reference.md) | Initialize, list, call, read, and prompt behavior from the client-facing gateway point of view. | +| 🛣️ [MCP Routing Semantics](mcp-routing-semantics.md) | How backend prefixes become the public tool, resource, and prompt namespace. | diff --git a/docs/book/src/mcp-method-reference.md b/docs/book/src/mcp-method-reference.md new file mode 100644 index 00000000..9162f559 --- /dev/null +++ b/docs/book/src/mcp-method-reference.md @@ -0,0 +1,63 @@ +# MCP Method Reference + +> 📋 **Reference lens:** this page lists what each MCP method does at the +> gateway today, from the client's point of view. For how prefixed names are +> split and merged, see [MCP Routing Semantics](mcp-routing-semantics.md). + +Gateway methods fall into three groups: `initialize` creates backend sessions, +routed methods use them, and a few methods remain local to the gateway process. + +## Initialize + +| Aspect | Behavior | +| --- | --- | +| Required context | RMCP `DownstreamSessionId`, `UserConfig`, `VirtualHostId`, and `ContextForgeClaims`. The `Mcp-session-id` header is not required yet. | +| Fanout | One `StreamableHttpClientTransport` per configured backend in the selected virtual host, opened concurrently with `futures::future::join_all`. | +| Backend failure | Not fatal. A backend that fails to initialize is stored with no running service; list calls skip it and routed calls to it fail. | +| Stored state | The local user session mapping, plus one `BackendTransports` entry per backend keyed by principal, backend name, and downstream session id. | +| Result | `InitializeResult` with the gateway's current fixed capability set: completions, prompts, resources, and tools enabled. Backend capabilities are stored with transport state but are not merged into the response yet. | + +## Routed List Methods + +`list_tools`, `list_resources`, `list_prompts`, and `list_resource_templates` +share one fanout path: + +| Aspect | Behavior | +| --- | --- | +| Fanout | Concurrent call to every connected backend in the session. | +| Namespacing | Every returned name is prefixed with its backend name. Resource templates get both the template name and the URI template prefixed. | +| Ordering | Merged output is sorted by name. | +| Failures | Failed or unavailable backends are logged and omitted from the merged result. | +| Pagination | One backend call per request and no downstream cursor; see [Known Gaps](mcp-routing-semantics.md#known-gaps). | + +## Routed Targeted Methods + +`call_tool`, `read_resource`, `get_prompt`, and `complete` share the prefix +splitter and resolve exactly one backend: + +| Method | Behavior | +| --- | --- | +| `call_tool` | Splits `{backend_name}-{tool_name}`, optionally runs the plugin pre hook, forwards the stripped tool name, tracks the downstream progress token, and optionally runs the plugin post hook on the result. Backend progress notifications for the tracked token are forwarded downstream, and a downstream cancellation is propagated to the backend call. | +| `read_resource` | Splits the prefixed resource name, strips the gateway prefix, and returns the single backend's result. | +| `get_prompt` | Splits the prefixed prompt name, strips the gateway prefix, and returns the single backend's result. | +| `complete` | Routes on the backend-prefixed prompt name or resource URI in `ref`, strips that prefix, and returns the selected backend's completion result. | + +Routed failures are JSON-RPC errors: a malformed prefixed name or an +unavailable backend returns an internal error, and duplicate backend matches +invalidate the session; see [Failure Modes](failure-modes.md). + +## Local Methods + +These methods pass through the same HTTP middleware but do not touch backends: + +| Method | Current behavior | +| --- | --- | +| `ping` | Returns success. | +| `subscribe`, `unsubscribe` | Mutate a local subscription set only. | + +## Session Delete + +A downstream `DELETE` with `Mcp-session-id` is handled by RMCP first. On a +successful response, `session_id_layer` removes the local user session mapping +and the `BackendTransports` entries for that principal and session id. See +[Session Ownership](session-ownership.md) for the cleanup rules. diff --git a/docs/book/src/mcp-routing-semantics.md b/docs/book/src/mcp-routing-semantics.md new file mode 100644 index 00000000..1495c6cc --- /dev/null +++ b/docs/book/src/mcp-routing-semantics.md @@ -0,0 +1,73 @@ +# MCP Routing Semantics + +The gateway presents multiple backend MCP servers as one downstream MCP server. +It does that by namespacing backend objects, fanning out list operations, and +routing exact calls back to the selected backend. + +## Backend Prefixes + +Backend names are part of the public namespace: + +```text +backend tool "increment" from backend "gateway-one" + -> "gateway-one-increment" + +backend resource "counter" from backend "gateway-one" + -> "gateway-one-counter" + +backend prompt "summarize" from backend "research" + -> "research-summarize" +``` + +The prefix is a routing contract, not a display detail. Renaming a backend +changes downstream tool, resource, and prompt names. + +## List Operations + +List operations fan out: + +```text +list_tools -> all connected backends -> merged sorted tools +list_resources -> all connected backends -> merged sorted resources +list_prompts -> all connected backends -> merged sorted prompts +list_resource_templates -> all connected backends -> merged sorted templates +``` + +Each successful backend result is rewritten with its backend prefix before the +merged response is returned. For resource templates, both the template name and +the URI template are prefixed with the backend name. Failed or unavailable backends are logged and +omitted from the current merged list result. + +## Routed Operations + +Calls that target one object split the prefixed name: + +```text +gateway-one-increment + -> backend_name = gateway-one + -> upstream tool name = increment +``` + +`call_tool`, `read_resource`, `get_prompt`, and `complete` all share the same +splitter. For `complete`, the routed value is the prompt name or resource URI +inside its `ref`. Backend names can themselves contain `-` (as in +`gateway-one`), so the splitter does not cut on the first `-`. Instead it walks +the configured backend names, takes the first one the prefixed name starts +with, and then requires a `-` immediately after that name. That is why +`gateway-one-increment` resolves to backend `gateway-one` and tool `increment`, +while a malformed name such as `gateway-oneincrement` is rejected. + +After the split, the gateway resolves exactly one connected backend service for +the principal and downstream session. Missing backends fail the call. Duplicate +matches are treated as invalid session state and trigger backend cleanup. + +## Known Gaps + +Pagination is not complete. `list_tools`, `list_resources`, `list_prompts`, +and `list_resource_templates` currently perform one backend call and return a +merged response with no downstream cursor. Full parity needs to gather all +backend pages or define a merged cursor strategy. + +Streaming/SSE behavior is also still a tracked design area. The target is to +stream downstream as backend chunks arrive while preserving plugin behavior, +backpressure, cancellation, and telemetry attribution. diff --git a/docs/book/src/operations.md b/docs/book/src/operations.md new file mode 100644 index 00000000..0c44a37c --- /dev/null +++ b/docs/book/src/operations.md @@ -0,0 +1,18 @@ +# Operations + +This section covers runtime behavior after the gateway is serving traffic: +plugins, diagnostics, failure modes, tests, load tests, and deployment +constraints. + +> 🧪 **Use this section when operating or verifying the gateway.** It focuses +> on plugin hooks, observability, failure boundaries, load testing, and +> deployment assumptions. + +| Page | What it covers | +| --- | --- | +| 🧩 [Plugins And Policy](plugins-and-policy.md) | Where request and response plugins fit, and why body access changes the runtime model. | +| 📈 [Telemetry And Diagnostics](telemetry-and-diagnostics.md) | Signals needed to debug authentication, config lookup, routing, upstream calls, and merged results. | +| 🚧 [Failure Modes](failure-modes.md) | Expected failures by boundary, including auth, Redis, virtual hosts, backend sessions, and upstream transport. | +| 🧪 [Testing](testing.md) | Workspace checks, in-repo integration tests, and the cf-integration full-stack test lanes. | +| ⚡ [Performance](performance.md) | Dataplane-only load testing, full-stack Locust runs, headless versus web UI, and benchmark settings. | +| 🏗️ [Deployment Notes](deployment-notes.md) | Front-door routing, GitHub Pages publication, session affinity, and cluster constraints. | diff --git a/docs/book/src/performance.md b/docs/book/src/performance.md new file mode 100644 index 00000000..879484fb --- /dev/null +++ b/docs/book/src/performance.md @@ -0,0 +1,86 @@ +# Performance + +> ⚡ **Two load paths:** `contextforge-load-test` measures the Rust dataplane +> alone, and the [`cf-integration`](https://github.com/contextforge-gateway-rs/cf-integration) +> harness measures the full nginx-to-control-plane-to-dataplane stack with +> Locust. Use the first to profile gateway changes and the second to measure +> what users would see. + +## Dataplane-Only Load Testing + +`crates/contextforge-load-test` is a [Goose](https://book.goose.rs/)-based +traffic driver that speaks the full streamable HTTP MCP flow against a running +gateway. Start the [local stack](running-the-gateway.md) with seeded user +config first, then: + +```bash +cargo run --release --bin contextforge-load-test -- \ + --host 'http://127.0.0.1:8001' \ + -u 120 -r 40 --run-time 120s \ + --report-file report.html +``` + +`-u` is concurrent users, `-r` is the spawn rate per second, and +`--report-file` writes an HTML report. This measures the Rust dataplane alone, +without a control plane or front door in the path. Curated run reports live in +the repository's `reports/` directory. + +## Full-Stack Load With Locust + +The [cf-integration harness](testing.md#full-stack-integration-harness) runs +Locust against the public nginx route with a streamable-HTTP-aware locustfile: + +| Command | What it runs | +| --- | --- | +| `scripts/cf-integration.sh smoke` | 1 user for 10 seconds — a quick sanity pass. | +| `scripts/cf-integration.sh locust` | The full load run, default 100 users for 5 minutes. | + +Tune with environment variables: + +```bash +LOCUST_USERS=20 LOCUST_SPAWN_RATE=5 LOCUST_RUN_TIME=2m \ + scripts/cf-integration.sh locust +``` + +`MCP_VIRTUAL_SERVER_ID` targets a UI-created virtual server instead of the +auto-registered Fast Time one, and `MCP_TOOL_NAMES` picks the tools to call. +Locust HTML/CSV output lands under `.integration/mcp-context-forge/reports/`; +curated run reports live in the harness `reports/` directory. + +## Headless Versus Locust Web UI + +The harness runs Locust headless by default (`LOCUST_MODE=headless`): a timed +run that writes the HTML and CSV reports and prints only the summary. Setting +`LOCUST_MODE` to any other value (for example `web`) switches the Locust +service to interactive mode: a master with the web UI on port `8089` and a +class picker (`LOCUST_EXPECT_WORKERS` controls the expected worker count). The +one-off `locust` command does not publish container ports, so for the web UI +start the Locust service through the stack's `testing` Compose profile — the +upstream stack maps `8089:8089` — then open `http://localhost:8089` and drive +the run from the browser. + +## Benchmark Settings + +The harness tunes config propagation for functional runs, not throughput: + +| Variable | Functional default | Benchmark value | +| --- | --- | --- | +| `CF_DATAPLANE_PUBLISHER_INTERVAL_SECONDS` | `2` (fast config publish) | `60` (upstream default) | +| `CF_DATAPLANE_USER_CONFIG_CACHE_EXPIRY_SECONDS` | `0` (cache disabled) | `60` (upstream default) | + +Restore both to `60` before measuring throughput, or the fast publish loop and +per-request Redis reads distort the numbers. + +## Control-Plane Baseline Load + +To compare against the stack without the dataplane in the path: + +```bash +scripts/cf-integration.sh down # frees the shared host ports +scripts/cf-integration.sh controlplane-locust +``` + +The baseline run defaults to the non-UI Locust class subset (health, Fast +Time, Fast Test, version/meta). `CONTROLPLANE_LOCUST_CLASSES=all` adds the +admin/UI/mutating surfaces. `LOCUST_USERS`, `LOCUST_SPAWN_RATE`, and +`LOCUST_RUN_TIME` apply here too. diff --git a/docs/book/src/plugins-and-policy.md b/docs/book/src/plugins-and-policy.md new file mode 100644 index 00000000..9ca71fec --- /dev/null +++ b/docs/book/src/plugins-and-policy.md @@ -0,0 +1,60 @@ +# Plugins And Policy + +Plugins are a policy boundary, not just middleware. They can inspect and mutate +payloads, so the gateway keeps the supported hook surface narrow and explicit. + +## Runtime Enablement + +Runtime plugins are disabled by default. When enabled, the binary creates a +CPEX runtime registry and the runtime initializes it before serving traffic. + +Plugin configuration is loaded from Redis at: + +```text +ContextForgeGatewayRuntimePluginConfig +``` + +The runtime registry builds an initialized immutable plugin manager from that +configuration. Reloading swaps the manager instead of mutating a live one. + +## Supported Hooks + +The supported surface is deliberately narrow: + +```text +cmf.tool_pre_invoke +cmf.tool_post_invoke +``` + +The gateway rejects route-based plugin selection, plugin directories, global +policies/defaults, non-tool hooks, and plugin conditions. Those features need +clear behavior for streaming, failures, timeouts, backpressure, context +propagation, and observability before they belong on the hot path. + +## Tool Call Behavior + +For `call_tool`, the pre hook runs after backend routing has selected the +backend and stripped the public prefix. The hook sees the backend name, routed +tool name, and arguments. It can: + +- leave arguments unchanged +- replace arguments +- deny the call + +After the upstream backend returns, the post hook can: + +- leave the result unchanged +- rewrite the result payload +- deny the response + +Hook state is carried across the upstream call so pre and post hooks can share +CPEX context for the same logical tool call. + +## Boundary Rules + +Plugin execution must not poison shared gateway state. A plugin denial becomes +an MCP error. Soft plugin errors are logged. Unsupported plugin configuration +fails validation before the runtime is accepted. + +Future hook expansion should define behavior for streaming/SSE, cancellation, +timeouts, backpressure, and telemetry before adding new hook points. diff --git a/docs/book/src/project.md b/docs/book/src/project.md new file mode 100644 index 00000000..dc55465b --- /dev/null +++ b/docs/book/src/project.md @@ -0,0 +1,11 @@ +# Project + +This section covers how to work on the repository and the book itself. + +> 🛠️ **Use this section for repo workflow.** It keeps contribution and +> publishing guidance separate from runtime architecture. + +| Page | What it covers | +| --- | --- | +| 🛠️ [Contributing To The Gateway](contributing.md) | Repository layout, expected validation, branch hygiene, and how to keep dataplane changes scoped. | +| 📚 [Publishing This Book](publishing-this-book.md) | How the mdBook build feeds GitHub Pages and how to preview the output before pushing. | diff --git a/docs/book/src/publishing-this-book.md b/docs/book/src/publishing-this-book.md new file mode 100644 index 00000000..65a66001 --- /dev/null +++ b/docs/book/src/publishing-this-book.md @@ -0,0 +1,63 @@ +# Publishing This Book + +> 📚 **Publishing path:** mdBook renders `docs/book/src/` into static HTML, +> and the GitHub Pages workflow deploys that HTML from `main`. + +## Local Build And Preview + +The Pages workflow uses mdBook `0.5.3`; use the same version locally: + +```bash +cargo install mdbook --version 0.5.3 --locked +``` + +Build and preview: + +```bash +mdbook build docs/book +mdbook serve docs/book --hostname 127.0.0.1 --port 3000 --open +``` + +The generated HTML lands in `docs/book/book/`, which is ignored by git. Never +edit files there; edit `docs/book/src/` instead. + +## Validation Before Pushing + +```bash +mdbook build docs/book +mdbook test docs/book +git diff --check +``` + +`mdbook test` runs Rust code blocks as tests and confirms that every chapter +in `SUMMARY.md` parses. + +## Workflow Behavior + +`.github/workflows/pages.yml` runs when a change touches `docs/book/**` or the +workflow file itself: + +| Trigger | Jobs | +| --- | --- | +| Pull request | `build` only: install mdBook, build the book, upload `docs/book/book` as the Pages artifact. | +| Push to `main` | `build`, then `deploy` publishes the artifact to GitHub Pages. | +| Manual `workflow_dispatch` | Same as a pull request run. | + +Builds are cancelled when a newer run starts on the same ref; deploys are +serialized and never cancelled mid-flight. + +## Repository Settings + +Publishing requires the repository's GitHub Pages source to be set to +`GitHub Actions`. Without that setting, the deploy job cannot publish the +uploaded artifact. + +## Adding Or Renaming Chapters + +1. Add or rename the Markdown file under `docs/book/src/`. +2. Update `docs/book/src/SUMMARY.md`; its order is the reader's path through + the book. +3. Run the validation commands above. + +Draft chapters use the `> Status: draft. To be implemented.` marker followed +by a `## To implement` list, so unfinished pages stay visible and navigable. diff --git a/docs/book/src/request-flow.md b/docs/book/src/request-flow.md new file mode 100644 index 00000000..1b142d4c --- /dev/null +++ b/docs/book/src/request-flow.md @@ -0,0 +1,196 @@ +# Request Flow + +> 🎯 **Flow invariant:** Axum builds request context before RMCP handlers route MCP +> methods. MCP handlers should read typed extensions, not parse headers, paths, +> or Redis keys directly. + +![Request flow](assets/request-flow.svg) + +## Graph Legend + +The graph uses color only to separate paths: + +| Color | Meaning | +| --- | --- | +| Blue | Request direction: listener, middleware, RMCP dispatch, and backend calls. | +| Green | Response direction: backend result, response unwind, and client response. | +| Amber | `initialize`, where backend MCP client sessions are created. | +| Purple | Authorized MCP calls after `Mcp-session-id` exists. | +| Red | Layer-local HTTP rejection before MCP method handling. | + +This page follows a normal streamable HTTP MCP request through the code. The +shape below is based on the current `main.rs`, `runtime.rs`, `Gateway::run_gateway`, +the request layers, and `McpService`. To watch the same flow with real requests, +follow [Run the Gateway Locally](running-the-gateway.md) alongside this page. + +## Startup Path + +Startup begins in `crates/contextforge-gateway-rs/src/main.rs`: + +```text +install rustls crypto provider + -> Config::parse() + -> logging::init_tracing_logging(&config) + -> Runtime::from(&config) + -> optional CpexRuntimeRegistry + -> Gateway::builder() + .with_config(config) + .with_user_config_store_type(UserConfigStoreType::Redis) + .with_session_manager(LocalSessionManager::default()) + .with_plugin_runtime(...) + .build() + -> runtime.execute(gateway, plugin_registry) +``` + +`runtime.execute` either runs one multi-thread Tokio runtime or starts multiple +current-thread runtimes. In both modes it initializes the optional CPEX runtime +and then calls `gateway.run_gateway()`. + +## HTTP Stack Order + +`Gateway::run_gateway` builds the service stack in `crates/contextforge-gateway-rs-lib/src/lib.rs`. +Tower layers execute from the outside in, so a normal MCP request reaches the +handler in this order: + +```text +TCP/TLS listener + -> HttpMetricsLayer + -> TraceLayer + -> /contextforge-rs nested router + -> CORS layer + -> virtual_host_id_layer + -> claims_layer + -> session_id_layer + -> user_config_store_layer + -> virtual_host_config_layer + -> /servers/{virtual_host_name}/mcp RMCP service +``` + +The inner Axum route is: + +```text +/servers/{virtual_host_name}/mcp +``` + +The public route is nested under: + +```text +/contextforge-rs/servers/{virtual_host_name}/mcp +``` + +The route segment is named `virtual_host_name`, but the layer stores the value +as `VirtualHostId`. + +## Flow Checkpoints + +| Checkpoint | Established fact | Next dependency | +| --- | --- | --- | +| Listener | The request reached the Rust dataplane over TCP or TLS. | Metrics, tracing, and nested routing can observe it. | +| Path extraction | The inner path matched `/servers/{virtual_host_id}/mcp`. | MCP handlers can resolve a `VirtualHost`. | +| Claims validation | The bearer token was accepted and `ContextForgeClaims` exists. | Config lookup can use `claims.sub`. | +| User config lookup | A `UserConfig` exists for the authenticated subject. | The virtual host check can run against that config. | +| Virtual host check | The path's virtual host id exists in the caller's config. | MCP validators can resolve the selected `VirtualHost`. | +| RMCP dispatch | The streamable HTTP request is mapped to an MCP method. | The handler chooses initialize, routed backend calls, or local behavior. | + +## Middleware Context + +The request layers insert the context used later by RMCP handlers: + +| Layer | Request behavior | Failure behavior | +| --- | --- | --- | +| `virtual_host_id_layer` | Extracts `/servers/{virtual_host_id}/mcp` and inserts `VirtualHostId`. | Returns `400` when the inner path does not match. | +| `claims_layer` | Validates `Authorization: Bearer ...` with configured RS/HMAC decoder, issuer, audience, and expiration. Inserts `ContextForgeClaims`. | Returns `401` for missing or invalid bearer auth. | +| `session_id_layer` | Reads `Mcp-session-id` and inserts `SessionId` when present. | Missing session id is allowed here; authorized MCP handlers reject it later when required. | +| `user_config_store_layer` | Uses `claims.sub` as `User::new(subject)`, loads `UserConfig`, and inserts it. | Returns `400` for missing config, `500` for other store failures, and `400` if claims are absent. | +| `virtual_host_config_layer` | Checks that the path's virtual host id exists in the loaded `UserConfig`. | Returns `404` with body `{"detail":"Server not found"}` when the virtual host is not in the caller's config. | + +For `DELETE`, `session_id_layer` also has response-side behavior. It lets RMCP +handle the request first. If the RMCP response succeeds and a session id exists, +it removes the local user session mapping and removes backend transports for +`principal + session_id`. + +## Initialize Flow + +`initialize` does not require the downstream `Mcp-session-id` header. RMCP +creates a `DownstreamSessionId` and places it in the request context. + +`McpService::initialize` runs this sequence: + +1. `InitializeCallValidator` reads `DownstreamSessionId`, `UserConfig`, + `VirtualHostId`, and `ContextForgeClaims`. +2. It resolves `user_config.virtual_hosts[virtual_host_id]`. +3. It reads the local user session mapping for `claims.sub + downstream_session_id`. +4. For every backend in the selected virtual host, it concurrently builds a + `StreamableHttpClientTransport` with the configured backend URL and serves a + `GatewayBackendClient` over that transport. +5. It collects backend capabilities and running RMCP client services. The + capabilities are stored with backend transport state for future routing, but + they do not shape the downstream initialize response yet. +6. It writes the local user session mapping. +7. It stores each backend service in `BackendTransports` keyed by principal, + backend name, and downstream session id. +8. It returns `InitializeResult` with the gateway's current fixed capability + set: completions, prompts, resources, and tools enabled. + +Backend initialization is concurrent through `futures::future::join_all`. + +## Authorized MCP Calls + +Routed MCP calls after initialization use `AuthorizedCallValidator`. It requires: + +```text +SessionId +UserConfig +VirtualHostId +ContextForgeClaims +``` + +The validator resolves the same virtual host from the authenticated user's +config, then `SessionManager` locates backend services for: + +```text +principal + backend_name + session_id +``` + +Current routed method families: + +| Method family | Flow | +| --- | --- | +| `list_tools`, `list_resources`, `list_prompts`, `list_resource_templates` | Borrow all configured backend services, call every available backend concurrently with `fan_out_list`, namespace results with the backend name, sort merged output, and return one list. | +| `call_tool` | Split `{backend_name}-{tool_name}`, resolve one backend, optionally run `before_tool_call`, apply argument/name changes, track the downstream progress token, call the backend, optionally run `after_tool_call`, and return the backend result. | +| `read_resource`, `get_prompt` | Split the prefixed resource or prompt name, resolve one backend, strip the gateway prefix, call the backend, and return the backend result. | +| `complete` | Split the backend-prefixed prompt name or resource URI in `ref`, resolve one backend, strip the gateway prefix, and return the backend completion result. | + +`GatewayBackendClient` handles backend progress notifications for `call_tool`. +If a progress token matches an in-flight downstream tool call, it optionally +runs the stream-event post hook and forwards the progress notification back to +the downstream client. + +`call_backend_tool` also watches the downstream cancellation token. If the +downstream call is cancelled before the backend responds, the gateway sends a +cancel request to the backend handle. + +## Local MCP Methods + +Some MCP methods are currently local to the gateway implementation rather than +backend-routed: + +| Method | Current behavior | +| --- | --- | +| `ping` | Returns success. | +| `subscribe`, `unsubscribe` | Mutate the local subscription set. | + +These paths still pass through the same HTTP middleware, but they do not use the +backend fanout or prefixed routing path today. + +## Response Path + +Backend responses return to `McpService` first. `call_tool` may run response +plugin hooks before returning. List calls merge and namespace backend output +before returning. Single-backend calls return the selected backend result after +gateway prefix removal. + +The HTTP response then unwinds through `virtual_host_config_layer`, +`user_config_store_layer`, `session_id_layer`, `claims_layer`, +`virtual_host_id_layer`, CORS, trace, and metrics. On successful `DELETE`, `session_id_layer` performs local session and +backend transport cleanup during this unwind. diff --git a/docs/book/src/running-the-gateway.md b/docs/book/src/running-the-gateway.md new file mode 100644 index 00000000..9d17d1d2 --- /dev/null +++ b/docs/book/src/running-the-gateway.md @@ -0,0 +1,263 @@ +# Run the Gateway Locally + +This page walks through a local run that proves the dataplane can authenticate a +caller, load runtime config from Redis, initialize backend MCP sessions, and +return merged MCP results to a downstream client. + +The local flow uses the repository's Docker Compose stack for Redis and two +sample backend MCP servers: + +| Service | Local port | Role | +| --- | --- | --- | +| `redis` | `6379` | Runtime config store for user config. | +| `gateway-one` | `5555` | Sample counter MCP backend. | +| `gateway-two` | `5556` | Sample conformance MCP backend. | +| `contextforge-gateway-rs` | `8001` | Rust dataplane process you run with Cargo. | + +> 🧪 The token and user-config endpoints below are local bootstrap helpers. They +> are compiled with `contextforge-gateway-rs-lib/with_tools`. In a real +> deployment, the external ContextForge control plane mints tokens and writes +> config to Redis. + +Run the numbered steps in order, in the same terminal. Later steps reuse shell +variables such as `${TOKEN}` and `${SESSION_ID}` that earlier steps set, so a +fresh shell will not have them. + +## Prerequisites + +- Rust toolchain matching the workspace `rust-version`. +- Docker Compose. +- Free local ports: `6379`, `16379`, `5555`, `5556`, and `8001`. The Compose + stack maps both plain (`6379`) and TLS (`16379`) Redis ports. +- Test keys under `assets/`: `jwt.key` and `jwt.key.pub`. + +## 1. Start Redis and Backend MCP Servers + +```bash +docker compose -f docker/docker-compose-local.yaml up -d +``` + +The sample backends default to a CPU limit of `8` and a reservation of `4`, +which Docker rejects on hosts with fewer CPUs +(`range of CPUs is from 0.01 to ...`). On smaller machines, override the +sizing knobs: + +```bash +GATEWAY_CPU_LIMIT=2 GATEWAY_CPU_RESERVATION=0.5 \ + docker compose -f docker/docker-compose-local.yaml up -d +``` + +Check that the local dependencies are running: + +```bash +docker compose -f docker/docker-compose-local.yaml ps redis gateway-one gateway-two +``` + +The backends listen on the host so the gateway can reach them at +`http://127.0.0.1:5555/mcp` and `http://127.0.0.1:5556/mcp`. + +## 2. Start the Gateway + +For the local bootstrap flow, run the binary with the `with_tools` dependency +feature so the admin token and config endpoints are available: + +```bash +cargo run -p contextforge-gateway-rs \ + --features contextforge-gateway-rs-lib/with_tools \ + --bin contextforge-gateway-rs -- \ + --address 127.0.0.1:8001 \ + --redis-address 127.0.0.1 \ + --redis-port 6379 \ + --redis-mode plain-text \ + --token-verification-public-key assets/jwt.key.pub \ + --token-verification-private-key assets/jwt.key \ + --upstream-connection-mode plain-text-or-tls \ + --number-of-cpus 4 +``` + +Keep this process running. The gateway exposes MCP traffic under: + +```text +http://127.0.0.1:8001/contextforge-rs/servers/{virtual_host_id}/mcp +``` + +The local command uses `--upstream-connection-mode plain-text-or-tls` because +the sample backend URLs are plain HTTP. Without that option, the default +upstream client is HTTPS-only. + +## 3. Mint a Test Token + +In another terminal, request a JWT for the test subject: + +```bash +TOKEN=$(curl --silent --show-error \ + --url http://127.0.0.1:8001/contextforge-rs/admin/tokens/admin@example.com) + +printf '%s\n' "${TOKEN}" +``` + +The token's `sub` claim is `admin@example.com`. The gateway uses that subject +as the Redis user-config key. + +## 4. Write User Config + +Seed Redis with one virtual host and two backend MCP servers: + +```bash +curl --silent --show-error --request POST \ + --url http://127.0.0.1:8001/contextforge-rs/admin/userconfigs/admin@example.com \ + --header 'content-type: application/json' \ + --data '{ + "virtual_hosts": { + "c0ffee00f001f00lf00ldeadbeefdead": { + "backends": { + "gateway-one": { + "name": "gateway-one", + "url": "http://127.0.0.1:5555/mcp", + "transport": "STREAMABLEHTTP", + "passthrough_headers": [], + "allowed_tool_names": [], + "allowed_resource_names": [], + "allowed_prompt_names": [] + }, + "gateway-two": { + "name": "gateway-two", + "url": "http://127.0.0.1:5556/mcp", + "transport": "STREAMABLEHTTP", + "passthrough_headers": [], + "allowed_tool_names": [], + "allowed_resource_names": [], + "allowed_prompt_names": [] + } + } + } + } + }' +``` + +The important relationship is: + +```text +JWT subject admin@example.com + -> Redis user config + -> virtual host c0ffee00f001f00lf00ldeadbeefdead + -> backend MCP URLs +``` + +## 5. Initialize an MCP Session + +Open a streamable HTTP MCP session and save the returned `mcp-session-id` +header: + +```bash +INIT_HEADERS=$(mktemp) + +curl --silent --show-error \ + --dump-header "${INIT_HEADERS}" \ + --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \ + --header "authorization: Bearer ${TOKEN}" \ + --header 'content-type: application/json' \ + --header 'accept: application/json, text/event-stream' \ + --data '{ + "jsonrpc": "2.0", + "id": 0, + "method": "initialize", + "params": { + "protocolVersion": "2025-11-25", + "capabilities": {}, + "clientInfo": { "name": "curl", "version": "0.1.0" } + } + }' + +SESSION_ID=$(awk 'tolower($1) == "mcp-session-id:" { gsub("\r", "", $2); print $2 }' "${INIT_HEADERS}") +printf '%s\n' "${SESSION_ID}" +``` + +During `initialize`, the gateway opens upstream MCP client sessions to the +configured backends and stores them under the downstream session id. + +Send the MCP initialized notification: + +```bash +curl --silent --show-error \ + --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \ + --header "authorization: Bearer ${TOKEN}" \ + --header "mcp-session-id: ${SESSION_ID}" \ + --header 'mcp-protocol-version: 2025-11-25' \ + --header 'content-type: application/json' \ + --header 'accept: application/json, text/event-stream' \ + --data '{"jsonrpc":"2.0","method":"notifications/initialized"}' +``` + +## 6. List Tools + +```bash +curl --silent --show-error \ + --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \ + --header "authorization: Bearer ${TOKEN}" \ + --header "mcp-session-id: ${SESSION_ID}" \ + --header 'mcp-protocol-version: 2025-11-25' \ + --header 'content-type: application/json' \ + --header 'accept: application/json, text/event-stream' \ + --data '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' +``` + +The response should contain tool names prefixed with their backend name, such as +`gateway-one-increment`. That prefix is the routing contract for later +`tools/call` requests. + +## 7. Call a Tool + +```bash +curl --silent --show-error \ + --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \ + --header "authorization: Bearer ${TOKEN}" \ + --header "mcp-session-id: ${SESSION_ID}" \ + --header 'mcp-protocol-version: 2025-11-25' \ + --header 'content-type: application/json' \ + --header 'accept: application/json, text/event-stream' \ + --data '{ + "jsonrpc": "2.0", + "id": 2, + "method": "tools/call", + "params": { + "name": "gateway-one-increment", + "arguments": {} + } + }' +``` + +The gateway strips `gateway-one-`, forwards `increment` to the `gateway-one` +backend session, and returns the backend result to the client. + +## 8. End the Session + +```bash +curl --silent --show-error --request DELETE \ + --url http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp \ + --header "authorization: Bearer ${TOKEN}" \ + --header "mcp-session-id: ${SESSION_ID}" +``` + +The gateway removes local session state and backend transports for that user +and MCP session id. + +## Troubleshooting + +| Symptom | Likely boundary | +| --- | --- | +| `401 Unauthorized` | Missing bearer token, invalid signature, expired token, wrong issuer, wrong audience, or no configured decoder key. | +| `400 Problem occurred retrieving the configuration` | Redis has no config for the JWT subject, or the Redis lookup returned no data. | +| `500 Problem occurred retrieving the configuration` | The stored config could not be decoded, the Redis key could not be encoded, or another non-missing config-store error occurred. | +| `404` with `{"detail":"Server not found"}` | The path virtual host id is not present in that user's config. | +| MCP session id is empty | The `initialize` call failed before RMCP created a downstream session. | +| Backend calls fail | Backend URL is wrong, backend process is down, or `--upstream-connection-mode` rejects the URL scheme. | +| Calls fail after gateway restart | Backend MCP session state is local process state today. Re-run `initialize`. | + +## Tear Down + +Stop the gateway with `Ctrl-C`, then stop local dependencies: + +```bash +docker compose -f docker/docker-compose-local.yaml down +``` diff --git a/docs/book/src/runtime-configuration.md b/docs/book/src/runtime-configuration.md new file mode 100644 index 00000000..5f32b4de --- /dev/null +++ b/docs/book/src/runtime-configuration.md @@ -0,0 +1,183 @@ +# Runtime Configuration + +> 🗂️ **Config boundary:** process config tells the gateway how to run. Runtime +> user config tells each request where it may route. Plugin config controls the +> optional CPEX hook runtime. + +![Runtime configuration](assets/runtime-config.svg) + +The gateway consumes configuration from three places. Keeping them separate is +important because they change at different times and are used by different +parts of the dataplane. + +## Config Surfaces + +| Surface | Source | Loaded | Main user | +| --- | --- | --- | --- | +| Process `Config` | CLI flags and `CONTEXTFORGE_GATEWAY_RS_*` env vars parsed by `clap`. | Startup. | Listener setup, JWT decoder keys, Redis connection, upstream HTTP client, telemetry, runtime shape. | +| `UserConfig` | `UserConfigStore`, currently Redis through `RedisUserConfigStore`. | Per request after JWT validation. | Virtual host and backend selection. | +| `RuntimePluginConfigDocument` | Redis key `ContextForgeGatewayRuntimePluginConfig` when runtime plugins are enabled. | Startup and watcher reload. | CPEX tool pre/post hooks. | + +The control plane owns durable authoring. This repo owns reading those values +and applying them on the request path. + +## UserConfig Shape + +The API crate defines the shared runtime routing model: + +```text +UserConfig + virtual_hosts: HashMap + +VirtualHost + backends: HashMap + +BackendMCPGateway + name: String + url: Url + transport: Transport + passthrough_headers: Vec + allowed_tool_names: Vec + allowed_resource_names: Vec + allowed_prompt_names: Vec +``` + +`Transport` currently declares: + +```text +STREAMABLEHTTP +SSE +STDIO +``` + +The Rust types above are easier to picture as JSON. For a complete, working +`UserConfig` document, see the seed request body in +[Run the Gateway Locally](running-the-gateway.md). + +## What The MCP Dataplane Uses Today + +The struct is already wider than the current MCP routing code. That is useful, +but the distinction should stay explicit: + +| Config field | Current MCP dataplane behavior | +| --- | --- | +| `UserConfig.virtual_hosts` | Required. `VirtualHostId` from the path selects one entry. | +| `VirtualHost.backends` map key | Required. This key is the public backend namespace used in tool/resource/prompt prefixes. | +| `BackendMCPGateway.url` | Required. Used to build the upstream `StreamableHttpClientTransport`. | +| `BackendMCPGateway.name` | Present in the model. Current routing uses the backend map key, not this field, as the namespace. | +| `transport` | Present in the model. Current upstream code always builds a streamable HTTP client transport. | +| `passthrough_headers` | Present in the model. Current MCP routing does not apply header pass-through policy from this field. | +| `allowed_tool_names` | Present in the model. Current list/call routing does not enforce it. | +| `allowed_resource_names` | Present in the model. Current resource routing does not enforce it. | +| `allowed_prompt_names` | Present in the model. Current prompt routing does not enforce it. | + +The current route selection is: + +```text +JWT subject + -> User::new(subject) + -> UserConfig + -> path VirtualHostId + -> VirtualHost + -> backend map key + -> BackendMCPGateway.url +``` + +Expected config growth beyond the current fields: + +- route selection across multiple MCP endpoints +- principal/virtual-host filters for tools, resources, and prompts +- backend auth/TLS material references +- request/response header pass/add/remove rules +- plugin/CPEX hook settings +- pagination/SSE behavior where protocol handling needs config +- future A2A and LLM routing/provider settings + +## Redis Storage + +`RedisUserConfigStore` stores user routing config as MessagePack: + +| Item | Encoding | +| --- | --- | +| Redis key | MessagePack-encoded `User::new(jwt_subject)`. | +| Redis value | MessagePack-encoded `UserConfig`. | +| Cache key | Raw subject string through `User::key()`. | +| Cache value | Decoded `UserConfig`. | + +The in-process cache is an implementation detail: + +| Setting | Value | +| --- | --- | +| Entries | 50,000 | +| Expiry | `--user-config-cache-expiry-seconds`, default 60 seconds; `0` disables caching | +| Redis connection retries | 1,000 | + +Routing code should stay behind the `UserConfigStore` trait. That keeps Redis, +MessagePack, and cache behavior out of MCP method handling. + +## Plugin Runtime Config + +When `runtime_plugins_enabled` is true, startup builds a +`CpexRuntimeRegistry::with_redis_config(...)`. The registry reads a separate +runtime plugin document: + +```text +Redis key: ContextForgeGatewayRuntimePluginConfig + +RuntimePluginConfigDocument + version: 1 + cpex: CpexConfig +``` + +The plugin config loader accepts JSON bytes or MessagePack bytes. It rejects +documents with the wrong version, missing `cpex` config, unsupported CPEX +features, or an unavailable config store. + +Current supported plugin scope is deliberately narrow: + +| CPEX feature | Current support | +| --- | --- | +| `TOOL_PRE_INVOKE` | Supported for `call_tool` before backend invocation. | +| `TOOL_POST_INVOKE` | Supported for `call_tool` responses and backend progress events. | +| CPEX routing | Rejected. Gateway routing is owned by `UserConfig` and MCP routing code. | +| Plugin conditions | Rejected. | +| Plugin dirs, global policies, global defaults | Rejected. | +| Other hook types | Rejected. | + +The registry has a watcher interval of 10 minutes. On valid reloads it swaps in +the new runtime. On invalid reloads it marks the runtime failed so new plugin +calls return an internal MCP error until a valid config is applied. + +## Process Config + +Process `Config` is not per-user routing state. It is parsed once and controls +the gateway shell: + +| Area | Examples | +| --- | --- | +| Listener | `address`, `tls_address`, downstream TLS certificate and private key. | +| Authentication | RSA public key path or HMAC secret for JWT verification. | +| Redis | Host, port, plain/TLS/mTLS mode, trust bundle, client cert, client key. | +| Upstream HTTP client | Plaintext/TLS/mTLS mode, upstream trust bundle, client cert, client key. | +| Telemetry | OpenTelemetry traces, metrics endpoint/protocol, OTLP headers, service name. | +| Runtime shape | CPU count and single-runtime versus multi-runtime mode. | +| Plugins | Whether runtime plugins are enabled. | +| Logging | Log name and rotation. | + +The process config decides whether the gateway can start and what dependencies +it can reach. It does not decide which backend a specific MCP caller may use; +that remains in `UserConfig`. + +## Selection Invariant + +Backend selection should only happen after these facts exist: + +```text +authenticated subject + + loaded UserConfig + + path VirtualHostId + + MCP session context +``` + +That invariant is why runtime config access is in middleware and MCP validators, +not hidden inside individual backend calls. diff --git a/docs/book/src/security-model.md b/docs/book/src/security-model.md new file mode 100644 index 00000000..d1643a7a --- /dev/null +++ b/docs/book/src/security-model.md @@ -0,0 +1,72 @@ +# Security Model And Trust Boundaries + +> 🔒 **Trust rule:** the gateway trusts its process config and the +> control-plane-authored data in Redis. It does not trust downstream callers +> beyond a validated JWT, and it reaches backends only through configured +> URLs. + +## Trust Boundaries + +| Boundary | Trust level | Enforced by | +| --- | --- | --- | +| Downstream client | Untrusted. Every request must present a valid bearer JWT; the session id alone grants nothing without matching principal state. | `claims_layer`, validators, and the principal-scoped backend session keys. | +| JWT verification material | Trust anchor. The RSA public key or HMAC secret in process config decides which tokens are accepted. | Process config; loaded at startup. | +| Redis | Control-plane trust boundary. Whoever can write Redis controls routing (`UserConfig`) and, when runtime plugins are enabled, which registered hooks execute (`ContextForgeGatewayRuntimePluginConfig`). | Redis TLS/mTLS connection modes; the dataplane never writes user config in production builds. | +| Backend MCP servers | Trusted per configured URL. The gateway forwards caller traffic to them and merges their responses. | `UserConfig` backend URLs plus the upstream connection mode. | +| Plugins | Fully trusted code. Hooks run in-process and can read and mutate tool payloads. | Compiled-in factories only; Redis config activates registered factories, it cannot load new code. | + +## Identity And Authorization + +Authentication is bearer-JWT only: + +- Accepted algorithms are `RS256/RS384/RS512` (public key configured) or + `HS256/HS384/HS512` (shared secret configured); anything else is rejected. +- `iss` must be `mcpgateway`, `aud` must be `mcpgateway-api`, and `exp` is + validated. There is no revocation list: a leaked token is valid until it + expires. +- Authorization is config existence. The `sub` claim selects the caller's + `UserConfig`; the path selects one virtual host inside it. A caller can + never reach a backend that is not in their own config, and unknown virtual + hosts return `404` before MCP handling. +- `jti`, `token_use`, `iat`, `teams`, `user`, and `scopes` are carried but not + yet enforced; `token_use`, `iat`, `teams`, and `scopes` are optional, as is + `user.full_name`. Fine-grained permissions are future policy work. + +## What Compromise Means + +| If this is compromised | Impact | +| --- | --- | +| JWT signing key or HMAC secret | Attacker mints tokens for any subject and reaches that subject's backends. Rotate the key and restart; there is no revocation. | +| Redis write access | Attacker rewrites routing (arbitrary backend URLs receive caller traffic) and, if runtime plugins are enabled, chooses which registered hooks run on payloads. Protect Redis with TLS/mTLS and control-plane-only write access. | +| A backend MCP server | Attacker sees the requests routed to that backend and controls its responses; the namespace prefix limits blast radius to that backend's objects. | +| The gateway process | Full compromise: it holds the decoding keys in memory and live backend sessions. | + +## Transport Security + +| Leg | Current posture | +| --- | --- | +| Downstream | TLS optional (`--tls-address`, no client auth — identity is the bearer token). Plain HTTP is acceptable only behind a trusted front door on a private network. | +| Upstream | HTTPS-only by default; plain HTTP must be opted into with `--upstream-connection-mode`. mTLS client identity is supported per process. | +| Redis | Plain, TLS, or mTLS via `--redis-mode`. Use TLS or mTLS anywhere Redis crosses a trust zone, because Redis is the config trust boundary. | + +CORS is currently wide open (any origin, method, and header). The API is +bearer-token based and cookie-free, so cross-site request forgery does not +apply, but expect this to tighten as policy work lands. + +## Local Bootstrap Helpers + +The `contextforge-gateway-rs-lib/with_tools` feature compiles in +`/contextforge-rs/admin/tokens/{user}`, +`/contextforge-rs/admin/userconfigs/{user}`, and `/contextforge-rs/health`. +These routes are registered outside the authentication middleware, so token +minting and config writes are unauthenticated by design — they exist only for +local bootstrap. Production builds must not enable this feature: in a real +deployment the control plane mints tokens and writes config. + +## Secrets Handling + +- The HMAC secret is held as a `SecretString`; key and certificate material is + read from disk paths at startup. +- Log hygiene is a standing rule: never log tokens, authorization headers, + secrets, Redis key/value bytes, full `UserConfig` documents, or backend + credentials. diff --git a/docs/book/src/session-ownership.md b/docs/book/src/session-ownership.md new file mode 100644 index 00000000..8c600023 --- /dev/null +++ b/docs/book/src/session-ownership.md @@ -0,0 +1,121 @@ +# Session Ownership + +> 🧷 **Session invariant:** backend MCP services are local process state keyed +> by authenticated principal, backend namespace, and downstream MCP session id. + +![Session ownership](assets/session-ownership.svg) + +The current gateway keeps backend MCP client services in memory. That choice is +simple and fast, but it defines how initialized MCP sessions can be routed in a +deployment. + +## Owned State + +Several pieces of state participate in one MCP session. They do not all have +the same owner. + +| State | Key | Owner today | Lifetime | +| --- | --- | --- | --- | +| RMCP downstream session | RMCP `DownstreamSessionId` and later `Mcp-session-id`. | RMCP `LocalSessionManager`. | Local process. | +| User session mapping | `UserSession { principal, downstream_session_id }`. | `LocalUserSessionStore`. | Local LRU cache, 50,000 entries, 1 hour. | +| Backend running service | `principal + backend_name + session_id`. | `BackendTransports`. | Local process. | +| Backend upstream MCP session | Managed inside RMCP running client service. | Backend service handle. | Local process and backend server. | + +`RedisUserSessionStore` exists, but `Gateway::run_gateway` wires +`LocalUserSessionStore` today. Even if the user session mapping moved to Redis, +the live RMCP backend services in `BackendTransports` would still be local +unless that architecture changes too. + +## Initialize Creates Backend Services + +`initialize` is the ownership creation point: + +```text +InitializeCallValidator + -> selected VirtualHost + -> claims.sub + -> RMCP DownstreamSessionId + -> one upstream client service per backend + -> BackendTransports entries +``` + +The backend transport key is: + +```text +BackendTransportKey + principal: claims.sub + backend_name: backend map key + session_id: downstream session id +``` + +The backend map key matters because one downstream MCP session can fan out to +many backend MCP sessions. The principal matters because two callers could +present the same downstream session id value. + +## Borrowing Backend Services + +`SessionManager` is a request-scoped view over the selected virtual host, +session id, principal, and shared backend transport map. + +| Operation | What it does | +| --- | --- | +| `get_backend_names()` | Reads backend namespaces from the selected `VirtualHost`. | +| `borrow_transports()` | Locks the map, finds entries for the current principal/session/backend names, and clones the shared `Arc`. | +| `cleanup_backends(reason)` | Removes backend entries for the current principal/session/backend names. | + +`borrow_transports()` does not move services out of the map in the current +code. It clones shared service handles, so there is no return step after a +request completes. + +## List Calls And Routed Calls + +Backend service lookup splits into two patterns: + +| MCP call shape | Session behavior | +| --- | --- | +| `list_tools`, `list_resources`, `list_prompts`, `list_resource_templates` | Borrow every available backend service for the selected virtual host, call them concurrently, then merge and namespace successful responses. | +| `call_tool`, `read_resource`, `get_prompt` | Split the prefixed name into `{backend_name}-{object_name}`, resolve the single backend service, strip the gateway prefix, and call that backend. | + +An initialized backend entry can still contain no running service if backend +initialize failed. List calls skip unavailable backends. Routed calls to that +backend return a routing error. + +If routed lookup ever detects duplicate backend matches, the session is treated +as invalid and `cleanup_backends` removes the local backend entries for that +principal and session. + +## Delete Cleanup + +`session_id_layer` owns response-side cleanup for downstream `DELETE`: + +```text +DELETE with Mcp-session-id + -> let RMCP handle the request + -> if RMCP response is successful + -> remove LocalUserSessionStore entry + -> remove BackendTransports entries for principal + session id +``` + +Cleanup is intentionally tied to a successful downstream delete response. If +RMCP rejects the delete, the layer returns that response and leaves local +session state untouched. + +## Load-Balancing Consequence + +Random load balancing is not safe for stateful MCP sessions today. After +`initialize`, later requests with the same `Mcp-session-id` must reach the +process that owns the backend running services. + +Known deployment options are: + +| Option | Tradeoff | +| --- | --- | +| Sticky routing by `Mcp-session-id` | Simple, preserves local state, but couples sessions to one process. | +| External session ownership | More flexible, but backend running service state must move out of process or become reconstructible. | +| Reinitialize after failover | Operationally simple, but clients must tolerate session loss. | + +Until backend service ownership changes, design request handling as if a +stateful MCP session belongs to one gateway process. The same constraint +appears inside one host in multi-runtime mode, where each runtime thread owns +its own backend transports; see +[Concurrency And Runtime Model](concurrency-and-runtime.md). diff --git a/docs/book/src/system-shape.md b/docs/book/src/system-shape.md new file mode 100644 index 00000000..18273b68 --- /dev/null +++ b/docs/book/src/system-shape.md @@ -0,0 +1,279 @@ +# System Shape + +> 🧭 **Architecture lens:** this page explains what the gateway is, what it is +> not, and which code owns each boundary. + +![System shape](assets/system-shape.svg) + +The ContextForge Gateway is the Rust dataplane process for MCP traffic. It is +not a second ContextForge application. It accepts downstream streamable HTTP MCP +requests, builds request context, loads runtime config, opens or reuses backend +MCP client sessions, and returns one merged MCP server view to the caller. + +That shape matters because this repository sits on the traffic path. Its design +should bias toward predictable request behavior, explicit state ownership, and +small hot-path dependencies. + +## Three-Layer Model + +The gateway is easiest to reason about as three layers: + +| Layer | Owns | Does not own | +| --- | --- | --- | +| ContextForge control plane | Management workflows, UI, IAM lifecycle, durable config ownership, policy authoring, observability storage. | Hot MCP request handling or in-process MCP sessions. | +| Rust gateway dataplane | Auth, config lookup, virtual host selection, MCP fanout, plugin hooks, telemetry emission, upstream calls. | Control-plane APIs, customer IAM, UI, durable metrics storage. | +| Backend MCP servers | Actual tools, resources, prompts, and backend protocol behavior. | Caller auth, virtual host selection, gateway-level policy. | + +In text form, the same model is: + +```text +ContextForge control plane + -> writes runtime config and owns management workflows + +contextforge-gateway-rs process + -> authenticates, loads config, routes, fans out, applies hooks, emits signals + +backend MCP servers + -> own the actual tools, resources, and prompts +``` + +> **Boundary rule:** Redis is the current config-store transport, not the +> architecture. Routing code should depend on `UserConfigStore`, not Redis +> commands. + +## Control Plane Boundary + +The external ContextForge control plane owns the slow-changing product surface: + +| Area | Why it stays outside this repo | +| --- | --- | +| Management APIs and UI | They are user/admin workflows, not request-path forwarding. | +| Credential and policy authoring | The dataplane consumes policy; it should not become the policy editor. | +| Persistent config ownership | Durable storage and schema lifecycle belong to the control plane. | +| Durable observability storage | The gateway emits telemetry; it should not become the metrics database. | +| Customer IAM lifecycle | The gateway validates presented identity, not the customer identity product. | + +The Rust dataplane owns the hot path: + +| Area | Code-facing meaning | +| --- | --- | +| Listener setup | Expose downstream TCP and/or TLS endpoints. | +| JWT validation | Convert bearer auth into request claims. | +| Runtime config lookup | Load the caller's `UserConfig` by JWT subject. | +| MCP fanout and routing | Present multiple backends as one MCP server. | +| Request and response hooks | Run plugin/policy hooks at explicit pipeline points. | +| Telemetry emission | Emit logs, traces, and metrics around the path. | + +The front door can route only MCP traffic to this process while leaving other +ContextForge traffic on existing paths: + +```text +/contextforge-rs/servers/{virtual_host_id}/mcp +``` + +From a client point of view, that endpoint behaves like one ContextForge MCP +server. Internally, it is a focused proxy, fanout, policy, and merge dataplane. + +## Process Assembly + +Startup begins in `crates/contextforge-gateway-rs/src/main.rs`. The binary crate +does process-level assembly: + +```text +install rustls crypto provider + -> Config::parse() + -> logging::init_tracing_logging(&config) + -> runtime::Runtime::from(&config) + -> optional CpexRuntimeRegistry + -> Gateway::builder() + .with_config(config) + .with_user_config_store_type(UserConfigStoreType::Redis) + .with_session_manager(LocalSessionManager::default()) + .with_plugin_runtime(...) + .build() + -> runtime.execute(gateway, plugin_registry) +``` + +| Step | Owner | Result | +| --- | --- | --- | +| Parse config | Binary crate | A process config value used to build the gateway. | +| Initialize logging | Binary crate | Console/file logging and trace export are ready before serving traffic. | +| Select runtime | `runtime::Runtime` | Either one multi-thread Tokio runtime or multiple current-thread runtimes. | +| Start plugin runtime | CPEX registry path | Optional plugin manager and reloadable config loop. | +| Build gateway | `Gateway::builder()` | Dataplane dependencies are assembled and handed to the runtime. | + +`runtime::Runtime` changes executor shape, not gateway semantics. The default +path is one multi-thread Tokio runtime. The binary crate also sets +`tikv_jemallocator` as the global allocator and owns file logging, console +logging, trace export, and metrics export setup. + +## Gateway Assembly + +`crates/contextforge-gateway-rs-lib/src/lib.rs` builds the dataplane stack in +`Gateway::run_gateway`: + +```text +RedisUserConfigStore +LocalUserSessionStore +BackendTransports +reqwest::Client for backend MCP calls +RMCP StreamableHttpService +Axum middleware stack +TCP and/or TLS listeners +``` + +The resulting route tree is: + +```text +/contextforge-rs + /servers/{virtual_host_name}/mcp + /admin/... local bootstrap helpers when with_tools is enabled + /health local bootstrap helper when with_tools is enabled +``` + +The route segment is named `virtual_host_name` in Axum, but the gateway extracts +the actual path value as a `VirtualHostId`. Routing code should treat it as a +virtual host id. + +## Workspace Layers + +The repository is a Cargo workspace with narrow responsibilities: + +| Crate | Responsibility | +| --- | --- | +| `contextforge-gateway-rs` | Process shell: CLI config, logging, telemetry exporters, Tokio runtime, allocator, plugin registry startup, gateway construction. | +| `contextforge-gateway-rs-lib` | Dataplane library: Axum stack, request middleware, config lookup, MCP fanout/routing, backend sessions, upstream clients, downstream transports. | +| `contextforge-gateway-rs-apis` | Shared contract: `UserConfig`, `VirtualHost`, `BackendMCPGateway`, Redis user key, plugin config document, schema generation. | +| `contextforge-gateway-rs-cpex` | Plugin integration: CPEX runtime registry, Redis plugin config loading, supported tool pre/post hooks, stream event adaptation. | +| `contextforge-load-test` | Performance harness: end-to-end MCP traffic driver. | + +Keep those boundaries stable. Adding dataplane behavior to the binary crate +makes the process shell harder to test and reuse. Adding control-plane behavior +to the library crate makes the hot path harder to reason about. + +## Pipeline Shape + +The target shape is a bidirectional AI traffic pipeline: authentication, +rate limiting, routing and protocol selection, request mutation, optional +retrieval, and request guardrails on the way upstream; response guardrails, +response mutation, and telemetry on the way back. Upstreams may eventually be +MCP, A2A, or model providers. Preserve the ordering as pieces land: auth and +config before backend selection, request plugins before upstream calls, +response plugins before returning, telemetry around both sides. + +The current code implements the MCP subset of that pipeline: + +```text +downstream request + -> virtual host extraction + -> JWT validation + -> session extraction + -> user config lookup + -> MCP handler validation + -> request plugin hooks + -> backend MCP call + +upstream response + -> response plugin hooks + -> merge, namespace, or pass through + -> metrics, tracing, and logging + -> downstream response +``` + +The Axum layer registration order is important because Tower layers execute +from the outside in. The stack is built so the request reaches handlers with: + +| Context | Inserted by | Used by | +| --- | --- | --- | +| `VirtualHostId` | `virtual_host_id_layer` | Initialize and authorized MCP validators. | +| `ContextForgeClaims` | `claims_layer` | Config lookup, session cleanup, routing. | +| `SessionId` | `session_id_layer` | Authorized MCP calls after initialize. | +| `UserConfig` | `user_config_store_layer` | Virtual host selection and backend selection. | + +After those extensions exist, `virtual_host_config_layer` rejects requests with +`404` when the path's virtual host id is not present in the loaded +`UserConfig`, so MCP handlers only see resolvable virtual hosts. + +`initialize` is the special call. RMCP provides a downstream session id before +the gateway has a `Mcp-session-id` header. The gateway fans out to configured +backends, opens upstream MCP client sessions, and stores those running services +under: + +```text +principal + backend_name + downstream_session_id +``` + +Later calls use the `Mcp-session-id` header to find those stored backend +services. + +## State Ownership + +The architecture keeps request state, session state, runtime config, and process +state in separate ownership scopes. The current ownership model is: + +| State | Owner | Lifetime | +| --- | --- | --- | +| CLI `Config` | Binary startup and `Gateway` | Process lifetime. | +| JWT decoders | `ContextForgeGatewayAppState` | Process lifetime. | +| User config | `UserConfigStore`, currently Redis plus in-process LRU | Control-plane authored, request-path consumed. | +| Request identity | Request extensions | One HTTP request. | +| Virtual host id | Request extensions | One HTTP request. | +| Downstream session id | RMCP plus `SessionId` extension | MCP session. | +| Backend RMCP services | `BackendTransports` map | Local process, per principal/backend/session. | +| Local user session mapping | `LocalUserSessionStore` | Local process, per principal/session. | +| Runtime plugin manager | `CpexRuntimeRegistry` and handle | Process lifetime, reloadable. | +| Logs, traces, metrics | Logging setup and Axum/tower layers | Process lifetime. | + +> **Session rule:** backend MCP services are local process state today. +> Load-balanced deployments need sticky routing, external session state, or a +> reinitialize-after-failover story. + +Redis-backed user session storage exists in code, but the default gateway +assembly wires `LocalUserSessionStore`. Do not assume session state is durable +or shared across gateway nodes. + +## Module Boundaries + +The library crate has deliberately narrow internal module roles: + +| Module | Owns | +| --- | --- | +| `common.rs` | CLI config shape, JWT claim shape, Redis config validation, upstream `reqwest::Client` construction. | +| `layers/` | HTTP request extension extraction and request-bound validation. | +| `gateway/` | MCP server behavior, initialize fanout, list merging, prefixed routing, backend service state. | +| `gateway/session_store/` | Local and Redis-capable user session storage abstractions. | +| `user_config_store/` | `UserConfigStore` trait and Redis-backed runtime config store. | +| `transports/` | Downstream TCP and TLS listener setup. | +| `tools.rs` | Local bootstrap helpers compiled only with `with_tools`. | + +Concrete Redis commands stay behind `UserConfigStore`. Downstream listener +transport stays in `transports/`. MCP method behavior stays in `gateway/`. +Request extension extraction stays in `layers/`. + +## Reusable Shell + +The code is MCP-first today, but the repository is intentionally not named or +structured as only an MCP proxy. Authentication, configuration ingestion, TLS +handling, plugin execution, telemetry, runtime shape, and session strategy are +gateway-shell concerns. + +MCP-specific behavior should remain isolated to the current MCP modules so +future A2A or model-provider routing can reuse the shell instead of growing a +parallel stack. + +## Why These Architecture Pages Exist + +The architecture section is split into subpages because each page tracks a +boundary that already exists in modules or runtime ownership: + +| Subpage | Reason it exists | +| --- | --- | +| Request Flow | Shows the ordered path through startup, middleware, initialize, and authorized calls. | +| Authentication And User Config Lookup | Keeps identity and config selection separate from MCP method handling. | +| Runtime Configuration | Describes the data model the gateway consumes, independent of Redis transport details. | +| Backend Connections And Transports | Separates downstream, upstream, and config-store transports. | +| Session Ownership | Makes local backend session state and load-balancing constraints explicit. | +| Architectural Choices | Records tradeoffs that should not be changed accidentally. | + +This split should make later pages easier to fill in without turning the +architecture chapter into one long mixed-concern essay. diff --git a/docs/book/src/telemetry-and-diagnostics.md b/docs/book/src/telemetry-and-diagnostics.md new file mode 100644 index 00000000..73a60ef9 --- /dev/null +++ b/docs/book/src/telemetry-and-diagnostics.md @@ -0,0 +1,166 @@ +# Telemetry And Diagnostics + +> 📈 **Observability lens:** the gateway emits three signals — logs, request +> traces, and HTTP metrics. This page explains how each is produced, how to +> verify them locally, and where to look when a request misbehaves. + +## Signal Overview + +| Signal | Produced by | Destination | +| --- | --- | --- | +| Logs | `tracing` events from layers, validators, and MCP handlers. | Console and a rotating file in the working directory. | +| Request traces | `tower_http::TraceLayer` spans around every HTTP request. | OTLP trace export when `--enable-open-telemetry` is set. | +| HTTP metrics | `axum-otel-metrics` (`HttpMetricsLayer`) request instruments. | OTLP metrics export when `--enable-otel-metrics` is set. | + +All export settings are process config; see the telemetry section of the +[Configuration Reference](gateway-options.md#telemetry-options). + +## Logging + +Console and file logging are always on. The file appender writes +`contextforge-gateway-rs.log` (configurable with `--log-name`) in the current +working directory and rotates hourly by default (`--log-rotation`). + +Three environment filters control verbosity independently: + +| Env var | Default | Controls | +| --- | --- | --- | +| `RUST_LOG` | `debug` | Console events. | +| `RUST_FILE_LOG` | `debug` | File events. | +| `RUST_TRACE_LOG` | `info` | Which spans reach the OTLP trace exporter. | + +Dataplane log messages use a stable `method_name - event text field = value` +shape, so boundary-specific prefixes are grep-friendly: `claims_layer`, +`user_config_store_layer`, `virtual_host_config_layer`, +`AuthorizedCallValidator::validate`, `initialize:`, `call_tool`, and so on. + +## Traces + +`TraceLayer` emits its request spans at `DEBUG` level. + +> ⚠️ **`RUST_TRACE_LOG=debug` is required for trace export.** The default span +> filter (`info`) drops the HTTP spans before they reach the OTLP exporter, so +> nothing arrives at the trace backend. + +Enable export with `--enable-open-telemetry true` plus the OTLP protocol, +endpoint, and header flags. Each handled HTTP request produces one span with +method, route, status, and latency. + +## Metrics + +`--enable-otel-metrics true` turns on the HTTP server instruments: request +duration histogram, active request gauge, and request/response body size +histograms, all labeled with method, status code, and `service.name`. + +Metrics are pushed by a `PeriodicReader` every **30 seconds**, so allow about +35 seconds after the first request before the first data point appears +downstream. + +## Local Verification Stack + +A complete local pipeline ships under `docker/` as overlays on top of the +[local run](running-the-gateway.md) stack: + +| Component | Role | Endpoint | +| --- | --- | --- | +| Langfuse | Trace backend and span viewer. | UI at `http://localhost:3100`, login `admin@example.com` / `changeme`, project `ContextForge Gateway (Rust)`. | +| OTel Collector | Receives OTLP from the gateway, fans traces and metrics out. | OTLP/HTTP on `:4318`, Prometheus exposition on `:8889`, raw dumps via `docker logs otel-collector`. | +| Prometheus | Scrapes the collector for browsable PromQL. | UI at `http://localhost:9090`. | + +### 1. Start the stack + +```bash +docker compose \ + -f docker/docker-compose-local.yaml \ + -f docker/docker-compose-langfuse.yaml \ + -f docker/docker-compose-otel-collector.yaml \ + up -d +``` + +Re-run with `ps` instead of `up -d` and wait until every container is healthy. + +### 2. Run the gateway with export enabled + +```bash +RUST_TRACE_LOG=debug \ +cargo run --release --bin contextforge-gateway-rs -- \ + --address 0.0.0.0:8001 \ + --redis-port 6379 --redis-address 127.0.0.1 --redis-mode=plain-text \ + --token-verification-public-key assets/jwt.key.pub \ + --number-of-cpus 4 \ + --upstream-connection-mode=plain-text-or-tls \ + --enable-open-telemetry true \ + --enable-otel-metrics true \ + --otlp-protocol http-protobuf \ + --otlp-endpoint http://127.0.0.1:3100/api/public/otel/v1/traces \ + --otlp-headers "Authorization=Basic cGstbGYtY29udGV4dGZvcmdlOnNrLWxmLWNvbnRleHRmb3JnZQ==" \ + --otlp-metrics-endpoint http://127.0.0.1:4318/v1/metrics \ + --otlp-service-name contextforge-gateway-rs +``` + +The `Authorization` header is Basic auth for the seeded Langfuse project keys +(`pk-lf-contextforge:sk-lf-contextforge`, base64-encoded). + +### 3. Generate traffic + +```bash +for i in {1..10}; do + curl -s -o /dev/null -w "%{http_code}\n" \ + http://127.0.0.1:8001/contextforge-rs/admin/tokens/admin@example.com +done +``` + +Any response counts: each request is traced and recorded as a metric sample. + +### 4. Inspect the data + +- **Langfuse:** open `http://localhost:3100` and check the project — one span + per request with method, route, status, and latency. +- **Prometheus:** open `http://localhost:9090`; `Status → Targets` should show + `otel-collector:8889` as **UP**, then try the starter queries below. +- **Collector stdout:** `docker logs otel-collector --tail 200` shows raw OTLP + trace and metric dumps. + +The data path is: + +```text +gateway :8001 + -> OTLP/HTTP traces -> Langfuse :3100 (span UI) + -> OTLP/HTTP metrics -> OTel Collector :4318 + -> stdout dump (docker logs) + -> Prometheus exposition :8889 -> Prometheus :9090 +``` + +Tear the stack down with the same three `-f` files and `down`. + +## Prometheus Starter Queries + +| Question | Query | +| --- | --- | +| Request count by method, status, and service | `http_server_request_duration_seconds_count` | +| p95 latency | `histogram_quantile(0.95, sum by (le) (rate(http_server_request_duration_seconds_bucket[1m])))` | +| In-flight requests | `http_server_active_requests` | +| Payload throughput | `http_server_request_body_size_bytes_sum` / `http_server_response_body_size_bytes_sum` | + +## Debugging Checklist + +Work down the boundaries in request order; each failure has an owning signal. +[Failure Modes](failure-modes.md) lists the exact responses per boundary. + +| Symptom | Where to look | +| --- | --- | +| `401` responses | `claims_layer` log lines: missing/invalid bearer token, unsupported algorithm, or no configured decoder key. | +| `400` config responses | `user_config_store_layer` log lines and Redis content for the JWT subject. | +| `404` `Server not found` | `virtual_host_config_layer` debug line showing the requested virtual host id and how many the caller's config has. | +| MCP routing errors | `AuthorizedCallValidator::validate` debug lines, then `call_tool`/`read_resource`/`get_prompt` warns for the split and backend resolution. | +| Backend failures | `initialize:` warns for backends that failed to connect; routed-call warns name the failing backend. | +| Plugin problems | CPEX pipeline error logs; an invalid reload marks the plugin runtime failed until a valid config is applied. | + +## Known Gaps + +Tracked upstream, not yet implemented here: + +- W3C trace-context propagation across gateway hops + ([mcp-context-forge#4723](https://github.com/IBM/mcp-context-forge/issues/4723)). +- MCP-semantic spans with tool names and JSON-RPC method attributes + ([mcp-context-forge#4722](https://github.com/IBM/mcp-context-forge/issues/4722)). diff --git a/docs/book/src/testing.md b/docs/book/src/testing.md new file mode 100644 index 00000000..218883f6 --- /dev/null +++ b/docs/book/src/testing.md @@ -0,0 +1,110 @@ +# Testing + +> 🧪 **Verification rings:** workspace checks prove the code compiles and unit +> behavior holds, in-repo integration tests prove MCP routing against mock +> backends, and the `cf-integration` harness proves the whole +> control-plane-to-dataplane path end to end. Load and benchmark runs have +> their own page: [Performance](performance.md). + +## Workspace Validation + +CI runs these on every change; run them locally before pushing: + +```bash +cargo fmt --all --check +cargo clippy --locked --workspace --all-targets -- -D warnings +cargo nextest run --locked --workspace +``` + +Use `cargo test` when nextest is unavailable. For book changes, also run +`mdbook build docs/book` and `mdbook test docs/book`. + +## In-Repo Integration Tests + +`crates/contextforge-gateway-rs-lib/tests/` exercises the gateway against +in-process mock MCP backends (shared helpers live in `tests/support/`): + +| Test file | Covers | +| --- | --- | +| `gateway_list_tools.rs` | List fanout, prefixing, and merged output. | +| `gateway_prompts.rs` | Prompt listing and prefixed `get_prompt` routing. | +| `gateway_resource_templates.rs` | Template fanout with prefixed names and URI templates, plus `read_resource` round-trips. | +| `gateway_plugins.rs` | CPEX pre/post tool hooks around `call_tool` and stream events. | + +These run in `cargo nextest run` with no Docker dependencies. + +## Full-Stack Integration Harness + +[`cf-integration`](https://github.com/contextforge-gateway-rs/cf-integration) +wires the external ContextForge control plane (`cf-controlplane`) to this +dataplane the way production intends: the stock upstream Compose stack, plus +exactly two intentional differences — nginx routes only +`/servers/{virtual_host_id}/mcp` to the dataplane (as +`/contextforge-rs/servers/{virtual_host_id}/mcp`), and the control plane runs +with `DATAPLANE_PUBLISHER=true` so virtual server configs reach the dataplane +through Redis. Because the stack otherwise matches upstream, test failures +measure dataplane behavior, not stack drift. + +### Quick Start + +```bash +scripts/cf-integration.sh up +``` + +This checks out the control plane under `.integration/mcp-context-forge`, +pulls the published dataplane image, and starts the combined stack plus a +local MCP counter backend. The admin UI is at +`http://localhost:8080/admin` (`admin@example.com` / `changeme`). A Fast Time +backend is auto-registered as a fixed virtual server, so the commands below +work with no manual UI step; backends added through the UI are published to +the dataplane by the control-plane publisher. + +### Route Probe + +```bash +scripts/cf-integration.sh probe +``` + +Verifies the public nginx-to-dataplane route end to end: a 401 negative check, +`initialize`, session reuse, `tools/list`, and `tools/call`. + +### Full Test Runs + +| Command | What it runs | +| --- | --- | +| `scripts/cf-integration.sh test-all` | Every live lane against the running stack, with per-test result rows and full output in a timestamped log under `.integration/test-logs/` (override with `CF_TEST_LOG_DIR`). | +| `CF_TEST_ALL_LOCUST=true scripts/cf-integration.sh test-all` | Same, plus the full Locust load run as a final lane. | +| `scripts/cf-integration.sh test-all-up` | Start or update the stack, then `test-all` without the load lane. | +| `scripts/cf-integration.sh test-all-up-load` | Start or update the stack, then `test-all` with the load lane. | + +Individual lanes are `live-mcp`, `live-rbac`, `live-protocol`, and `live-all`. +`live-mcp` is the green lane: the full MCP protocol end-to-end suite passes +against this harness. Remaining failures in the other lanes measure known +dataplane feature gaps; the harness `reports/` directory keeps the current +classification. + +### Control-Plane Baseline + +To separate dataplane regressions from upstream behavior, the harness can run +the stock control-plane-only stack (no dataplane, no nginx split, no +publisher) with the same commands: + +```bash +scripts/cf-integration.sh down # frees the shared host ports +scripts/cf-integration.sh controlplane-test-all # up + live core + locust +``` + +Individual steps are `controlplane-up`, `controlplane-live-core`, +`controlplane-live-all`, `controlplane-locust`, and `controlplane-down`. The +baseline load run is covered in [Performance](performance.md). + +### Key Settings + +| Variable | Purpose | +| --- | --- | +| `CF_DATAPLANE_IMAGE` / `CF_DATAPLANE_VERSION` | Which published dataplane image the stack runs. | +| `CF_CONTROLPLANE_IMAGE` / `CF_CONTROLPLANE_REF` | Which control-plane image and git ref to use. | +| `NGINX_PORT` | Public front-door port (default `8080`). | +| `CF_TEST_LOG_DIR` | Where `test-all` writes timestamped logs. | + +See the harness README for the full command and override list. diff --git a/docs/book/src/usage.md b/docs/book/src/usage.md new file mode 100644 index 00000000..d44a7812 --- /dev/null +++ b/docs/book/src/usage.md @@ -0,0 +1,30 @@ +# Getting Started + +This section is for the first working run of the gateway: start the local +dependencies, launch the Rust dataplane, seed runtime config, and send real MCP +traffic through it. + +> 🚀 **Goal:** get from a clean checkout to one client-facing MCP endpoint at +> `/contextforge-rs/servers/{virtual_host_id}/mcp`. + +| Page | What it covers | +| --- | --- | +| 🚀 [Run the Gateway Locally](running-the-gateway.md) | Docker services, local bootstrap helpers, user config, MCP session setup, and smoke tests. | +| ⚙️ [Configuration Reference](gateway-options.md) | Listener settings, Redis wiring, JWT verification, upstream transport, telemetry, logging, and runtime knobs. | + +## What "started" means + +A useful local gateway run has these pieces: + +| Piece | Why it matters | +| --- | --- | +| Redis | Holds runtime `UserConfig` keyed by JWT subject. | +| Backend MCP servers | Provide the tools, resources, and prompts the gateway merges. | +| Gateway listener | Accepts downstream streamable HTTP MCP traffic. | +| JWT verification key or secret | Lets the gateway authenticate downstream requests. | +| User config | Maps the caller's JWT subject to virtual hosts and backend MCP URLs. | +| MCP session | Binds downstream session state to backend MCP client sessions. | + +If one of those is missing, the gateway should fail at the boundary that owns +that fact: authentication, config lookup, virtual host resolution, session +lookup, routing, or upstream transport. diff --git a/docs/book/src/what-is-contextforge-gateway.md b/docs/book/src/what-is-contextforge-gateway.md new file mode 100644 index 00000000..bc10540b --- /dev/null +++ b/docs/book/src/what-is-contextforge-gateway.md @@ -0,0 +1,123 @@ +# What is ContextForge Gateway? + +Welcome to the ContextForge Gateway book. + +`contextforge-gateway-rs` is the Rust **dataplane** for ContextForge: a single +MCP entry point that sits in front of many backend MCP servers and makes them +look like one. If you have used an API gateway or reverse proxy for HTTP APIs, +this is the same idea for the Model Context Protocol (MCP). + +Concretely, the gateway accepts MCP streamable HTTP traffic from a client, +authenticates the caller, loads that caller's runtime configuration, opens MCP +client sessions to the backend MCP servers that configuration allows, and +presents those backends to the client as one merged MCP server. + +Throughout this book, *downstream* means the client side of the gateway and +*upstream* means the backend side. + +![Gateway overview](assets/gateway-overview.svg) + +Blue arrows show request traffic. Green arrows show backend responses returning +to the gateway and the merged MCP response going back to the client. + +| Layer | What happens | +| --- | --- | +| Client edge | An MCP client calls `/contextforge-rs/servers/{virtual_host_id}/mcp` with a bearer token and MCP session headers. | +| Gateway hot path | The gateway validates identity, loads runtime config, selects the virtual host, and routes MCP methods. | +| Backend edge | The gateway opens or reuses MCP client sessions to configured backend MCP servers and merges what the client sees. | + +> **Key idea:** downstream clients interact with one logical MCP server. The +> gateway decides which configured backend sessions are involved. + +The gateway is not the management application. It does not own the UI, IAM +lifecycle, tenant administration, durable metrics storage, or long-lived +configuration editing. Those concerns stay in the external ContextForge control +plane. This repository owns the hot request path and the runtime pieces needed +to make that path fast, observable, and enforceable. + +Most operators should not think about backend MCP servers directly. They should +think about one client-facing MCP endpoint: + +```text +/contextforge-rs/servers/{virtual_host_id}/mcp +``` + +Behind that endpoint, the gateway has three main boundaries: + +- the downstream boundary, where clients connect over streamable HTTP and + present their bearer token and MCP session id +- the configuration boundary, where the gateway turns the JWT subject into a + runtime `UserConfig` +- the upstream boundary, where the gateway opens or reuses MCP client sessions + to the configured backend servers + +Most bugs in this service come from confusing those boundaries. A downstream +request is not allowed to pick arbitrary upstream backends. Redis is not the +routing model; it is the current config-store transport. Backend sessions are +not durable cluster state; they are local process state today. + +| Boundary | Input | Output | Owner | +| --- | --- | --- | --- | +| Downstream | HTTP request, JWT, MCP headers | request extensions and downstream session id | gateway | +| Configuration | JWT subject and virtual host id | `UserConfig` and selected `VirtualHost` | control plane data, gateway enforcement | +| Upstream | selected backend names and URLs | running backend MCP client services | gateway | + +## Key terms + +These words appear on almost every page. It is worth pinning them down once. + +| Term | Meaning in this book | +| --- | --- | +| Dataplane | The process on the live request path (this repository). It handles MCP traffic. | +| Control plane | The external ContextForge application that authors config, policy, and identity. It is not in this repository. | +| Downstream | The client side of the gateway: the calling MCP client and its requests. | +| Upstream | The backend side of the gateway: the configured backend MCP servers. | +| Virtual host | A named routing group inside the caller's config. The URL path selects exactly one. | +| Backend | One configured MCP server behind the gateway. Its name prefixes every tool, resource, and prompt it exposes. | +| Principal | The authenticated caller identity, taken from the JWT `sub` claim. | +| Session | An initialized MCP session, tracked by `Mcp-session-id`, that owns the per-caller backend client sessions. | + +## Mental model + +The gateway is easiest to understand as one logical MCP server assembled from a +set of configured backend MCP servers. + +On `initialize`, the gateway validates the caller, resolves the requested +virtual host, and creates upstream MCP client sessions for that virtual host's +backends. On list calls, it fans out to those backends and merges the result. On +targeted calls, it uses the backend prefix in the public tool, resource, or +prompt name to route to exactly one backend. + +For a stateful MCP call to work, five facts must line up: + +```text +JWT subject + -> user config + -> virtual host id + -> downstream MCP session id + -> backend session map entries +``` + +If any part is missing, the gateway should fail at the layer that owns that +fact: auth, config lookup, virtual host resolution, session lookup, routing, or +upstream transport. + +> **Operational consequence:** stateful MCP traffic needs session affinity +> until backend session state moves out of the gateway process. + +## Non-goals + +This repository should not grow control-plane behavior by accident. It should +not become the admin UI, tenant management API, policy authoring system, +credential store, or durable observability backend. + +It also should not expose backend topology as more than the MCP routing +contract requires. Backend names are visible in prefixed tool, resource, and +prompt names, but clients should still experience one gateway endpoint and one +logical MCP server. + +## Where to go next + +- To run the gateway yourself, start with [Getting Started](usage.md). +- To understand how requests move through it, read [Architecture](architecture.md). +- For the exact client-facing protocol behavior, see [MCP Behavior](mcp-behavior.md).