feat: create agent context in line with bob ai training

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

Author: Lang-Akshay

AGENTS.md

@@ -1,5 +1,18 @@
 # AGENTS.md
 
+## Start with the wiki
+
+At the start of each task, check `_context/wiki/index.md` to decide
+whether wiki context is needed before acting. Don't read the wiki in
+full. Use the index and follow links only when they are relevant to
+the task.
+
+## Update the wiki
+
+After completing a task, offer to update the wiki if the task yielded durable knowledge that could benefit future work, then wait for user approval. This includes new processes, architecture decisions, or insights that go beyond the immediate task.
+
+---
+
 Guidance for agents working on `contextforge-gateway-rs`.
 
 This repo is the Rust dataplane part of ContextForge. It must stay compatible

_context/wiki/index.md

@@ -0,0 +1,21 @@
+# ContextForge Data Plane — Wiki
+
+This wiki captures durable project context and working preferences.
+Check this index at the start of a task to decide whether deeper context is needed,
+then follow only the links that are relevant.
+
+## Pages
+
+| File | What it covers |
+| --- | --- |
+| [project.md](project.md) | What the project is, goals, stakeholders, key modules, active work |
+| [preferences.md](preferences.md) | Working standards, code style, logging rules, AI interaction preferences |
+
+## Quick orientation
+
+- **Repo**: `contextforge-gateway-rs` — the Rust dataplane for ContextForge.
+- **Core invariant**: this crate is pure routing logic. No IAM, UI, or metrics storage.
+- **Protocol target**: MCP `2026-07-28` over Streamable HTTP. Legacy SSE paths are being removed.
+- **Architecture book**: [`docs/book/src/`](../../docs/book/src/) — read the relevant page before touching the hot path.
+- **Validation gate**: `cargo test` + `cargo clippy` must be clean before any change is done.
+- **System topology**: `client → nginx → [dataplane | control-plane]`; config flows from control-plane via `dataplane_publisher.py` → Redis → dataplane. See [project.md § System topology](project.md#system-topology).

_context/wiki/preferences.md

@@ -0,0 +1,54 @@
+# Working Preferences and Standards
+
+## Validation gate — definition of "done"
+
+A change is not done until:
+1. `cargo test` passes with no failures.
+2. `cargo clippy` is clean — no new warnings.
+3. If the change touches the hot path, the matching book page in [`docs/book/src/`](../../docs/book/src/) is updated in the same change.
+
+## Code style
+
+- **Idiomatic Rust** — no unnecessary clones, heap allocations, `Arc`, or `Mutex` unless justified by the design.
+- Most product behavior lives in `contextforge-gateway-rs-lib`. Do not let dataplane logic accumulate in the binary crate.
+- Typed errors — propagate errors rather than swallowing them silently.
+- Keep change size minimal. Every changed line must trace directly to the task at hand.
+
+## Logging (tracing)
+
+- Use `tracing` for all log output.
+- **Prefer message-embedded fields**: `level!("method_name - event field = {val} other_field = {other}")`.
+  Do **not** use structured field syntax (`, field = val`) for dataplane logs.
+- Keep method/event prefixes stable and reuse the same field names and order for related events.
+- `warn!` is for unexpected conditions that need operator attention. Expected user/config misses → `debug!` or `info!`.
+- **Never log**: tokens, authorization headers, secrets, Redis key/value bytes, full `UserConfig`, or backend credentials.
+
+## Change discipline
+
+- Make the **minimal change** that solves the problem. No speculative refactors, no added abstractions beyond the task scope.
+- Do not clean up surrounding code that is unrelated to the task.
+- Do not add error handling for scenarios that cannot happen.
+- Always **read relevant code before suggesting or making changes**. Never speculate about code that hasn't been opened.
+
+## Architectural rules (non-negotiable)
+
+- The dataplane is pure routing logic. **No IAM, UI, or metrics-storage concerns.**
+- Config access goes through `UserConfigStore` only — never push Redis details into routing code.
+- The backend prefix naming contract must not change without updating merge logic, split logic, and tests.
+- Legacy SSE transport and old `initialize`/session behavior are being **removed** — do not build on temporary shims.
+- Prefer the right architecture over backward compatibility; this project has no external users yet.
+
+## Protocol target
+
+- All new behavior targets MCP protocol version **`2026-07-28`** over **Streamable HTTP**.
+- New tests and examples use `server/discover`, per-request client metadata, and the `2026-07-28` version.
+- Do not add new compatibility for older MCP protocol versions.
+
+## AI interaction preferences
+
+- **Read before acting**: always investigate relevant files before making suggestions or edits.
+- **Minimal scope**: stay tightly scoped to the task — no unsolicited refactors or cleanups.
+- **Plan first for complex tasks**: for changes with multiple moving parts, propose the approach before implementing.
+- **Run validation**: run `cargo test` and `cargo clippy` after changes and report results before declaring done.
+- **Update the book**: when hot-path behavior changes, include the book page update in the same task.
+- **No hallucination**: if something is unclear, ask rather than guess.

_context/wiki/project.md

@@ -0,0 +1,100 @@
+# Project Overview
+
+## What this project is
+
+`contextforge-gateway-rs` is a Rust-based MCP (Model Context Protocol) gateway — the **dataplane** component of ContextForge. It acts as a scalable, secure proxy layer that routes AI tool calls from MCP clients to one or more backend MCP servers.
+
+It is paired with the external ContextForge control plane at [`IBM/mcp-context-forge`](https://github.com/IBM/mcp-context-forge). The two components have a strict division of responsibility:
+
+| Layer | Owns |
+| --- | --- |
+| **This repo (dataplane)** | Request routing, auth enforcement, backend fan-out, session ownership |
+| **Control plane** | IAM, UI, metrics storage, legacy MCP client compatibility |
+
+The dataplane must never take on control-plane concerns.
+
+## Goals and objectives
+
+- Provide a **production-grade, low-latency routing layer** between MCP clients and backend MCP servers.
+- Target **MCP protocol version `2026-07-28`** over Streamable HTTP as the sole downstream contract.
+- Enforce a clean **dataplane/control-plane boundary** — no IAM, UI, or metrics storage logic in this repo.
+- Keep config access behind the **`UserConfigStore` abstraction** (backed by Redis/MessagePack).
+- Remain in the right architectural shape during early development, prioritising correctness over backward compatibility.
+
+## Key stakeholders and users
+
+- **Platform teams** — deploy and operate the gateway as infrastructure.
+- **AI application developers** — use the gateway as the MCP proxy layer for their applications.
+- **Internal contributors** — engineers evolving the dataplane toward the `2026-07-28` protocol target.
+
+## Key modules and architecture
+
+The architecture book at [`docs/book/src/`](../../docs/book/src/) is the authoritative reference. Key pages:
+
+| Page | Covers |
+| --- | --- |
+| [`system-shape.md`](../../docs/book/src/system-shape.md) | Crate layout, pipeline shape, state ownership, module boundaries |
+| [`request-flow.md`](../../docs/book/src/request-flow.md) | Startup, middleware order, fan-out, response path |
+| [`mcp-routing-semantics.md`](../../docs/book/src/mcp-routing-semantics.md) | Backend prefix namespace and routing contract |
+| [`authentication-and-user-config.md`](../../docs/book/src/authentication-and-user-config.md) | JWT validation, config keying, cache behavior |
+| [`architectural-choices.md`](../../docs/book/src/architectural-choices.md) | Invariants and tradeoffs that must not change accidentally |
+
+**Crate structure:**
+- `contextforge-gateway-rs-lib` — all product/routing logic lives here.
+- Binary crate — thin entrypoint only. Dataplane logic must not accumulate here.
+
+**Key invariants:**
+- Redis/config access goes through `UserConfigStore` only — never leak Redis details into routing code.
+- The backend prefix naming contract must not change without updating merge logic, split logic, and tests.
+- When behavior on the hot path changes, the matching book page must be updated in the same change.
+
+## Active work (near-term)
+
+- **Protocol migration**: replacing all remaining legacy MCP paths (SSE transport, `initialize`/session shims) with `2026-07-28` equivalents over Streamable HTTP.
+- Legacy SSE transport and old session behavior are **being removed**, not maintained. Do not build new behavior on temporary shims.
+- New tests and examples should use `server/discover`, per-request client metadata, and protocol version `2026-07-28`.
+
+## System topology
+
+All external traffic enters through **nginx**, which fans out to either the dataplane or the control plane:
+
+```mermaid
+flowchart LR
+    client(["client"]) --> nginx["nginx"]
+    nginx --> dataplane["data-plane"]
+    nginx --> controlplane["control-plane"]
+    dataplane --> redis["redis"]
+    controlplane --> redis
+    controlplane --> postgres["postgres\n(via pgbouncer)"]
+    dataplane --> fastts["fast_time_server"]
+```
+
+### How the control plane publishes config to the dataplane
+
+The control plane and dataplane do **not** communicate over HTTP. Config is exchanged exclusively through Redis:
+
+1. The control plane runs **`dataplane_publisher.py`** — a publisher script that writes dataplane configuration (user config, backend definitions, etc.) into Redis.
+2. The dataplane reads that config from Redis via the **`UserConfigStore`** abstraction (MessagePack-encoded `UserConfig`).
+
+This means:
+- The dataplane is a **pure reader** of Redis config. It never writes back to the control-plane's Redis keys.
+- The control plane is the **sole writer** of dataplane config; the dataplane has no direct dependency on the control-plane process at runtime.
+- Config changes from the control plane are picked up by the dataplane through normal cache refresh / Redis reads — no restart or direct RPC required.
+
+### Per-component responsibilities
+
+| Component | Role | Persistence |
+| --- | --- | --- |
+| **nginx** | TLS termination, routing fan-out | — |
+| **dataplane** (`contextforge-gateway-rs`) | MCP routing, auth enforcement, fan-out to backends | Redis (read-only for config) |
+| **control-plane** (`IBM/mcp-context-forge`) | IAM, UI, metrics, legacy MCP clients, config publishing | Redis (write) + PostgreSQL (via pgbouncer) |
+| **redis** | Runtime config store, inter-component pub/sub channel | In-memory + persistence |
+| **postgres** (via pgbouncer) | Control-plane relational store | Durable |
+| **fast_time_server** | High-resolution time source used by the dataplane | — |
+
+## External dependencies and integration points
+
+- **Redis** — runtime config store (MessagePack-encoded `UserConfig`). Populated by `dataplane_publisher.py` on the control plane; read by the dataplane via `UserConfigStore`.
+- **Control plane** (`IBM/mcp-context-forge`) — owns legacy MCP client routes and publishes dataplane config via `dataplane_publisher.py`. Does not route through this dataplane at runtime.
+- **fast_time_server** — high-resolution time source consumed by the dataplane.
+- **Tokio + Axum** — fixed async runtime and web framework.