fix: use opaque dataplane subject ids

[gandhipratik203]

Summary

• Publish dataplane user configs under EmailUser.id instead of email addresses.
• Mint API-token JWT subjects from EmailUser.id while retaining signed user.email metadata.
• Resolve signed user email metadata in token scoping middleware so UUID-sub tokens keep existing team and ownership checks working.

Notes

This is the control-plane portion of the dataplane subject migration. Part of #5462.

Matching Rust dataplane/demo fixture updates are open in contextforge-org/contextforge-data-plane#61.

Migration / Compatibility

Redis user configs are now keyed by the opaque user UUID instead of email. Existing email-keyed Redis entries are not migrated; they expire through the normal dataplane publisher TTL.

Session tokens already use UUID subjects, so they continue to match UUID-keyed configs.

Pre-existing API tokens minted before this change may still carry email subjects. Those tokens must be rotated or recreated during rollout. After the old email-keyed Redis entry expires, an email-sub token no longer matches a published config and dataplane requests can fail with HTTP 400: Problem occurred retrieving the configuration until the token is replaced.

No Rust dataplane fallback is added because the dataplane treats JWT sub as an opaque lookup key and does not own email-to-UUID identity mapping.

Validation Performed

Python unit coverage:

uv run pytest tests/unit/mcpgateway/services/test_token_catalog_service.py \
  tests/unit/mcpgateway/middleware/test_token_scoping.py \
  tests/unit/mcpgateway/services/test_dataplane_publisher.py -q

uv run pytest tests/unit/mcpgateway/transports/test_streamablehttp_transport.py -q

uv run pytest tests/unit/mcpgateway/test_auth_context_email_precedence.py \
  tests/unit/mcpgateway/middleware/test_token_usage_middleware.py \
  tests/unit/mcpgateway/middleware/test_token_scoping.py -q

uv run pytest tests/unit/mcpgateway/test_auth.py -k "uuid_sub or api_token" -q

uv run ruff check mcpgateway/services/dataplane_publisher.py \
  mcpgateway/services/token_catalog_service.py \
  mcpgateway/middleware/token_scoping.py \
  mcpgateway/auth_context.py \
  mcpgateway/transports/streamablehttp_transport.py \
  mcpgateway/middleware/token_usage_middleware.py \
  tests/unit/mcpgateway/services/test_dataplane_publisher.py \
  tests/unit/mcpgateway/services/test_token_catalog_service.py \
  tests/unit/mcpgateway/middleware/test_token_scoping.py \
  tests/unit/mcpgateway/test_auth_context_email_precedence.py \
  tests/unit/mcpgateway/transports/test_streamablehttp_transport.py \
  tests/unit/mcpgateway/middleware/test_token_usage_middleware.py

git diff --check

Cross-repo dataplane smoke test:

1. Started local Redis and one sample MCP backend from the Rust gateway repo:

cd ../contextforge-gateway-rs
docker compose -f docker/docker-compose-local.yaml up -d redis gateway-one

2. Started one Rust dataplane process locally:

cargo +1.96 run --bin contextforge-gateway-rs -- \
  --address 0.0.0.0:8001 \
  --redis-port 6379 \
  --redis-address 127.0.0.1 \
  --token-verification-public-key assets/jwt.key.pub \
  --number-of-cpus 4 \
  --redis-mode=plain-text \
  --upstream-connection-mode=plain-text-or-tls

3. Used the Python dataplane publisher code path to publish one user config to Redis with:

• subject/user key: 11111111-1111-1111-1111-111111111111
• virtual server: c0ffee00f001f00lf00ldeadbeefdead
• backend URL: http://127.0.0.1:5555/mcp

4. Minted an RS256 JWT with:

• sub: 11111111-1111-1111-1111-111111111111
• user.email: admin@example.com

5. Called the Rust dataplane endpoint:

POST http://127.0.0.1:8001/contextforge-rs/servers/c0ffee00f001f00lf00ldeadbeefdead/mcp

Results and confirmation points:

The smoke script stdout confirmed the Python-side Redis write and MCP responses:

published Redis key (UserConfig, 11111111-1111-1111-1111-111111111111)
email Redis key absent for admin@example.com
uuid Redis key TTL 130
initialize status 200
initialized notification status 202
tools/list status 200
tools/list names ['decrement', 'echo', 'get_session_id', 'get_value', 'increment', 'long_task', 'say_hello', 'sum']

The smoke script also asserted that no email-keyed Redis config existed for admin@example.com, so the published key was UUID-only.

The Rust dataplane process logs confirmed that the JWT sub was used as the Redis lookup key and that the config loaded successfully:

user_config_store_layer - getting user config for request subject = 11111111-1111-1111-1111-111111111111
RedisUserConfigStore::get_config - loaded user config blob from Redis subject = 11111111-1111-1111-1111-111111111111
RedisUserConfigStore::get_config - decoded user config subject = 11111111-1111-1111-1111-111111111111 virtual_hosts = 1
user_config_store_layer - loaded user config subject = 11111111-1111-1111-1111-111111111111 virtual_hosts = 1
list_tools: backend gateway-one completed (8 items)

So the result was confirmed in two places:

• Python smoke script output: Redis key written, email key absent, MCP request statuses, returned tool names.
• Rust dataplane logs: UUID subject used for config lookup, Redis config loaded, backend tools/list completed.

Semi-automated cross-repo smoke script

This is opt-in/manual validation. It uses one Redis container, one sample backend container (gateway-one), and one local Rust dataplane process.

Start the Rust-side services:

export CONTEXTFORGE_GATEWAY_RS_REPO=/path/to/contextforge-gateway-rs

cd "$CONTEXTFORGE_GATEWAY_RS_REPO"
docker compose -f docker/docker-compose-local.yaml up -d redis gateway-one

cargo +1.96 run --bin contextforge-gateway-rs -- \
  --address 0.0.0.0:8001 \
  --redis-port 6379 \
  --redis-address 127.0.0.1 \
  --token-verification-public-key assets/jwt.key.pub \
  --number-of-cpus 4 \
  --redis-mode=plain-text \
  --upstream-connection-mode=plain-text-or-tls

In another terminal, from this Python repo branch:

cat > /tmp/uuid_dataplane_smoke.py <<'PY'
"""Cross-repo smoke for UUID-keyed dataplane Redis config."""

from __future__ import annotations

import asyncio
import os
from pathlib import Path
import time
import uuid

DB_PATH = Path("/tmp/contextforge_uuid_dataplane_smoke.db")
RS_REPO = Path(os.environ["CONTEXTFORGE_GATEWAY_RS_REPO"])

USER_ID = "11111111-1111-1111-1111-111111111111"
USER_EMAIL = "admin@example.com"
VIRTUAL_HOST_ID = "c0ffee00f001f00lf00ldeadbeefdead"
GATEWAY_ID = "gateway-one"
BACKEND_URL = "http://127.0.0.1:5555/mcp"
DATAPLANE_URL = f"http://127.0.0.1:8001/contextforge-rs/servers/{VIRTUAL_HOST_ID}/mcp"

TOOL_NAMES = [
    "decrement",
    "echo",
    "get_session_id",
    "get_value",
    "increment",
    "long_task",
    "say_hello",
    "sum",
]

os.environ["DATABASE_URL"] = f"sqlite:///{DB_PATH}"
os.environ.setdefault("REDIS_URL", "redis://127.0.0.1:6379/0")
os.environ.setdefault("JWT_SECRET_KEY", "uuid-dataplane-smoke-test-secret")
os.environ.setdefault("AUTH_ENCRYPTION_SECRET", "uuid-dataplane-smoke-test-encryption-secret")

import httpx
import jwt
import msgpack
import redis.asyncio as redis
from sqlalchemy import insert

from mcpgateway.db import Base, EmailUser, Gateway, Server, SessionLocal, Tool, engine, server_tool_association
from mcpgateway.services.dataplane_publisher import DataplanePublisherService, USER_CONFIG_KEY, get_publisher_ttl


def reset_smoke_db() -> None:
    if DB_PATH.exists():
        DB_PATH.unlink()

    Base.metadata.create_all(bind=engine)

    with SessionLocal() as db:
        db.execute(
            insert(EmailUser).values(
                id=USER_ID,
                email=USER_EMAIL,
                password_hash="not-used-in-smoke",
                full_name="Smoke Test User",
                is_admin=True,
                is_active=True,
                auth_provider="local",
                password_hash_type="argon2id",
            )
        )
        db.execute(
            insert(Gateway).values(
                id=GATEWAY_ID,
                name=GATEWAY_ID,
                slug=GATEWAY_ID,
                url=BACKEND_URL,
                transport="STREAMABLEHTTP",
                capabilities={},
                enabled=True,
                reachable=True,
                status="active",
                owner_email=USER_EMAIL,
                visibility="public",
                passthrough_headers=[],
            )
        )
        db.execute(
            insert(Server).values(
                id=VIRTUAL_HOST_ID,
                name="uuid-smoke-server",
                description="UUID dataplane smoke server",
                enabled=True,
                owner_email=USER_EMAIL,
                visibility="public",
            )
        )

        for tool_name in TOOL_NAMES:
            tool_id = f"tool-{tool_name.replace('_', '-')}"
            db.execute(
                insert(Tool).values(
                    id=tool_id,
                    name=f"{GATEWAY_ID}-{tool_name}",
                    original_name=tool_name,
                    url=f"{BACKEND_URL}#{tool_name}",
                    input_schema={"type": "object", "properties": {}},
                    output_schema=None,
                    custom_name=tool_name,
                    custom_name_slug=tool_name.replace("_", "-"),
                    display_name=tool_name,
                    integration_type="MCP",
                    request_type="STREAMABLEHTTP",
                    headers={},
                    enabled=True,
                    reachable=True,
                    gateway_id=GATEWAY_ID,
                    owner_email=USER_EMAIL,
                    visibility="public",
                )
            )
            db.execute(insert(server_tool_association).values(server_id=VIRTUAL_HOST_ID, tool_id=tool_id))

        db.commit()


async def publish_uuid_config() -> None:
    service = DataplanePublisherService()
    payload = await service.fetch_payload()
    if payload is None:
        raise RuntimeError("publisher returned None")
    if USER_ID not in payload:
        raise RuntimeError(f"publisher payload missing UUID key {USER_ID}; keys={list(payload)}")
    if USER_EMAIL in payload:
        raise RuntimeError("publisher payload unexpectedly contains an email key")

    client = redis.Redis(host="127.0.0.1", port=6379, decode_responses=False)
    uuid_key = msgpack.dumps((USER_CONFIG_KEY, USER_ID), use_bin_type=True)
    email_key = msgpack.dumps((USER_CONFIG_KEY, USER_EMAIL), use_bin_type=True)
    await client.delete(uuid_key, email_key)
    await client.set(uuid_key, msgpack.dumps(payload[USER_ID], use_bin_type=True), ex=get_publisher_ttl())

    ttl = await client.ttl(uuid_key)
    email_exists = await client.exists(email_key)
    await client.aclose()

    if email_exists:
        raise RuntimeError("email-keyed Redis config exists after cleanup/publish")

    print(f"published Redis key ({USER_CONFIG_KEY}, {USER_ID})")
    print(f"email Redis key absent for {USER_EMAIL}")
    print(f"uuid Redis key TTL {ttl}")


def mint_uuid_token() -> str:
    now = int(time.time())
    private_key = (RS_REPO / "assets/jwt.key").read_bytes()
    claims = {
        "sub": USER_ID,
        "jti": str(uuid.uuid4()),
        "token_use": "api",
        "iat": now,
        "iss": "mcpgateway",
        "aud": "mcpgateway-api",
        "exp": now + 3600,
        "teams": ["team_awesome"],
        "user": {
            "email": USER_EMAIL,
            "auth_provider": "api_token",
            "full_name": "Smoke Test User",
            "is_admin": True,
        },
    }
    return jwt.encode(claims, private_key, algorithm="RS256", headers={"kid": "test"})


def parse_jsonrpc_response(response: httpx.Response) -> dict:
    text = response.text.strip()
    if text.startswith("event:") or text.startswith("data:"):
        data_lines = [line.removeprefix("data:").strip() for line in text.splitlines() if line.startswith("data:")]
        text = "\n".join(data_lines)
    return response.json() if text == response.text.strip() else __import__("json").loads(text)


async def call_dataplane(token: str) -> None:
    headers = {
        "authorization": f"Bearer {token}",
        "content-type": "application/json",
        "accept": "application/json, text/event-stream",
    }

    async with httpx.AsyncClient(timeout=20.0) as client:
        init_response = await client.post(
            DATAPLANE_URL,
            headers=headers,
            json={
                "jsonrpc": "2.0",
                "id": 0,
                "method": "initialize",
                "params": {
                    "protocolVersion": "2025-11-25",
                    "capabilities": {},
                    "clientInfo": {"name": "uuid-smoke", "version": "0.1.0"},
                },
            },
        )
        print(f"initialize status {init_response.status_code}")
        init_response.raise_for_status()

        session_id = init_response.headers.get("mcp-session-id")
        if not session_id:
            raise RuntimeError("initialize response did not include mcp-session-id")

        session_headers = {**headers, "mcp-session-id": session_id, "mcp-protocol-version": "2025-11-25"}

        initialized_response = await client.post(
            DATAPLANE_URL,
            headers=session_headers,
            json={"jsonrpc": "2.0", "method": "notifications/initialized"},
        )
        print(f"initialized notification status {initialized_response.status_code}")
        initialized_response.raise_for_status()

        tools_response = await client.post(
            DATAPLANE_URL,
            headers=session_headers,
            json={"jsonrpc": "2.0", "id": 1, "method": "tools/list"},
        )
        print(f"tools/list status {tools_response.status_code}")
        tools_response.raise_for_status()

        body = parse_jsonrpc_response(tools_response)
        names = [tool["name"] for tool in body["result"]["tools"]]
        print(f"tools/list names {names}")

        missing = sorted(set(TOOL_NAMES) - set(names))
        if missing:
            raise RuntimeError(f"tools/list missing expected tools: {missing}")


async def main() -> None:
    reset_smoke_db()
    await publish_uuid_config()
    await call_dataplane(mint_uuid_token())


if __name__ == "__main__":
    asyncio.run(main())
PY

uv run python /tmp/uuid_dataplane_smoke.py

Expected output:

published Redis key (UserConfig, 11111111-1111-1111-1111-111111111111)
email Redis key absent for admin@example.com
uuid Redis key TTL 130
initialize status 200
initialized notification status 202
tools/list status 200
tools/list names ['decrement', 'echo', 'get_session_id', 'get_value', 'increment', 'long_task', 'say_hello', 'sum']

Cleanup:

cd "$CONTEXTFORGE_GATEWAY_RS_REPO"
docker compose -f docker/docker-compose-local.yaml down

Live ContextForge protocol/RBAC validation:

A fresh clone of this PR branch was built and run on test-vm-cf.

Environment:

• Branch: issue-5462-dataplane-uuid-subjects
• Gateway URL: http://127.0.0.1:8080
• Runtime: Python MCP transport (x-contextforge-mcp-transport-mounted: python)
• Gateway: single running gateway container behind nginx
• Image: mcpgateway/mcpgateway:latest built from this PR branch

Commands:

MCP_CLI_BASE_URL=http://127.0.0.1:8080 \
JWT_SECRET_KEY=my-test-key-but-now-longer-than-32-bytes \
PLATFORM_ADMIN_EMAIL=admin@example.com \
MCP_E2E_CLIENT_TIMEOUT=15 \
uv run pytest tests/live_gateway/mcp/test_mcp_protocol_e2e.py -v -s --tb=short

Result:

19 passed, 3 skipped in 6.34s

MCP_CLI_BASE_URL=http://127.0.0.1:8080 \
JWT_SECRET_KEY=my-test-key-but-now-longer-than-32-bytes \
PLATFORM_ADMIN_EMAIL=admin@example.com \
uv run pytest -p playwright tests/live_gateway/mcp/test_mcp_rbac_transport.py -v -s --tb=short

Result:

40 passed in 19.79s

Note: the first protocol run exposed stale/incomplete test-stack setup: fast-time-* tools were missing from the live catalog. After rerunning the register_fast_time setup job, the catalog contained both fast-time-* and fast-test-* tools, and the full protocol suite passed.

Single-instance Playwright token smoke

Ran against the rebased PR head 5fc641e791 on test-vm-cf, using a single make dev ContextForge instance at http://localhost:8000.

Setup:

• Installed Node.js v22.23.1 / npm 10.9.8 on the VM so the admin UI bundle could be built.
• Ran make js-build.
• Started ContextForge with make dev.
• Verified GET /health returned 200.

Command shape:

TEST_BASE_URL=http://localhost:8000 \
JWT_SECRET_KEY=<throwaway-test-secret> \
AUTH_ENCRYPTION_SECRET=<throwaway-test-encryption-secret> \
BASIC_AUTH_PASSWORD=<throwaway-test-basic-password> \
PLATFORM_ADMIN_PASSWORD=<throwaway-test-admin-password> \
MCPGATEWAY_UI_ENABLED=true \
MCPGATEWAY_ADMIN_API_ENABLED=true \
AUTH_REQUIRED=true \
uv run pytest -p playwright \
  tests/playwright/security/test_token_lifecycle.py \
  tests/playwright/security/test_token_lifecycle_enforcement.py \
  tests/playwright/security/test_token_scope_matrix.py \
  --browser chromium -q

Result:

16 passed, 2 warnings in 10.67s

This covers API token create/list/update/revoke, using newly generated API tokens as bearer tokens, scoped-token permission enforcement, revocation enforcement, and the admin UI token revoke flow.

[lucarlig]

Hi @gandhipratik203 — thanks for pushing the opaque-subject migration through the publisher, token minting, and Streamable HTTP paths. I reviewed cdf369e6 and reran the focused tests. The overall direction looks good, but I found two blocking compatibility issues:

1. Session tokens lose team/admin scope in token-scoping middleware. Session JWTs created by create_access_token() use a UUID sub without embedded email metadata. get_jwt_user_email_from_payload() therefore returns None, and resolve_session_teams() immediately returns [] without performing its UUID-to-email or DB team lookup. Targeted team/private resource requests are then denied, including for admins. The current session-token tests use email-valued subjects, so they do not exercise the production token shape.

2. New API tokens are rejected by shared require_auth paths. API tokens now have UUID subjects, but _enforce_revocation_and_active_user() only resolves UUID subjects when token_use == "session". With the default REQUIRE_USER_IN_DB=true, it queries EmailUser.email == <uuid> and returns 401. This affects the reverse-proxy, metrics, well-known-status, and docs-auth paths that use require_auth.

I reproduced both cases independently. The four focused pytest invocations listed in the PR pass locally, and the full CI pytest run itself passed (20,743 passed, 771 skipped, 2 xfailed). The job currently fails only at the changed-line coverage gate: 87% versus the required 93%.

Could you update both identity-resolution paths and add regression tests using the actual UUID-only session payload and a newly minted UUID-sub API token through require_auth? That should also provide the missing coverage needed for CI.

[gandhipratik203]

Thanks, fixed in a5cba64ab.

This resolves UUID-only session subjects before session team scope, fixes UUID-sub API tokens in require_auth, and adds regression tests for both paths.

Validated with the focused auth/scoping/transport pytest suites plus ruff and git diff --check.

[lucarlig]

Blocking — reverse-proxy ownership still compares UUID to email

This is separate from the resolved require_auth issue: authentication now succeeds, but the downstream ownership identities differ. New API tokens carry the opaque UUID in sub and the signed email in user.email. Reverse-proxy WebSocket authentication goes through get_current_user() and stores the session owner as the resolved email, while the HTTP session endpoints receive the raw payload from require_auth() and _get_user_from_credentials() still selects sub first. For the same non-admin API token, /sessions therefore hides its own session and the disconnect/request/SSE paths return 403 because <uuid> != <email>.

I reproduced this with a UUID-sub payload containing user.email: _get_user_from_credentials() returned the UUID, followed by 403 Not authorized against a session owned by that email.

Would you update the reverse-proxy credential extraction to use the canonical signed-email helper (for example, get_jwt_user_email_from_payload()) and add a regression test covering UUID sub + nested user.email against an email-owned session?

[gandhipratik203]

Thanks Luca, fixed in 87054f732.

Reverse-proxy credential/session-owner extraction now uses the canonical signed-email helper, so UUID sub + user.email tokens compare against email-owned sessions correctly. Added regressions for _get_user_from_credentials, session listing, and ownership validation.

Validated with:
uv run pytest tests/unit/mcpgateway/routers/test_reverse_proxy.py -q
ruff check/format --check on the touched files

[msureshkumar88]

Nice fix for the core problem in #5462 — minting the API-token sub from EmailUser.id and keying the Redis UserConfig publisher off that same UUID is the right approach, and the test coverage on the touched files (UUID-with-metadata, UUID-without-metadata, unknown-UUID rejection, reverse-proxy ownership, publisher keying) is solid.

A few things worth resolving before merge — the PR introduces a new canonical pattern for resolving email from a JWT payload (auth_context.get_jwt_user_email_from_payload / resolve_jwt_user_email_from_payload), but a couple of call sites that are just as security-relevant as the ones you did update still run the old logic:

Blocking:

1. mcpgateway/middleware/csrf_middleware.py (~line 149) — the fallback identity resolution still does payload.get("sub") or payload.get("email") or ..., which picks the raw UUID before checking signed user.email. The primary path a few lines up binds CSRF tokens to user.email, so a UUID-sub token hitting the fallback branch could get a CSRF mismatch. Worth swapping to get_jwt_user_email_from_payload() for consistency with the rest of the PR.
2. mcpgateway/auth.py — get_current_user() (the dependency behind Admin UI cookie sessions, CSRF issuance, logout, password-change, and MCP initialize/ping/websocket) and get_user_email_from_token() still have their own independent UUID→email resolution that doesn't check signed user.email metadata first, and on an unresolved UUID one of them keeps the raw UUID as email rather than failing closed like the rest of this PR does. Given how central this path is, could these be migrated to the new helper in the same PR rather than left as follow-up?
3. mcpgateway/middleware/token_usage_middleware.py (~line 232) — the new JTI-fallback lookup does except Exception: user_email = None with no log line, unlike the rest of the file. A real DB error here becomes indistinguishable from "token not found" during an incident. A logger.debug/warning would help.

Worth a look, not blocking:

• verify_credentials._enforce_revocation_and_active_user: when require_user_in_db=False and a UUID sub can't be resolved, the function returns before ever checking is_active — under that config the active-user gate is silently skipped for unverifiable UUID identities. Might be intentional given the existing dev-mode leniency, just flagging.
• The Redis TTL cutover for legacy email-sub tokens (documented in the PR body) is a reasonable tradeoff, but a log line when an old-format token is seen post-cutover would make the rollout easier to observe/debug.

Happy to take another pass once these land — the core direction here is good.

[msureshkumar88]

One more thing worth calling out explicitly: this is a breaking change for existing deployments, not just an internal refactor.

Pre-existing API tokens carry sub=email. They'll keep authenticating fine after this merges (JWT signature still checks out), but dataplane MCP calls against them will start failing once the old email-keyed Redis UserConfig entry's TTL expires — the token's sub no longer matches any published config, so dataplane requests fail with HTTP 400: Problem occurred retrieving the configuration. Session tokens are unaffected (already UUID-sub before this PR).

This is already disclosed in the PR description's "Migration / Compatibility" section, which is great, but there's no code-level signal for it — no warning log when a legacy email-sub token is used post-cutover, and no dual-publish/migration window to soften the transition. Right now the only way an operator finds out a token needs rotating is via the 400s in production.

Given the blast radius (every pre-existing API token routed through the dataplane), could we either:

• log a warning server-side when a legacy email-sub token is seen after cutover, so this is discoverable before it pages someone, and/or
• call this out as a breaking change in the release notes / upgrade guide with an explicit "rotate your API tokens before upgrading" step

Not asking to block on a full migration shim — just want to make sure this doesn't surprise anyone in production.

[msureshkumar88]

Dug into the actual publisher/config code to size the breaking-change risk more precisely — wanted to correct/sharpen my earlier comment rather than leave a vague "this could break things."

There are two separate breaking-change vectors here, with very different blast radii:

Vector A — JWT sub shape change (token_catalog_service.py)
Unconditional, ships on every install regardless of feature flags. But it only affects newly minted API tokens — already-issued tokens are immutable JWTs and keep sub=email until they expire/rotate, so this is a slow-burn exposure, not a cliff. Session tokens are unaffected (already UUID sub pre-PR). The risk here is that every token minted after this ships exercises the sub-as-email gaps flagged in the earlier review comment (get_current_user, csrf_middleware.py, get_user_email_from_token) — permanently, app-wide, not just on dataplane routes.

Vector B — Redis UserConfig key cutover (dataplane_publisher.py)
Confirmed this is gated behind dataplane_publisher (config.py, default False) — the background task only starts if settings.dataplane_publisher (main.py:1769), and the module docstring calls it out as WIP/disabled-by-default. So by default this vector affects no one.

For the subset of deployments that do set dataplane_publisher=True, though, the cutover is sharper than "expires eventually": dataplane_publisher_interval_seconds defaults to 60s, and the TTL is interval*2+10 = 130s. The publisher writes UUID-only keys on its very next cycle after deploy; old email-keyed entries aren't migrated, just left to expire on their existing TTL. So in practice every active email-sub API token routed through the dataplane stops working within ~130 seconds of deploy, globally, all at once — a hard cutover rather than a grace period.

Suggested mitigation for vector B — dual-publish during a deprecation window, no Rust-side changes needed since the dataplane treats sub as opaque:

# get_data_from_db() already has both id and email per user_row
return {
    key: self._build_user_data(user_email, ...)
    for user_email, teams in user_teams_map.items()
    for key in (
        [user_subject_key_by_email[user_email], user_email]
        if settings.dataplane_publisher_dual_key_compat
        else [user_subject_key_by_email[user_email]]
    )
}

Add a dataplane_publisher_dual_key_compat setting (default True), publish under both the UUID key (canonical) and the email key (deprecated) for a release or two, then flip the default and drop the email branch in a follow-up once operators have had time to rotate tokens.

For vector A — this reinforces the earlier ask to migrate get_current_user, get_user_email_from_token, and the CSRF fallback onto resolve_jwt_user_email_from_payload, since new UUID-sub tokens will hit those paths from day one regardless of the dataplane flag.

For both — a rate-limited log line when a legacy email-sub token is seen post-cutover on a dataplane-routed request would turn "users report 400s" into "ops sees a list of tokens that still need rotation."

Given dataplane_publisher defaults off, vector B likely isn't live in production for the general user base today, so this doesn't need to block merge on its own — but worth having the dual-key compat path ready before anyone flips that flag on.

[lucarlig]

Thanks — the UUID-sub authentication, token-scoping, and reverse-proxy ownership fixes now look good, and the focused regression suites are green.

Blocking documentation follow-up

Could you update the JWT documentation and examples that still describe sub as the user email? New API tokens now use EmailUser.id (UUID) in sub, retain the human identity in signed user.email, and continue accepting legacy email-sub tokens.

At minimum, the following are stale:

• docs/docs/architecture/security-features.md still shows "sub": user.email as the email-auth token shape.
• mcpgateway/auth.py documents sub as “The user unique identifier (email).”

Please also check adjacent user-facing token examples for the same assumption and clarify the UUID-sub versus legacy email-sub behavior. This keeps the documented JWT contract aligned with the implementation.

[lucarlig]

@msureshkumar88 Thanks for flagging the legacy email-sub observability point. Since the dataplane has not been released or deployed yet, there is no live legacy dataplane token population or production cutover to observe. We can omit a post-cutover warning/log in this PR; the documented rotation note is sufficient for pre-release fixtures and local environments. We can revisit rollout-specific telemetry when the dataplane is introduced into a live environment.

[msureshkumar88]

Fair enough on the telemetry — agreed it's not worth carrying for a WIP-flagged feature. Two small corrections/asks and then I'm happy:

• The publisher module has actually shipped in v1.0.3–v1.0.6 (added in d2d77009f), just default-off. So the population isn't strictly zero — anyone who set dataplane_publisher=true on those releases would see the ~130s cutover. Given the module docstring flags it as WIP with no production guarantees, I'm fine treating that as out of scope.
• The Migration/Compatibility section in the PR description covers this well. Could that rotation step get mirrored into the release notes / upgrade guide so it's discoverable outside the PR?

On vector A — I re-checked and withdraw most of it: get_current_user and get_user_email_from_token already resolve UUID subjects, so new UUID-sub tokens are fine on those paths. The one loose end is csrf_middleware.py:149, which reads sub first while token generation uses user.email — but that's pre-existing (session tokens were already UUID-sub), so it belongs in a separate issue rather than here.

[gandhipratik203]

Thanks for the review. I pushed the follow-up changes.

What changed:

• Centralized JWT email resolution so signed user.email is preferred, UUID sub resolves through the DB when needed, and legacy email sub still works.
• Updated /mcp streamable auth, CSRF fallback identity, REST token helpers, and token usage logging to use that canonical resolution path.
• Added regression coverage for UUID-sub auth, unresolved UUID denial, legacy email-sub compatibility, CSRF fallback, and usage logging attribution.
• Updated the docs examples to show opaque sub with human email in signed metadata.

[lucarlig]

LGTM

[msureshkumar88]

Thanks for pushing this through the review rounds — the direction (opaque EmailUser.id subject, signed user.email metadata, and the centralized auth_context.get_jwt_user_email_from_payload/resolve_jwt_user_email_from_payload resolver) is the right shape, and the fail-closed behavior on unresolvable UUID subjects is a real security improvement over the previous "keep the UUID as email" fallback.

Two things on the current head commit (04d350ec6) look like they'd block merge on their own:

Blocking

1. CI is currently red — pylint (mcpgateway) fails on a new cyclic import. mcpgateway/api/v1/__init__.py:1: R0401: Cyclic import (mcpgateway.auth -> mcpgateway.auth_context). auth_context.py already has a module-level from mcpgateway.auth import normalize_token_teams (pre-existing, one-directional on main). This PR adds the reverse edge via the new function-local imports in auth.py (from mcpgateway.auth_context import resolve_jwt_user_email_from_payload / get_jwt_user_email_from_payload) — pylint's import-graph check tracks these regardless of scope, so # pylint: disable=import-outside-toplevel doesn't suppress it. Could we close the cycle instead of accepting it — e.g. drop auth_context.py's dependency on normalize_token_teams from auth.py, or relocate the new helpers so the dependency only flows one direction?

2. .secrets.baseline needs a regen. The pre-commit detect-secrets hook is failing on CI (pure line-number drift from the diff shifting other files, no new secrets) — just needs make detect-secrets-scan and the updated baseline committed.

Worth considering (non-blocking, feature is still default-off)

3. verify_credentials.py::require_admin_auth (~line 1677) still has its own independent UUID-detection logic (hand-rolled regex + direct EmailUser.id query) rather than using the new resolve_jwt_user_email_from_payload helper. I didn't find a live bug here — the regex fallback resolves correctly — but it's now the last unconsolidated "is this sub a UUID" implementation in the codebase, and that's exactly the pattern that produced most of the blocking findings earlier in this review. Might be worth folding onto the canonical helper while it's fresh.
4. No log line yet when a legacy email-sub token is seen post-cutover — agreed this doesn't need to hold up this PR since dataplane_publisher ships default-off, but flagging it as something to land before that flag is promoted toward default-on, given the ~130s global cutover once it's enabled.
5. A dual-key Redis publish window (UUID + email, gated by a dataplane_publisher_dual_key_compat-style setting) was discussed earlier in the thread as a way to soften that cutover — same "before wider rollout" caveat as above, not this PR.
6. Could the rotation-step guidance from the PR description's "Migration / Compatibility" section also get mirrored into docs/docs/manage/upgrade.md? Right now it only lives in the PR body, which won't be discoverable post-merge.

One more note not really actionable by you: the last LGTM review is showing as DISMISSED against the current head (a test-only commit landed after it), so there's no standing approval on 04d350ec6 right now — probably just needs a re-look once the CI items above are sorted.

Happy to take another pass once the pylint/secrets-baseline items are in.

[gandhipratik203]

Updated the PR with the CI fixes:

• Regenerated .secrets.baseline.
• Removed the remaining auth_context -> auth pylint cycle by moving normalize_token_teams into auth_context.py and re-exporting it from auth.py.
• Updated the token migration tests for the DB-session based lookup path.

Local checks pass: focused auth/token tests, pylint, and make detect-secrets-hook.

[msureshkumar88]

Re-reviewed the current head (cb6fb6cf). Nice work pushing this through — the opaque-subject direction and the centralized get_jwt_user_email_from_payload/resolve_jwt_user_email_from_payload resolver are the right shape, and the fail-closed behavior on unresolvable UUID subjects is a real improvement over the old "keep the UUID as email" fallback.

Both previously-blocking items are resolved:

1. Cyclic import — confirmed fixed. normalize_token_teams moved into auth_context.py, which no longer imports anything from auth.py at module or function scope. Verified with pylint --enable=R0401 on auth.py/auth_context.py/api/v1/__init__.py — clean. CI's pylint (mcpgateway) job is green.
2. .secrets.baseline — regenerated locally via make detect-secrets-scan; diff against the committed baseline is a no-op (timestamp only, zero finding drift).

Ran the PR's full stated unit-test list locally (11 files) — all pass.

One thing worth a conscious decision before merge (not blocking, but flagging clearly):

TokenCatalogService._generate_token() is the single general-purpose minter behind the standard /tokens API-token-catalog endpoint — not a dataplane-specific path. It now unconditionally sets sub = str(user.id) for every newly-issued API token platform-wide, regardless of whether dataplane_publisher is enabled (it's default=False). Issue #5462's stated scope is "control-plane token generation and Redis UserConfig publication for CF Dataplane integration" — this change is broader than that: any external tool/integration that decodes a ContextForge API token and reads sub as an email will start seeing a UUID for all new tokens, not just dataplane-bound ones. Inbound compat is solid (legacy email-sub tokens still authenticate fine), but this is an outbound breaking change for anyone integrating against sub externally.

Worth considering: gate the new sub format behind settings.dataplane_publisher (or a dedicated flag), or if the broader change is intentional, call it out explicitly in the "Migration/Compatibility" section as affecting all API tokens rather than just dataplane traffic.

Suggestions (non-blocking):

• The "reject unresolved UUID sub" check is now implemented three times with slightly different logic: _enforce_revocation_and_active_user (verify_credentials.py), the inline gate in _StreamableHttpAuthHandler._auth_jwt (streamablehttp_transport.py), and the hand-rolled regex path in require_admin_auth. Worth consolidating behind one function at some point — divergent copies of the same security-critical check are how one gets a fix the others don't.
• async def resolve_uuid_subject(user_id): return await asyncio.to_thread(_get_email_by_id_sync, user_id) is defined identically four times (auth.py, middleware/token_scoping.py, utils/verify_credentials.py, transports/streamablehttp_transport.py). Could be a single shared resolver imported at each call site.
• mcpgateway/utils/time_restrictions.py (9 call sites) and mcpgateway/admin.py:4963 still log payload.get("sub") directly into audit/security log lines — these will now print a UUID instead of an email. Not a functional break, but worth routing through get_jwt_user_email_from_payload() for log readability during incident response.
• test_generate_token_basic (the no-user-object fallback path) checks user_data["email"] but doesn't explicitly assert data["sub"] == user_email for that branch — cheap to add.

Minor:

• security-features.md/a2a.md docs are accurate but don't call out that the sub format change applies to all API tokens platform-wide, not just dataplane-bound ones — one line for operators integrating externally against sub would help.

No Alembic migration needed here and correctly none was added — EmailUser.id is a pre-existing UUID primary key, this just starts using it as the JWT subject. No unrelated changes found in the diff.

[gandhipratik203]

Thanks, updated this to make the compatibility scope explicit.

The docs now state that newly issued token-catalog API tokens use opaque UUID sub platform-wide, not just for Rust dataplane traffic, and that the human email is carried in signed user.email metadata. Legacy, hand-minted, and no-user-record fallback tokens with sub=<email> still authenticate for compatibility.

I also added the explicit fallback assertion in test_generate_token_basic.

[msureshkumar88]

Ran a full live end-to-end validation against a real running gateway (not just the unit suites) to close out the open review rounds. Summary below — everything checked out, approving.

Environment

Isolated stack, nothing shared with any other deployment:

• Gateway: uv run uvicorn mcpgateway.main:app --host 127.0.0.1 --port 14444
• Redis: fresh redis:latest container, port 16379
• DB: fresh sqlite file (throwaway)
• Config: DATAPLANE_PUBLISHER=true, DATAPLANE_PUBLISHER_INTERVAL_SECONDS=3 (fast publish cycle), AUTH_REQUIRED=true, PUBLIC_REGISTRATION_ENABLED=true, MCPGATEWAY_ADMIN_API_ENABLED=true, PLUGINS_ENABLED=false, JWT_SECRET_KEY/AUTH_ENCRYPTION_SECRET as generated 32+ char test secrets

What was checked and what came back

1. Core claim shape — Registered a real user, minted a token through POST /tokens. Decoded payload:

{"sub": "4d379145-8871-4d04-a4b2-0ae9c52c6516",
 "user": {"email": "e2e-pr5708-user@example.com", "is_admin": false},
 "teams": ["a32331cebb19407d9557f9f94dd3e0e3"]}

sub is the opaque EmailUser.id UUID, user.email carries the human identity. ✅

2. Round 1 (session/API-token scoping through require_auth) — GET /admin/well-known with the UUID-sub token → 200 (previously 401'd here pre-fix). ✅

3. Team-scoping, positive + negative control — Token sees a server owned by its own team; a second unrelated user/team's server is correctly excluded from the same token's GET /servers. Confirms real filtering, not open visibility. ✅

4. Round 2 (reverse-proxy ownership) — GET /v1/reverse-proxy/sessions executes cleanly on a UUID-sub payload (no crash in _get_user_from_credentials/get_jwt_user_email_from_payload). Couldn't get a live WebSocket session established in this from-scratch config (both session and API tokens 403'd identically at the ASGI layer with no app-level log — looks like an environment/config gate, not something this PR introduced, since it hits the pre-existing session-token path too). Falling back to tests/unit/mcpgateway/routers/test_reverse_proxy.py -q: 80/80 passed. ✅ (flagging this as unit-test corroboration rather than full live coverage, in the interest of transparency)

5. Backward compatibility — Hand-signed a legacy sub=<email> JWT with the same secret, hit a plain authenticated route → 200, log confirms ✓ Authenticated user: e2e-pr5708-user@example.com. Legacy tokens keep working as documented. ✅

6. Dataplane publisher UUID-keying — After ~15s (5 publish cycles), Redis holds only UUID-keyed UserConfig entries, no email-keyed ones. ✅

7. Cyclic import / secrets baseline (round 5) — pylint --enable=cyclic-import on auth.py/auth_context.py/api/v1/__init__.py scores clean, no R0401 — confirmed auth_context.py no longer imports from auth.py at all (only the reverse edge remains). .secrets.baseline diff is timestamp-only. ✅

8. Regression sweep — test_token_catalog_service.py, test_token_scoping.py, test_dataplane_publisher.py, test_streamablehttp_transport.py, test_reverse_proxy.py, test_csrf_middleware.py, test_csrf_fixes.py, plus test_auth.py and a broader -k "auth or csrf or admin" sweep across tests/unit/mcpgateway/ — all green, only expected environment-conditional skips (no Postgres test DB, plugins/UI disabled in this harness). ✅

Teardown confirmed clean — no containers, ports, or processes left behind from this run.

One non-blocking note

The current head has a merge conflict against main (.env.example, admin.py CSRF cookie naming, admin_ui/llmModels.js) — unrelated to this PR's own changes, main has just moved on. Worth a rebase before merge, but not something I'd hold the review on.

Approving — the opaque-subject migration, the fail-closed UUID resolution, and the backward-compat path all hold up under live verification, and nothing in the surrounding auth/CSRF/admin code broke.