Extend the interaction suite to the 2026-07-28 spec - #3018
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
No issues found across 29 files
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.
Tip: cubic could auto-approve low-risk PRs like this, if it thinks it's safe to merge. Learn more
Re-trigger cubic
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I didn't find any bugs in this change, but it's a large addition (~109 new test functions across 29 files) that pins spec interpretations and 65 documented divergences for the 2026-07-28 revision, so it deserves a human maintainer's review rather than an automated approval.
Extended reasoning...
Overview
This PR extends tests/interaction/ from the 2025-11-25 baseline to the 2026-07-28 spec revision. It touches 29 test files (no src/ changes), adding roughly 109 new test functions covering MRTR (multi-round-trip requests), the modern streamable-HTTP entry, x-mcp-header validation, SEP-2549 caching hints, the RFC 9207 iss validation table, OAuth refresh rotation / AS binding / step-up bounds, JSON Schema dialect handling, and assorted lifecycle/versioning behaviours. It also reworks the requirements manifest (399 → 605 entries) and registers 64 deferred entries plus 65 recorded divergences.
Security risks
None direct: the change is test-only and does not modify any production or security-sensitive code paths. The auth-related tests exercise the OAuth client/server harness but only add coverage; the harness changes (_harness.py, _provider.py) add test-only knobs (code/iss overrides, non-rotating refresh provider) that are scoped to the test suite.
Level of scrutiny
Although the production-risk level is low (tests only), the PR encodes a large number of spec interpretations and intentionally pins current SDK behaviour where it diverges from the 2026-07-28 spec (65 divergences). Whether those interpretations and pins are the right ones is a maintainer judgment call about the project's conformance posture, not something an automated review can sign off on. The sheer size also exceeds what is appropriate for shadow approval.
Other factors
The bug-hunting pass found no bugs. The author reports the suite green over ten consecutive runs with pyright/ruff clean and the manifest↔test coverage gate passing at every commit, and the commits are structured to be individually reviewable. There are no prior reviews or unresolved reviewer comments on the PR.
There was a problem hiding this comment.
I didn't find any bugs, but this is a very large change (~109 new test functions, manifest restructuring, 65 pinned divergences) that warrants a human look — note also that the diff touches src/mcp/server/streamable_http.py (coverage pragma removals), while the description says no src/ changes.
Extended reasoning...
Overview
This PR extends tests/interaction/ from the 2025-11-25 baseline to the 2026-07-28 spec revision: ~109 new test functions across 40 test files, manifest groundwork (requirement id renames, era added_in/removed_in linkage, 64 deferred entries), new test families (MRTR, modern streamable-HTTP entry, x-mcp-header, SEP-2549 caching, RFC 9207 iss validation, AS binding, JSON Schema dialects), and a per-cell coverage gate in test_coverage.py. The only src/ change is removing two coverage pragmas (# pragma: no branch / # pragma: no cover) in streamable_http.py, which is behaviour-neutral but contradicts the PR description's claim that only tests/interaction/ is touched.
Security risks
No production code paths change, so there is no direct security exposure. However, several new tests deliberately pin security-adjacent divergences as green (e.g. the registration application_type echo corruption, the missing capability embed gates, unvalidated Mcp-Param-* headers, the audience-mismatch acceptance) — pinning these is a documented suite policy, but a human should confirm each pinned divergence is intentional rather than accidentally normalising a behaviour the team would prefer to fix first.
Level of scrutiny
Test-only changes are normally low risk, but this PR encodes substantial policy decisions: which spec obligations are deferred, which SDK divergences are accepted and pinned, and how the coverage manifest is structured going forward. Those choices shape future conformance work and the v2 burn-down list, so they merit maintainer review even though the runtime SDK is untouched. The sheer volume (1009 collected cells, 605 manifest entries, a ~440k-character diff) also makes it impractical to certify correctness automatically.
Other factors
The bug-hunting system found no bugs, the author reports ten consecutive green runs with pyright/ruff clean, and the manifest↔test coverage contract is enforced at import. The 15 commits are described as individually reviewable, which should help a human reviewer work through the divergence and deferral decisions commit by commit.
b46806f to
011bbd5
Compare
There was a problem hiding this comment.
I didn't find any bugs in the new tests, but this is a large PR (~109 new test functions across 29 files) that pins many spec divergences as expected behaviour and reshapes the requirements manifest, so it warrants a human maintainer's review of those judgment calls.
Extended reasoning...
Overview
This PR touches only tests/interaction/ — no src/ changes — extending the interaction suite from the 2025-11-25 baseline to the 2026-07-28 spec revision. It adds roughly 109 new test functions across 29 files, plus harness/provider extensions (refresh-token rotation knob, iss/code overrides on the headless OAuth callback, persistent step-up shim) and substantial manifest re-keying (requirement id renames, era links, 64 deferred entries, 65 recorded divergences).
Security risks
There is no production code change, so no direct security exposure. The auth tests do exercise security-relevant client behaviour (RFC 9207 iss validation, refresh-token rotation, AS-binding credential discard, scope step-up), and several tests deliberately pin current SDK behaviour that diverges from the spec (e.g. pre-registered credentials silently replaced rather than erroring, Mcp-Param header values not validated, no embed capability gate). Pinning these as green tests is a policy decision: it makes today's gaps regression-stable, but a maintainer should confirm each pinned divergence is acceptable rather than something that should be fixed before being enshrined in the test suite.
Level of scrutiny
As a tests-only change the blast radius is limited to CI signal, which lowers the bar relative to runtime code. However, the PR is far from mechanical: it encodes interpretations of a draft 2026 spec, decides which behaviours are SDK-defined versus spec-mandated, and restructures the requirements manifest that gates coverage. Those are design/judgment decisions the approval guidelines say a human should weigh in on, and the sheer volume (1000+ added lines of intricate, heavily-documented test logic) exceeds what I can confidently rubber-stamp.
Other factors
The bug-hunting pass found no bugs, the description reports ten consecutive green runs with pyright/ruff clean, and the tests themselves are unusually well documented (each pins a named requirement with rationale). There are no prior reviewer comments to address. The main thing a human reviewer should focus on is the divergence list and the manifest id renames rather than line-by-line test logic.
There was a problem hiding this comment.
I didn't find any bugs in this change — it's test-only (no src/ changes), but at ~109 new test functions across 29 files plus the manifest/divergence bookkeeping it's too large and design-laden for an automated approval, so it deserves a human pass.
Extended reasoning...
Overview
This PR extends tests/interaction/ from the 2025-11-25 baseline to the 2026-07-28 spec revision: new MRTR (multi-round-trip request) coverage, the modern stateless streamable-HTTP entry, x-mcp-header validation, SEP-2549 caching hints, RFC 9207 iss validation, refresh-token rotation, AS-binding migration scenarios, and assorted smaller obligations. It touches 29 files, adds roughly 109 test functions, and registers 64 deferred manifest entries plus 65 recorded divergences. No production (src/) code is modified.
Security risks
None direct — the changes are confined to the test suite and its harnesses. Several tests pin current SDK behaviour that diverges from spec security requirements (e.g. Mcp-Param-* header values not validated against the body, the input_required capability embed gate not enforced, audience mismatch acceptance), which is intentional divergence-pinning rather than a regression introduced here, but a maintainer should confirm those pins match the project's divergence-lifecycle policy and that the referenced burn-down items exist.
Level of scrutiny
Moderate. As test-only code it cannot break production paths, and the bug-hunting pass found no issues. However, the PR encodes many normative judgments — which behaviours are spec-mandated vs SDK-defined, which divergences are acceptable to pin green, and how the manifest's era/supersedes metadata is structured — and those judgments shape future conformance work and issue filing. That is exactly the kind of design-level content a maintainer should sanity-check rather than a bot.
Other factors
The diff is internally consistent with the existing suite's conventions (requirement decorators, snapshot pinning, recording-first assertions), the author reports green runs, pyright/ruff clean, and a coverage-contract check at every commit. There are no prior reviews or unresolved comments on the thread. The size alone (well beyond a simple, mechanical change) is the main reason not to shadow-approve.
b4f5bfb to
1f3b5f0
Compare
There was a problem hiding this comment.
I didn't find any bugs in this change, but it's a large test-suite extension (~109 new test functions across 32 files) that pins 65 recorded divergences from the 2026-07-28 spec — including auth-flow behaviour — so the divergence-pinning decisions deserve a human maintainer's judgment.
Extended reasoning...
Overview
This PR touches only tests/interaction/ — no src/ changes. It extends the interaction suite from the 2025-11-25 baseline to the 2026-07-28 spec revision: MRTR round-trips, the modern streamable-HTTP entry, x-mcp-header handling, SEP-2549 caching hints, server/discover, RFC 9207 iss validation, refresh-token rotation, AS-binding, DCR defaults, and JSON Schema dialect handling. It also updates the requirements manifest and test harnesses (_harness.py, _provider.py, _connect.py).
Security risks
No production code is modified, so there is no direct security exposure. However, several new tests deliberately pin divergent current behaviour green in security-adjacent areas (e.g. audience-mismatch acceptance, pre-registered credentials with a mismatched issuer being silently replaced, the missing capability embed gates, Mcp-Param-* values not validated against the body). Pinning these as passing tests is intentional per the suite's documented divergence lifecycle, but it encodes a policy decision about which spec violations are tolerated for now — a maintainer should confirm those pins match the v2 burn-down plan.
Level of scrutiny
Test-only changes normally warrant lighter scrutiny, but the sheer size (~3,500+ changed lines, ~109 new test functions, 32 files) and the number of behaviour-pinning decisions (65 divergences, era gating of transports, manifest restructuring with supersedes/superseded_by links) make this more than a mechanical addition. The tests themselves look carefully constructed — deterministic, event-driven waits under anyio.fail_after(5), wire-level assertions where the typed API can't observe behaviour — and I found no correctness bugs in the test logic.
Other factors
The bug-hunting pass surfaced no issues, the PR description reports ten consecutive green runs plus clean pyright/ruff, and each divergence pin carries a re-pin instruction in its docstring. The remaining open questions (whether the recorded divergences and deferred entries are the right call for the v2 line) are project-direction decisions rather than code-correctness ones, which is why I'm deferring rather than approving.
1f3b5f0 to
e75603f
Compare
e75603f to
210f9c5
Compare
| "Not implemented in the SDK: the client OAuth provider keeps no cross-request memory " | ||
| "of scope-upgrade attempts. The 403 insufficient_scope branch " | ||
| "(src/mcp/client/auth/oauth2.py:704-734) performs one inline step-up per send with no " | ||
| "attempt counter and no (resource, operation) key, and OAuthContext (oauth2.py:98) " | ||
| "carries no field recording prior step-up failures, so a second send for the same " | ||
| "resource and operation re-attempts the upgrade unconditionally. The per-send " | ||
| '"repeated 403s do not loop" half of this spec line is client-auth:403-scope-upgrade.' | ||
| ), | ||
| ), |
There was a problem hiding this comment.
🟡 The new client-auth:stepup:attempt-tracking entry's deferred text ends with 'The per-send "repeated 403s do not loop" half of this spec line is client-auth:403-scope-upgrade' — but this same PR deleted that clause from client-auth:403-scope-upgrade and re-homed it into the new client-auth:stepup:retry-cap entry, which the entry's own note two lines above correctly names. Fix by pointing the sentence at client-auth:stepup:retry-cap (or dropping it, since the note already says it); the hardcoded oauth2.py line numbers (704-734, 98) in the same deferral are also worth dropping, matching the line-number removal this PR does elsewhere.
Extended reasoning...
What the bug is. This PR reworks the step-up family of manifest entries in tests/interaction/_requirements.py. Before the PR, client-auth:403-scope-upgrade's behavior string read "A 403 with WWW-Authenticate triggers a scope-upgrade authorization attempt; repeated 403s do not loop." The PR narrows it to just "A 403 with WWW-Authenticate triggers a scope-upgrade authorization attempt." and re-homes the no-loop obligation into the new client-auth:stepup:retry-cap entry ("Step-up re-authorization is bounded per request send: one re-authorization and one retry, after which a further insufficient_scope 403 on the retried request surfaces to the caller as an error without another authorization attempt."). But the also-new client-auth:stepup:attempt-tracking entry's deferred text still ends with: 'The per-send "repeated 403s do not loop" half of this spec line is client-auth:403-scope-upgrade.' — a pointer this same PR made stale.\n\nWhy it is internally inconsistent. The very same entry's note field, two lines above the deferral, says the correct thing: "The per-send bound is client-auth:stepup:retry-cap." And the reworked docstring in tests/interaction/auth/test_lifecycle.py for the 403 step-up test likewise says "the per-send bound is pinned by client-auth:stepup:retry-cap." So the deferral's final sentence contradicts both the entry's own note and the test suite's updated prose — three statements about the same ownership, one of them wrong.\n\nStep-by-step proof. (1) Grep the PR head for client-auth:403-scope-upgrade: its behavior at tests/interaction/_requirements.py:6689 is now the single-clause string with no "repeated 403s do not loop" text. (2) Read client-auth:stepup:retry-cap (~line 6702): its behavior is exactly the per-send no-loop bound, and its note explicitly delegates cross-request tracking to client-auth:stepup:attempt-tracking. (3) Read client-auth:stepup:attempt-tracking (lines 6737-6752): the note says "The per-send bound is client-auth:stepup:retry-cap", but the deferred text's last sentence (line 6750-6751) names client-auth:403-scope-upgrade for the same clause. So a future maintainer who closes the attempt-tracking gap and follows the deferral's pointer lands on an entry that no longer carries the behaviour they're looking for.\n\nWhy it matters despite being prose. The PR's own design treats these cross-references as curated queryable data: the coverage gate enforces supersedes links at import, and the deferral texts are explicitly the navigation map used to find the owning entry when a gap closes. A wrong pointer in that map defeats its purpose. That said, nothing executes this text — no test, gate, or runtime behaviour depends on it, the correct pointer already exists in the same entry's note, and merging as-is breaks nothing — hence nit severity.\n\nHow to fix. Change the deferral's final sentence to name client-auth:stepup:retry-cap, or delete the sentence entirely since the entry's note already carries the correct pointer. A minor supporting cleanup in the same lines: the deferral hardcodes source line numbers (src/mcp/client/auth/oauth2.py:704-734 and oauth2.py:98), a style this same PR removes elsewhere (the hosting:auth:as:token-redirect-mismatch divergence changed token.py:157 to token.py); dropping them here keeps the references from rotting on the first oauth2.py edit.
Eight entries cited pages or anchors that do not exist in the 2026-07-28 specification tree (the basic/lifecycle page was split into basic/versioning and server/discover; two anchors were renamed). Repoint each to the verified live section. Also add the missing added_in="2026-07-28" on client-auth:authorization-response:iss-verify: RFC 9207 iss validation is SEP-2468, new at 2026-07-28, and the prior 2025 source page carries no such requirement. Cell generation is unchanged (830 before and after).
Rename 14 requirement ids to the names shared with the typescript-sdk e2e suite (sampling:create:*, elicitation:url:action:*, protocol:meta:*, the client-auth stepup/iss/scope/dcr family, the mcpserver context helpers), updating every @requirement decorator to match. The resources:read:unknown-uri name previously sat on an entry whose test proves lowlevel handler-error passthrough, not the unknown-resource rule; that entry is re-described honestly as protocol:error:handler-error- passthrough (source 'sdk'), and the real unknown-resource entry (-32602 with the URI in error.data, SEP-2164) takes the vacated name with its source repointed to the 2026 page that mandates it. The protocol:meta:request-to-handler 2026 arm exclusion now carries the accurate reason (legacy-only-vocabulary) and a note explaining the envelope- key merge that breaks the equality assertion, so the re-admission checklist finds it. Three test docstrings quoting pre-rename spec anchors updated. Cell generation unchanged (830 before and after).
client-transport:http:protocol-version-stored and transport:streamable-http:origin-validation were second labels on assertions their tests already pin under client-transport:http:protocol-version-header and hosting:http:dns-rebinding respectively (decorators removed, tests kept). lifecycle:stateless:no-initialize described a pin API that no longer exists, deferred against coverage that does not exist, and bound no tests. transport:streamable-http:server-to-client is NOT deleted: the underlying behaviour is real 2025-era wire truth that 2026 forbids (SEP-2322), so it gets removed_in="2026-07-28" with the supersession note; the MRTR successor link lands with the era pass. Cells unchanged (830 before and after).
Sixteen fixes on sixteen entries, each evidenced against the bound test body or the spec text: narrow over-claiming strings to the assertions that exist (403-scope-upgrade's unproven no-loop clause, iss mismatch-only coverage, tools-only registration, arrival-only HTTP notification delivery, the custom-client auth clause), correct two entries that attributed the modern classifier's version rejection to the legacy transport, record the 2026 per-request logLevel gate divergence on the Context logging helper and the list-vs-object structured-content gap on text-mirror, re-ground the null-id deferral on the now-existing fault channel, and fix the discover result field name (supportedVersions) plus the 404 status the spec mandates for initialize at the modern entry. Cells unchanged (830 before and after).
… data Execute the era/supersession pass: 37 new successor entries (the MRTR family that replaces server-initiated requests, the per-request log-level pair, the subscriptions/listen family, discover-side successors), all registered deferred ahead of their tests; 82 existing entries edited - version-wide 2026 arm exclusions on era-retired behaviours become removed_in with a superseded_by link and an explanatory note (transport-shaped exclusions stay), no-heir removals get tombstone notes, and the per-request logLevel divergence is recorded on the three logging entries whose tests pin un-gated delivery on live 2026 cells. 62 supersession pairs, all bidirectional and versioned, enforced by the coverage gate at import. The twelve surviving version-wide exclusions are exactly the documented re-admission checklist. Cells unchanged (830 before and after).
The multi-round-trip request pattern (SEP-2322, the 2026-07-28 replacement for server-initiated requests) gets its first end-to-end coverage: - lowlevel/test_mrtr.py (new): the write-once roundtrip (byte-exact requestState echo, opacity via a non-parseable state, fresh JSON-RPC id on retry), state-only retry, omit-when-absent, and parallel-call isolation via a symmetric rendezvous that provably holds both loops mid-flight. - test_elicitation.py: form-mode elicitation over MRTR (basic, decline, cancel, schema primitives) and the capability gate, which pins the current un-gated embed behaviour with a recorded divergence. - test_resources.py / test_prompts.py: the resources/read and prompts/get MRTR origins (lowlevel-only; MCPServer admits InputRequiredResult on the tools path only). - test_sampling.py / test_roots.py: sampling/createMessage and roots/list embedded as MRTR input requests, with model preferences, system prompt, and context-inclusion pass-through. Seventeen requirement entries flip from deferred to tested; five entries are minted (the request-state client obligations and the two non-tools origins). 830 -> 859 collected cells, every new cell accounted; suite green three consecutive runs; 100% line and branch on the new file with no coverage pragmas.
…rectionality pins Ten more tests: the multi-round completion loop, the rounds cap, the at-least-one-of construction-site rejection, the inputResponses structural validation and key correspondence, the -32042 emission-ban wire scan, and the 2026 directionality edges - the push-API loud-fail split (the standalone leg pins NoBackChannelError green on both 2026 cells; a dedicated in-memory test pins the request-scoped leg still transmitting the forbidden frame, recorded as a per-transport, per-leg divergence so the eventual era-gate fix re-pins mechanically), a wire-trace proof that a 2026 exchange contains no server-initiated requests and no client-sent responses, and the sampling and roots embed capability gates (both pinned un-gated with recorded divergences, completing the embed-gate family). Five entries flip from deferred, five origin-new entries are minted with their tests. 859 -> 876 collected cells, every node accounted; suite green three consecutive runs.
The SEP-2243 header derivation pipeline gets full coverage: static definition validation with per-tool eviction and the logged warning (RFC 9110 token rule, control characters, case-insensitive duplicates, the number type the spec now forbids, items/nested-properties reachability), the base64 sentinel encoding both ways including the collision-escape row from the spec's own table, null/absent argument omission, and the Mcp-Method/Mcp-Name mismatch rejections (400, -32020). The known server-side gap - Mcp-Param-* values are not validated against the body - is pinned as a divergence carrying issue=L110 with the recognized-header judgement call recorded so the fix re-pins under either shape. The modern entry itself: response modes, lazy SSE upgrade, cacheable stamping, disconnect cancellation, header validation arms, and the initialize-removed rejections. 28 entries minted (23 tested, 5 deferred), one flip, and the ledger riders: issue=L109 on the three embed-gate divergences, issue=L107 on the push-API pin. 876 -> 909 collected cells, all accounted; suite green three runs.
…n three tests Add the HTTP request-scoped loud-fail test, completing all four legs of the push-API divergence record (both transports x both legs). Add the missing templates/list decorator on the static-and-templated listing test. Redesign the post-connect registration fixture to mutate the tool set between requests rather than from inside a handler, so the fixture itself no longer violates the 2026 list-stability requirement on live cells. Assert that an iss-mismatch rejection never exchanges the authorization code (with a liveness guard on the recorded /token calls). 909 -> 910 cells.
SEP-2549 server-side caching: cache hints pass through unmodified on prompts, resource-template, and discover results (the discover hints were previously pinned nowhere); ttlMs zero means immediately stale; absent hints default per the 2025 rules on the legacy cells; the interim input_required frame carries no hints while the same exchange's complete result does. Two recorded divergences: cross-page cacheScope consistency is delegated to handler authors (the spec MUST binds the server), and a negative ttlMs raises a validation error where the spec says clients should ignore-as-zero - the divergence note records that emission-side strictness is correct and only the inbound parse should clamp. Discover and versioning: instructions and derived capabilities ride DiscoverResult (with vacuity guards against silent legacy fallback), auto mode probes before negotiating, era-cached results are reused identically, the -32022 supported-list retry, dual-era precedence, and the era method gate - a 2025 method on a 2026 connection is method-not-found before any handler lookup, proven by a registered handler that never runs. 27 entries minted (13 tested, 14 deferred with greppable re-open tokens), 3 flips. 910 -> 931 cells, all accounted; suite green three runs.
…and refresh The full iss validation table from the 2026 authorization-response rules: match accepted, trailing-slash difference rejected without normalization (both comparison strings pinned as harness literals so server-side issuer serialization changes cannot invert the test), missing-iss rejected when advertised and tolerated when not, an unadvertised-but-present iss still validated, and an error redirect with a mismatched iss rejected on iss before the missing-code error - the ordering that proves validation applies equally to error responses. Step-up bounds: a second insufficient-scope 403 after one step-up surfaces as an error without another authorize round trip, and a 403 on the GET stream open steps up and reopens with the upgraded token (era-bound: the GET stream is removed at 2026-07-28). DCR defaults (grant_types omitted and passed through verbatim), refresh-token rotation handling at the single-refresh seam (replacement stored, preservation honoured when the server does not rotate), and a non-2xx token response surfacing typed. The as-binding entry splits into its two spec obligations (re-register and no-credential-reuse), both decorating the existing test unchanged. Harness: three small review-approved knobs (iss visibility, code override, persistent step-up shim, non-rotating provider). 16 entries minted (13 tested, 3 deferred), 931 -> 944 cells exact; suite green three runs.
…_type pass-through Pre-registered credentials bound to a different issuer are silently discarded and re-registered - the path the spec blesses only for DCR-persisted credentials; for manually provisioned ones it says an error should surface. The new test pins the silent replacement (flow completes, no error, the seeded credential never presented, storage rebound to the current AS) under a recorded divergence scoped to the issuer-stamped arm; the unbound arm is a documented limitation in the entry note, since a mismatch cannot be detected for credentials that never recorded a binding. A consumer-set application_type of web on a loopback redirect - a value the derivation heuristic would never produce - reaches the /register body verbatim, distinguishing pass-through from any future heuristic. Also: the last caching deferral gains its greppable re-open token, the omit_iss precedence is documented in the harness, and the app-type heuristic note cross-references its tested override sibling. 944 -> 946 cells; suite green three runs.
… capability Every behaviour the analysis identified that the SDK cannot yet express now has a manifest entry with a deferral stating exactly what is missing at this commit: the subscriptions/listen runtime family (types vendored, runtime absent, all carrying the greppable re-open token), the requestState integrity obligations (application-owned, the SDK passes opaque state through), the extension declaration surface, the legacy 2025 jsonschema wrap family (era-bound to the cells where it applies), the hosting-side auth surfaces, stdio-2026 service, and the cross-AS credential obligations (m2m credentials re-spelled into the as-binding family, targeting the spec obligation rather than another SDK's knob). Deferral reasons are re-grounded at this commit - no stale premises, no PR numbers, no internal references; two stale source attributions upgraded to the spec URLs that carry the requirement verbatim. Cells unchanged (946): deferred entries register coverage debt without running anything.
JSON Schema handling: prefixItems vocabulary enforcement, the 2020-12 default dialect (with a declared-dialect violation arm proving validation follows the tag), falsy structured content reaching the validator, and non-object outputs - plus the null structured-content divergence: a tool legitimately returning JSON null is indistinguishable from one returning nothing (the model collapses both to None and the dump drops them), so a spec-legal value raises; pinned with the fix direction recorded (absent-vs-null at the model layer, not a looser client check). MRTR edges: a retry missing a requested key is re-prompted rather than errored, unknown response keys are ignored, the resultType seam (absent means complete, input_required is never masked, unrecognized values rejected - flipping the deferred entry to a pinned divergence), and the max-tokens pass-through. Auth: refresh tokens are not reused across an AS change, CIMD documents are portable, the all-scopes single challenge, and the bundled AS registration echo dropping application_type (pinned against its ledger row). The scatter: list results are connection-independent and deterministically ordered, an empty-string cursor is a valid cursor (a 2026 rule the changelog never mentioned), cancellation stops notification delivery, SSE comment lines are ignored, legacy error codes pass through opaquely, sampling messages are not retained across rounds, multi-content reads, path-traversal rejection, and resource links in prompt content. 26 entries minted, one flipped, four divergences recorded. 946 -> 1009 cells, every node accounted; suite green three consecutive runs.
The unrecognized-resultType and scope-aggregation divergences gained tracking entries after their pins landed; wire the issue fields so the fixer trail is complete for every pinned divergence in the manifest.
MCPServer now passes InputRequiredResult through its prompt and resource pipelines, so the two origin entries' notes and the matching test docstrings no longer claim it cannot; the mcpserver mirrors are recorded as possible and not yet covered. No behaviour or assertion changes - the full suite is green unchanged against current main.
Cut the comment and docstring volume of the new tests by half: docstring first lines are single short sentences, narration and restating-the-code comments are gone, and surviving comments carry only what the code cannot say - provenance labels, the divergence re-pin instructions, and the non-obvious traps (coverage branch quirks, opaque-state shape). No code or assertion changes: every file parses identically with docstrings stripped.
…haviour The client now clamps a negative inbound ttlMs to 0 before validation, and the modern HTTP entry validates Mcp-Param-* headers against the tool's advertised schema. Both tests flip from pinning the old gap to asserting the spec-mandated outcome, and their divergence records are removed.
The merge window landed the listen server runtime, the client response cache, requestState integrity, stream-pair 2026 serving, and the client extension API — invalidating the premises of ~45 deferral texts and notes written against the older base. Each text now states what is true at this pin: entries whose blocking surface landed flip to 'Not yet covered here' naming what a test will drive; entries still blocked name the precise missing surface. Also fixes the README era-lock line (stateless is locked too), documents the default-on client cache in the transport-matrix section, and renames one resource test whose name predated its retarget.
…lify four tests The server extension API can now author an arbitrary resultType through public surface, so the scripted memory-stream peer is replaced with an MCPServer extension driven through the connect fixture (one node becomes the entry's two cells). The parallel-MRTR isolation test drops its wire recording for handler capture (the rendezvous still forces interleaving), the GET-stream step-up shim moves into the auth harness beside its POST twin, a redundant metadata shim argument goes away, and the claimed-shape extension test pins its ValidationError by structured fields instead of accepting any validation failure.
The push-API prohibition is now enforced locally on every modern dispatch path: the request-scoped in-memory leg no longer transmits the forbidden frame, so its test flips from pinning the transmitted-and-refused shape to asserting the typed local NoBackChannelError, and the divergence record is removed — every constructible 2026 arm now refuses locally. The stdio serving entry's era-lock description is updated to the landed semantics: the era settles on the first era-distinctive frame to reach a client-visible success, no failure locks, and a cancelled-away success does not lock.
…e the era-gate test through the public request seam Port the branch-authored httpx/httpx_sse lines main's rename never saw to httpx2, including the four aconnect_sse readers in the modern hosting tests, and delete the dead httpx_sse import. Drive the 2026 resources/subscribe era-gate test through ClientSession.send_request instead of the deprecated typed subscribe wrapper.
Main now stamps io.modelcontextprotocol/serverInfo into the _meta of every 2026-era result and ships an era-aware strip helper. Route the branch's own result comparisons through it: era-parametrized tests thread the unstamped fixture; tests pinned to a single modern connection call the strict module helper directly, aliased strip_stamp wherever a file also uses the fixture so the name never carries two meanings; raw-wire snapshots pin the stamp on the wire, with ad-hoc test servers given an explicit version so no snapshot pins an empty one. Drop the DiscoverResult(server_info=...) kwargs the reshaped model silently ignored, and correct the tests/_stamp.py docstring to match the new direct usage. No-Verification-Needed: test-only change
…t main A per-item sweep of all 166 deferral premises and 54 untracked divergences found 42 no longer true at this pin (21 false, 21 narrowed), several of them false since before the branch was cut. Rewrite each to what actually holds: the subscriptions/listen client driver, client-side cancellation over the 2026 transports, stdio serving 2026 with the era chosen from the opening request, the widened resolver capability gate, and the response-cache list_changed eviction all landed, so their "missing" premises become flip-candidates with true reasons; the stdio dual-era text is rewritten a fourth time against the settled loop. Reconcile the entries main now covers itself: delete the two split cancel deferrals superseded by main's bound stream-frame and cancel-closes-stream pins, drop client:listen:signal-only in favour of main's tested iteration entry, drop the sdk-sourced list-changed capability-gated position the client does not hold, and let the branch's spec-anchored graceful-close entry win its true duplicate on main by re-binding main's existing test to it. Drop the two divergences whose gap does not exist (Annotations last_modified round-trips; the read timeout is an enforced maximum) and the four false arm-exclusion notes on the cancel family. Fix the six test docstrings that repeated the retired claims. 617 entries, 60 divergences, 160 deferrals. No-Verification-Needed: test-only change
Reword the unsupported-dialect deferral to describe the validator-class selection the client now performs, and name the public ClientSession.validate_tool_result in the null-structured-content note. No-Verification-Needed: prose-only edit to the test manifest
…te with main Reconcile the branch with main's landed behaviour on top of the rebase: - Adopt main's hosting:auth:as:register-echo entry over the branch's register-echo-application-type entry and its now-closed divergence, drop the stale pinning test that superseded it, and remove its unused imports. - Rewrite the no-further-notifications cancellation test onto the doomed-call pattern: a cancelled request is no longer answered, so the call is parked and abandoned instead of expecting an error, tolerating only the legacy streamable HTTP terminator. - Reword the no-further-notifications note to match the narrowed in-flight divergence. - Assert the persisted application_type echo in the DCR pass-through test now that the registration echo carries it. No-Verification-Needed: test-only diff, no runtime surface to drive
210f9c5 to
72edf90
Compare
There was a problem hiding this comment.
Beyond the inline findings (both prose-only manifest notes), one other candidate was examined and ruled out this run: the logging:per-request-level:invalid-level deferral's claim that the SDK rejects an unrecognized io.modelcontextprotocol/logLevel value was suspected of being false, but it holds — the 2026-07-28 request _meta model types that key as the LoggingLevel enum (src/mcp-types/mcp_types/_v2026_07_28/__init__.py), so surface validation does reject out-of-range values before any handler runs.
Extended reasoning...
This run's two findings are both nit-severity wording issues in manifest note/deferral prose in tests/interaction/_requirements.py; nothing executes that text and neither blocks merge. The one additional candidate raised — that the logging:per-request-level:invalid-level deferral overstates SDK validation — was verified against the code: the 2026-07-28 request meta model declares io.modelcontextprotocol/logLevel as LoggingLevel | None, so an unrecognized value fails typed parsing as the deferral describes. Recording it here so a later pass does not re-explore it. Given the PR's size (33 files, ~1000 test cells) this stays with human review; no approval decision is implied.
| note=( | ||
| "Only observable over streamable HTTP: server/discover is served only by the modern " | ||
| "HTTP entry (the in-memory 2026 connection synthesizes its DiscoverResult client-side " | ||
| "and never sends the request). The pinned 0/private values are the SDK's " | ||
| "CacheableResult defaults -- no handler authors discover hints -- so the test pins " | ||
| "the stamping mechanism, not authored pass-through. Completes the six-operation map " | ||
| "with caching:hints:prompts-list (family index) and hosting:http:modern:cacheable-stamping." | ||
| ), |
There was a problem hiding this comment.
🟡 The note on caching:hints:server-discover claims "server/discover is served only by the modern HTTP entry (the in-memory 2026 connection synthesizes its DiscoverResult client-side and never sends the request)" — but server/discover is a core registry handler served on every 2026 dispatch path, and this PR's own in-memory auto-mode tests (e.g. test_discover_carries_server_instructions_and_omits_them_when_undeclared) prove the in-process path serves the probe; the parenthetical is true only for the version-PINNED client the matrix's in-memory 2026 cell uses. The transports=("streamable-http",) scoping itself is sound (hint presence is only wire-observable; typed models default-fill ttl_ms/cache_scope) — just reword the first clause to state the cell-scoping fact instead of a false SDK-wide serving claim.
Extended reasoning...
What the bug is. The new manifest entry caching:hints:server-discover (tests/interaction/_requirements.py:4333-4349) justifies its transports=("streamable-http",) scoping with: "Only observable over streamable HTTP: server/discover is served only by the modern HTTP entry (the in-memory 2026 connection synthesizes its DiscoverResult client-side and never sends the request)." The first clause is factually false at this PR's head: server/discover is not a modern-HTTP-entry feature. It is registered in the low-level server's core spec registry — src/mcp/server/lowlevel/server.py:448 registers ("server/discover", types.RequestParams, self._handle_discover) in _spec_requests — so every 2026 dispatch path serves it: the modern HTTP entry, the in-process DirectDispatcher pair, and the stdio stream-pair loop.
The code path that contradicts the note. Client.__aenter__ with the default mode='auto' calls negotiate_auto(session) (src/mcp/client/client.py:449), which sends a real server/discover request; _connect_inproc routes any non-legacy in-memory connection through a DirectDispatcher pair served by modern_on_request(server, ...), so the in-memory probe genuinely reaches the server's _handle_discover. Only a version-pinned client (mode=<version>) takes the session.adopt(self.prior_discover or _synthesize_discover(self.mode)) branch at client.py:451 — that is the client-side synthesis the parenthetical describes, and it is exactly (and only) what the matrix's in-memory 2026 cell uses.
Step-by-step proof from this PR's own tests. (1) tests/interaction/lowlevel/test_client_connect.py::test_discover_carries_server_instructions_and_omits_them_when_undeclared opens a bare Client(Server("guided", instructions="Call the add tool.")) — in-memory, default mode='auto' — and asserts client.instructions == "Call the add tool.". Instructions are server-authored data that only travels in the server/discover result; a client-side synthesized DiscoverResult could not carry them. (2) test_discover_capabilities_reflect_registered_handlers likewise receives handler-derived ServerCapabilities from a bare in-memory Client(Server(...)) — again data only the server's _handle_discover can produce. (3) The same manifest's transport:stdio:dual-era-serving deferral says a mode='auto' client "negotiates 2026-07-28 via server/discover ... over a real child-process pipe", i.e. the stream-pair loop (src/mcp/server/runner.py) serves the probe on stdio too. So the note's serving claim is contradicted twice within the PR itself.
Why the scoping decision is still correct. The entry's actual load-bearing scoping fact is fine: hint presence (ttlMs/cacheScope keys on the frame) is only observable on the raw wire, because typed client models default-fill ttl_ms/cache_scope, and the matrix's in-memory 2026 cell uses a pinned client that never sends the probe at all. So restricting the entry to streamable-http is the right call — only the recorded justification misstates why.
Impact. The PR's design treats these notes as curated queryable data (the description calls the manifest "queryable data"; the coverage gate enforces its links). A maintainer consulting this note would wrongly conclude no non-HTTP path serves server/discover and could, e.g., skip in-memory/stdio auto-mode coverage for discover behaviours or mis-triage a future in-memory discover bug. Same class as the internally-inconsistent client-auth:stepup:attempt-tracking pointer already flagged on this PR.
How to fix. Reword the first clause to the accurate cell-scoping fact, e.g.: "Only observable over streamable HTTP: the hints are only visible on the raw wire (typed models default-fill ttl_ms/cache_scope), and the matrix's in-memory 2026 cell uses a pinned client that synthesizes its DiscoverResult locally and never sends the request (an auto-mode client would be served by the in-process dispatch path)." Prose-only; nothing executes this text, so it is not merge-blocking.
| note=( | ||
| "removed in 2026-07-28 (SEP-2322/SEP-2575); with server-initiated requests retired there is " | ||
| "nothing in flight on the client for a server to cancel, and servers MUST NOT send " | ||
| "notifications/cancelled except to tear down a subscriptions/listen stream (pinned separately as " | ||
| "protocol:cancel:server-listen-only when the cancellation slice lands it). No replacement." | ||
| ), |
There was a problem hiding this comment.
🟡 The note on the retired protocol:cancel:server-to-client entry says the MUST NOT is "pinned separately as protocol:cancel:server-listen-only when the cancellation slice lands it" — but this PR's own cancellation slice already landed that entry (lines 924-941), and it landed as a deferral whose own text says the prohibition is "vacuously satisfied with nothing to observe", so it is not (and cannot currently be) pinned. Reword the parenthetical to something like "(tracked as protocol:cancel:server-listen-only, deferred: the prohibition is vacuous at this pin — no server-side notifications/cancelled emitter is reachable at 2026-07-28)".
Extended reasoning...
What the issue is. The retired entry protocol:cancel:server-to-client (tests/interaction/_requirements.py:950-964, note text added by this PR) ends its note with: "servers MUST NOT send notifications/cancelled except to tear down a subscriptions/listen stream (pinned separately as protocol:cancel:server-listen-only when the cancellation slice lands it). No replacement." Both halves of that parenthetical are made false by this same PR.
The stale conditional. "when the cancellation slice lands it" is a forward reference to work this PR itself contains: the cancellation slice did land protocol:cancel:server-listen-only at tests/interaction/_requirements.py:924-941. The conditional phrasing was resolved by the very PR that carries it, so the note reads as pointing at future work that is in fact a few hundred lines up in the same file.
The wrong status. More substantively, the entry is not "pinned separately" — it landed as a deferral, and its own deferred text explains why it cannot be pinned at this pin: "the one permitted emission never occurs -- listen-stream teardown exists (src/mcp/server/subscriptions.py) but deliberately answers with the stamped empty result rather than notifications/cancelled -- and the only other emitter, the courtesy cancel on abandoning a server-initiated request (src/mcp/shared/jsonrpc_dispatcher.py), is unreachable at 2026-07-28 where no server-initiated JSON-RPC requests exist, leaving the prohibition vacuously satisfied with nothing to observe" (lines 932-939). The sibling entry protocol:cancel:listen-teardown-cancelled (lines 872-897) additionally records that pinning the teardown arm would take a side in a spec self-contradiction between the cancellation and subscriptions pages — so the pin is deliberately withheld, not merely pending.
Why the pointer graph matters here. The PR's own design treats these cross-references as curated queryable data: the era pass and its supersedes/superseded_by links are the PR's centerpiece, enforced by the coverage gate at import, and the deferral prose is the navigation map a maintainer follows when a gap closes. The manifest elsewhere labels deferral status correctly in exactly this situation — e.g. lifecycle:discover:era-cached's note says "the re-probe-on-stale follow-on is lifecycle:mode:prior-discover-stale-reprobe (deferred)". This note is the one place a pointer asserts a pin that does not — and per the target's own deferral, currently cannot — exist.
Step-by-step proof. (1) Read the note at tests/interaction/_requirements.py:957-962: it says the MUST NOT is "pinned separately as protocol:cancel:server-listen-only when the cancellation slice lands it". (2) Search the same file for protocol:cancel:server-listen-only: it exists at lines 924-941, added by this PR's cancellation slice — the conditional is resolved. (3) Read that entry: it carries a deferred= field and no bound test; the deferral text says the prohibition is "vacuously satisfied with nothing to observe". (4) A maintainer following the pointer expecting a bound test pinning the MUST NOT instead finds an unpinnable deferral — the pointer's claim ("pinned") is wrong on arrival.
Why not a duplicate. The existing timeline comment about client-auth:stepup:attempt-tracking covers a different stale pointer with a different root cause: that one names an entry this PR stripped of the referenced clause; this one asserts a nonexistent pin and carries a temporal conditional resolved by this same PR.
How to fix. Reword the parenthetical, e.g.: "(tracked as protocol:cancel:server-listen-only, deferred: the prohibition is vacuous at this pin — no server-side notifications/cancelled emitter is reachable at 2026-07-28)". Prose-only: nothing executes this text, no test or gate depends on it, and merging as-is breaks nothing — hence nit severity.
1 participant
Extends
tests/interaction/from its 2025-11-25 baseline to full coverage of the 2026-07-28 revision. Only tests are touched — nosrc/changes (one smalltests/_stamp.pydocstring exception, noted below).Part of #2891.
What's here
The 26 commits are individually reviewable; the first fifteen are the original coherent batches that each landed green, and the trailing commits carry the rebase reconciliation described below.
Manifest groundwork (first five): dead 2026 source URLs repointed at live sections, ids aligned with the typescript-sdk e2e suite vocabulary, three redundant entries retired, over-claiming behaviour strings narrowed to what their tests actually prove, and the era pass — retired behaviours get
removed_in, their replacements getadded_in, and the pairs are linkedsupersedes/superseded_by(62 bidirectional pairs, enforced by the coverage gate at import). The 2025→2026 transition is queryable data, and no test body branches on a version literal.The 2026 families (next nine): MRTR end to end — the write-once roundtrip,
requestStateecho/omission/opacity, parallel-call isolation via a symmetric rendezvous, multi-round completion and bounds, and all three origin methods — plus the 2026 message-direction rules (a wire trace contains no server-initiated requests and no client-sent responses), the modern streamable-HTTP entry (response modes, lazy SSE upgrade, header validation, cacheable stamping), thex-mcp-headerpipeline including both directions of the base64 sentinel encoding, SEP-2549 caching,server/discoverand versioning (including the era method gate: a 2025 method on a 2026 connection is method-not-found before any handler lookup), the auth additions (the RFC 9207issvalidation table, step-up bounds, DCR defaults, refresh rotation, AS binding and its pre-registered-credentials arm), JSON Schema handling (dialects, prefixItems, falsy/non-object/null structured content), and a tail of smaller obligations down to the empty-string-cursor rule.Tracking for what the SDK can't express yet (one commit): the deferred entries registered with reasons grounded at the pin — the manifest records the full 2026 coverage surface, not just what runs today, and every deferral names what unblocks it.
What the rebase changed
Main moved substantially while this was open, and it moved into the suite, so the branch was rebased with a re-evaluation each time rather than a mechanical replay. The trailing commits carry that work:
_meta, so the branch's result assertions route through main's era-aware strip fixture (and pin the stamp on the wire where the assertion is about the raw frame).The one non-
tests/interaction/change: two docstring paragraphs intests/_stamp.py, corrected because this suite's direct use of that helper made them false.Numbers
830 → 1054 collected cells, 399 → 619 manifest entries, 59 recorded divergences, 160 deferrals. The suite is green over three consecutive runs at the tip, pyright and ruff are clean, and the manifest↔test coverage contract passes.
Divergences
Where current SDK behaviour differs from the spec, the suite follows its documented divergence lifecycle: the test pins today's behaviour green and the entry records the divergence, with the re-pin instruction in the test docstring so the eventual fix is mechanical. 59 divergences are recorded. The ones with a verified root cause carry an
issue=tag referencing the v2 burn-down list; these will swap to GitHub issue links as those are filed. The ones most worth knowing about:input_requiredcapability embed gate is enforced only on the MCPServer resolver path; a hand-built low-levelinput_requiredrequest is transmitted ungated,nullstructured content collapses to absent, and the spec-legalnullis rejected client-side,resultType,scopeparameter.Conformance: cross-checked against the conformance suite at the CI pin — behaviours covered there and here agree; gaps that exist only upstream are tracked for separate filing.
AI Disclaimer