Repin conformance harness to the published 0.2.0-alpha.10 - #3184
Conversation
The pkg.pr.new pin of conformance main@da56f663 was a stopgap until a release carrying #403 (sep-2575 checks aligned with spec #3002) and #406 (per-check expected-failures) shipped. 0.2.0-alpha.10 is now on npm with both, so point CONFORMANCE_PKG at the published spec and drop the sha256 fetch-and-verify steps that only the tarball URL needed. No-Verification-Needed: CI-config-only conformance pin bump
There was a problem hiding this comment.
LGTM — straightforward CI-only repin of the conformance harness to the published npm release.
What was reviewed:
- The workflow diff:
CONFORMANCE_PKGmoves from the pkg.pr.new tarball URL back to an exact registry spec (0.2.0-alpha.10), and the fetch-and-verify steps that only the URL pin needed are removed symmetrically from both jobs. - Checked for dangling references to
CONFORMANCE_PKG_SHA256or pkg.pr.new across.github/— none remain;run-server.sh/run-client.shconsumeCONFORMANCE_PKGunchanged. expected-failures.ymlchanges are comment-only (removing stale references to the da56f663 pin); no baseline entries added or removed.
Extended reasoning...
Overview
This PR touches only two CI files: .github/workflows/conformance.yml and .github/actions/conformance/expected-failures.yml. It repins the conformance harness from a temporary pkg.pr.new tarball build (main@da56f663) to the published @modelcontextprotocol/conformance@0.2.0-alpha.10 on npm, and removes the sha256 fetch-and-verify steps that existed solely to pin the bytes of the tarball URL. The expected-failures baseline changes are comment wording only — no entries added or removed, consistent with the PR description's claim that the two extra commits in the release don't change results against this SDK.
Security risks
The supply-chain posture is a mild consideration: the previous state verified a recorded sha256 digest of the tarball, while the new state relies on npm's registry integrity checking for an exact-version spec. Exact versions on the npm registry are immutable, and this restores the workflow's documented normal state (the pre-existing comment said to use a published version spec). This is a return to the established pattern, not a weakening of it. No secrets, permissions, or auth surfaces are touched; the workflow keeps permissions: contents: read and persist-credentials: false.
Level of scrutiny
Low — this is CI-only configuration with no effect on shipped code. The main things worth checking were (a) that the fetch-and-verify removal was symmetric across both jobs (it is), (b) that nothing else references the now-deleted CONFORMANCE_PKG_SHA256 env var or the pkg.pr.new URL (grep over .github/ confirms nothing does), and (c) that the run scripts still receive CONFORMANCE_PKG in a form npx --yes accepts (a plain registry spec, which is the original supported form).
Other factors
The bug hunting system found no issues. The PR author reports running all six CI legs locally against the 0.2.0-alpha.10 tarball with all legs passing and no stale baseline entries, and CI itself will independently verify this claim on the PR — a stale or newly-failing entry would fail the run. There are no outstanding reviewer comments. The change follows the workflow's own documented bump procedure ("Bump deliberately and reconcile both expected-failures*.yml files in the same change").
Repins the conformance harness from the temporary pkg.pr.new build of
main@da56f663to the published@modelcontextprotocol/conformance@0.2.0-alpha.10, and drops the sha256 fetch-and-verify steps that only the tarball URL needed.Motivation and Context
The pkg.pr.new pin was a stopgap until an npm release carrying conformance #403 (sep-2575 checks aligned with spec #3002) and #406 (per-check expected-failures granularity) shipped.
0.2.0-alpha.10is now on npm (thealphadist-tag) with both, soCONFORMANCE_PKGcan go back to a plain registry spec — npm's own integrity check replaces the recorded digest.The release adds two commits beyond the previous pin (conformance #409, #383); neither changes results against this SDK, so both
expected-failures*.ymlbaselines are unchanged apart from comment wording.How Has This Been Tested?
Ran all six CI legs locally against the
0.2.0-alpha.10tarball (server active/draft/2026-07-28/all, client all/2026-07-28) — every leg passes with the existing baselines and no stale entries.Breaking Changes
None; CI-only.
Types of changes
Checklist