Skip to content

Commit f904b5f

Browse files
CopilotmsyycCopilot
authored
Detect mock api uri / spec route mismatches in http-specs validation (#10978)
`tsp-spector validate-mock-apis` only checked that each scenario had *a* mock API; it never compared the mock API `uri` against the route declared in `main.tsp`. When they diverge, generated clients call one path while the mock server serves another (404 at runtime) — and CI stayed green. The `knock`/server tests can't catch it either, since they derive both the served and called path from the same `uri`. ### Changes - **`@typespec/spector` — new `utils/route-utils.ts`**: `isMockApiUriConsistentWithRoute(template, uri)` compares a spec route against a mock uri segment-by-segment. URI template expressions (`{param}`, `{+param}`, `{param*}`, `{/param}`) are treated as wildcards; literal segments must match exactly; the uri may carry extra trailing segments (for `@path`-annotated params, server-templated api-versions, reserved expansions). Also normalizes express `\:` escaping and strips query strings. - **`@typespec/spector` — `validate-mock-apis.ts`**: each mock api `uri` must be consistent with one of the scenario's spec routes; otherwise a diagnostic is reported and the action exits non-zero. - **`@typespec/http-specs` — `routes/mockapi.ts`**: fixes a genuine mismatch surfaced by the new check — the `Routes_fixed` and `Routes_InInterface` mock uris were swapped. - Unit tests for the matching logic + changesets. ### Example diagnostic ``` ✘ Scenario Parameters_Query_SpecialChar_dollarSign has a mock api uri "/parameters/query/special-char/dollar-sign" that does not match (segment-by-segment, treating route template params as wildcards) any of the routes defined in the spec: "/parameters/query/special-char/dollarSign". ``` ### Note The `Parameters_Query_SpecialChar_dollarSign` mismatch is intentionally left in place (its data fix is handled by #10962) and serves as the live confirmation that the check now fails CI on a real mismatch. Validated against all specs: the check produces no false positives and flags only genuine divergences. ## Test After fix, CI could catch the mismatch now <img width="1097" height="886" alt="image" src="https://github.com/user-attachments/assets/bcaa1647-143c-40b6-99a5-234f80b86b94" /> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com> Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 450e28b commit f904b5f

7 files changed

Lines changed: 442 additions & 5 deletions

File tree

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
changeKind: fix
3+
packages:
4+
- "@typespec/http-specs"
5+
---
6+
7+
Fix the swapped mock api uris for the `Routes_fixed` and `Routes_InInterface` scenarios so they match the routes defined in the spec.
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
changeKind: fix
3+
packages:
4+
- "@typespec/spector"
5+
---
6+
7+
`validate-mock-apis` now verifies that every route defined in a scenario's `main.tsp` is served by at least one of the scenario's mock API `uri`s, so a mismatch between the spec route and the mock api uri (which would make a generated client get a 404 from the mock server) is detected by CI.

packages/http-specs/specs/routes/mockapi.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,8 +49,8 @@ function createTests(uri: string) {
4949
});
5050
}
5151

52-
Scenarios.Routes_InInterface = createTests("/routes/fixed");
53-
Scenarios.Routes_fixed = createTests("/routes/in-interface/fixed");
52+
Scenarios.Routes_InInterface = createTests("/routes/in-interface/fixed");
53+
Scenarios.Routes_fixed = createTests("/routes/fixed");
5454
Scenarios.Routes_PathParameters_templateOnly = createTests("/routes/path/template-only/a");
5555
Scenarios.Routes_PathParameters_explicit = createTests("/routes/path/explicit/a");
5656
Scenarios.Routes_PathParameters_annotationOnly = createTests("/routes/path/annotation-only/a");

packages/spector/src/actions/validate-mock-apis.ts

Lines changed: 82 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,19 @@
1+
import type { Operation } from "@typespec/compiler";
12
import pc from "picocolors";
23
import { logger } from "../logger.js";
34
import { findScenarioSpecFiles, loadScenarioMockApiFiles } from "../scenarios-resolver.js";
4-
import { importSpecExpect, importTypeSpec } from "../spec-utils/import-spec.js";
5+
import { importSpecExpect, importTypeSpec, importTypeSpecHttp } from "../spec-utils/import-spec.js";
56
import { createDiagnosticReporter } from "../utils/diagnostic-reporter.js";
7+
import {
8+
getServerPathPrefixSegmentCount,
9+
isMockApiUriConsistentWithRoute,
10+
normalizeMockApiUri,
11+
} from "../utils/route-utils.js";
12+
13+
interface OperationRouteInfo {
14+
routePath: string;
15+
serverPrefixSegmentCounts: number[];
16+
}
617

718
export interface ValidateMockApisConfig {
819
scenariosPath: string;
@@ -20,6 +31,7 @@ export async function validateMockApis({
2031

2132
const specCompiler = await importTypeSpec(scenariosPath);
2233
const specExpect = await importSpecExpect(scenariosPath);
34+
const httpLib = await importTypeSpecHttp(scenariosPath);
2335
const diagnostics = createDiagnosticReporter();
2436
for (const { name, specFilePath } of scenarioFiles) {
2537
logger.debug(`Found scenario "${specFilePath}"`);
@@ -60,13 +72,80 @@ export async function validateMockApis({
6072

6173
const scenarios = specExpect.listScenarios(program);
6274

75+
// Resolve the real HTTP route of every operation from the spec. Unlike the route summary
76+
// attached to each scenario endpoint, these routes are fully resolved (e.g. ARM routes,
77+
// `@path` parameters and api-version path segments are included), so they can be reliably
78+
// compared against the mock api uris.
79+
const routeInfoByOperation = new Map<Operation, OperationRouteInfo[]>();
80+
const [httpServices] = httpLib.getAllHttpServices(program);
81+
for (const service of httpServices) {
82+
const servers = httpLib.getServers(program, service.namespace);
83+
// A service may declare several `@server`s with different path prefix lengths. Keep every
84+
// distinct prefix length so a mock uri can be matched against any of them rather than forcing
85+
// a single (e.g. shortest) prefix, which could otherwise produce false mismatches.
86+
const serverPrefixSegmentCounts =
87+
servers && servers.length > 0
88+
? [...new Set(servers.map((server) => getServerPathPrefixSegmentCount(server.url)))]
89+
: [0];
90+
for (const httpOperation of service.operations) {
91+
const infos = routeInfoByOperation.get(httpOperation.operation) ?? [];
92+
infos.push({ routePath: httpOperation.path, serverPrefixSegmentCounts });
93+
routeInfoByOperation.set(httpOperation.operation, infos);
94+
}
95+
}
96+
6397
let foundFailure = false;
6498
for (const scenario of scenarios) {
65-
if (mockApiFile.scenarios[scenario.name] === undefined) {
99+
const mockApiScenario = mockApiFile.scenarios[scenario.name];
100+
if (mockApiScenario === undefined) {
66101
foundFailure = true;
67102
diagnostics.reportDiagnostic({
68-
message: `Scenario ${scenario.name} is missing implementation in for ${name} scenario file.`,
103+
message: `Scenario ${scenario.name} is missing an implementation in the ${name} scenario file.`,
69104
});
105+
continue;
106+
}
107+
108+
// Ensure every route defined in the spec is served by at least one mock api `uri`. Otherwise
109+
// a generated client (which calls the spec route) would get a 404 from the mock server.
110+
//
111+
// The check is done per spec route (rather than per mock api uri) on purpose: a scenario may
112+
// legitimately register extra mock handlers that are not declared operations in the spec
113+
// (e.g. long-running-operation status-polling urls or server-driven pagination continuation
114+
// pages), and those should not be flagged.
115+
if (scenario.endpoints.length > 0 && Array.isArray(mockApiScenario.apis)) {
116+
const mockUris = mockApiScenario.apis
117+
.filter((api) => api.kind === "MockApiDefinition")
118+
.map((api) => api.uri);
119+
120+
if (mockUris.length > 0) {
121+
for (const endpoint of scenario.endpoints) {
122+
const routeInfos = routeInfoByOperation.get(endpoint.target);
123+
// Only validate when the route could be resolved. If it could not (e.g. an operation
124+
// without an HTTP route), skip rather than risk a false positive.
125+
if (!routeInfos || routeInfos.length === 0) {
126+
continue;
127+
}
128+
const matched = routeInfos.some((info) =>
129+
mockUris.some((uri) =>
130+
info.serverPrefixSegmentCounts.some((serverPrefixSegmentCount) =>
131+
isMockApiUriConsistentWithRoute(info.routePath, uri, serverPrefixSegmentCount),
132+
),
133+
),
134+
);
135+
if (!matched) {
136+
foundFailure = true;
137+
diagnostics.reportDiagnostic({
138+
message: `Scenario ${scenario.name} defines the route ${routeInfos
139+
.map((info) => `"${info.routePath}"`)
140+
.join(
141+
" or ",
142+
)} but none of its mock api uris match it (route template params are treated as wildcards). Mock api uris: ${mockUris
143+
.map((uri) => `"${normalizeMockApiUri(uri)}"`)
144+
.join(", ")}.`,
145+
});
146+
}
147+
}
148+
}
70149
}
71150
}
72151

packages/spector/src/utils/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@ export * from "./diagnostic-reporter.js";
33
export * from "./file-utils.js";
44
export * from "./misc-utils.js";
55
export * from "./request-utils.js";
6+
export * from "./route-utils.js";
Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
import { describe, expect, it } from "vitest";
2+
import {
3+
getServerPathPrefixSegmentCount,
4+
isMockApiUriConsistentWithRoute,
5+
normalizeMockApiUri,
6+
} from "./route-utils.js";
7+
8+
describe("normalizeMockApiUri", () => {
9+
it("drops the query string", () => {
10+
expect(normalizeMockApiUri("/foo/bar?baz=1")).toBe("/foo/bar");
11+
});
12+
13+
it("removes backslash escapes", () => {
14+
expect(normalizeMockApiUri("/versioning/removed/api-version\\:v1/v3")).toBe(
15+
"/versioning/removed/api-version:v1/v3",
16+
);
17+
});
18+
});
19+
20+
describe("getServerPathPrefixSegmentCount", () => {
21+
it("returns 0 for an undefined server", () => {
22+
expect(getServerPathPrefixSegmentCount(undefined)).toBe(0);
23+
});
24+
25+
it("returns 0 for the default localhost server", () => {
26+
expect(getServerPathPrefixSegmentCount("http://localhost:3000")).toBe(0);
27+
});
28+
29+
it("returns 0 for a bare host server (e.g. ARM)", () => {
30+
expect(getServerPathPrefixSegmentCount("https://management.azure.com")).toBe(0);
31+
});
32+
33+
it("counts the path segments after an {endpoint} template", () => {
34+
expect(
35+
getServerPathPrefixSegmentCount(
36+
"{endpoint}/resiliency/service-driven/client:v2/service:{serviceDeploymentVersion}/api-version:{apiVersion}",
37+
),
38+
).toBe(5);
39+
});
40+
41+
it("counts the path segments after localhost:3000", () => {
42+
expect(getServerPathPrefixSegmentCount("http://localhost:3000/my/prefix")).toBe(2);
43+
});
44+
});
45+
46+
describe("isMockApiUriConsistentWithRoute", () => {
47+
it("matches identical routes", () => {
48+
expect(
49+
isMockApiUriConsistentWithRoute("/parameters/basic/simple", "/parameters/basic/simple"),
50+
).toBe(true);
51+
});
52+
53+
it("detects a mismatch in a literal segment", () => {
54+
// A literal route segment must match exactly: `dollarSign` (camelCase) must not be considered
55+
// consistent with a `dollar-sign` (kebab-case) uri.
56+
expect(
57+
isMockApiUriConsistentWithRoute(
58+
"/parameters/query/special-char/dollarSign",
59+
"/parameters/query/special-char/dollar-sign",
60+
),
61+
).toBe(false);
62+
});
63+
64+
it("detects swapped routes", () => {
65+
expect(isMockApiUriConsistentWithRoute("/routes/fixed", "/routes/in-interface/fixed")).toBe(
66+
false,
67+
);
68+
});
69+
70+
it("treats whole-segment template expressions as wildcards", () => {
71+
expect(
72+
isMockApiUriConsistentWithRoute(
73+
"/routes/path/template-only/{param}",
74+
"/routes/path/template-only/a",
75+
),
76+
).toBe(true);
77+
});
78+
79+
it("matches template expressions embedded in a segment", () => {
80+
expect(
81+
isMockApiUriConsistentWithRoute(
82+
"/routes/path/simple/standard/primitive{param}",
83+
"/routes/path/simple/standard/primitivea",
84+
),
85+
).toBe(true);
86+
});
87+
88+
it("matches path expansion template expressions that expand to extra segments", () => {
89+
expect(
90+
isMockApiUriConsistentWithRoute(
91+
"/routes/path/path/standard/primitive{param}",
92+
"/routes/path/path/standard/primitive/a",
93+
),
94+
).toBe(true);
95+
});
96+
97+
it("detects a uri that is missing a trailing literal segment present in the route", () => {
98+
expect(isMockApiUriConsistentWithRoute("/parameters/basic/simple", "/parameters/basic")).toBe(
99+
false,
100+
);
101+
});
102+
103+
it("ignores trailing slashes", () => {
104+
expect(
105+
isMockApiUriConsistentWithRoute(
106+
"/azure/special-headers/x-ms-client-request-id/",
107+
"/azure/special-headers/x-ms-client-request-id",
108+
),
109+
).toBe(true);
110+
});
111+
112+
it("ignores the query string of both the route and the uri", () => {
113+
expect(
114+
isMockApiUriConsistentWithRoute(
115+
"/routes/query/query-continuation/standard/primitive?fixed=true",
116+
"/routes/query/query-continuation/standard/primitive?fixed=true&param=a",
117+
),
118+
).toBe(true);
119+
});
120+
121+
it("matches routes containing escaped characters in the uri", () => {
122+
expect(
123+
isMockApiUriConsistentWithRoute(
124+
"/versioning/removed/api-version:{version}/v3",
125+
"/versioning/removed/api-version\\:v1/v3",
126+
),
127+
).toBe(true);
128+
});
129+
130+
describe("with a server path prefix", () => {
131+
// Resiliency service-driven: the api-version/client/service version segments come from the
132+
// `@server` url and may legitimately differ from the mock uri (e.g. client:v1 vs client:v2).
133+
const serverPrefix = getServerPathPrefixSegmentCount(
134+
"{endpoint}/resiliency/service-driven/client:v2/service:{serviceDeploymentVersion}/api-version:{apiVersion}",
135+
);
136+
137+
it("skips the server prefix segments before comparing", () => {
138+
expect(
139+
isMockApiUriConsistentWithRoute(
140+
"/add-optional-param/from-none",
141+
"/resiliency/service-driven/client\\:v1/service\\:v1/api-version\\:v1/add-optional-param/from-none",
142+
serverPrefix,
143+
),
144+
).toBe(true);
145+
});
146+
147+
it("still detects a mismatch in the operation route after the server prefix", () => {
148+
expect(
149+
isMockApiUriConsistentWithRoute(
150+
"/add-optional-param/from-none",
151+
"/resiliency/service-driven/client\\:v2/service\\:v2/api-version\\:v2/add-operation",
152+
serverPrefix,
153+
),
154+
).toBe(false);
155+
});
156+
});
157+
158+
describe("ARM routes", () => {
159+
it("matches a fully resolved ARM tracked resource route", () => {
160+
expect(
161+
isMockApiUriConsistentWithRoute(
162+
"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Azure.ResourceManager.Resources/topLevelTrackedResources/{topLevelTrackedResourceName}",
163+
"/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/test-rg/providers/Azure.ResourceManager.Resources/topLevelTrackedResources/top",
164+
),
165+
).toBe(true);
166+
});
167+
168+
it("matches an extension resource route whose {resourceUri} spans several scope segments", () => {
169+
const route =
170+
"/{resourceUri}/providers/Azure.ResourceManager.Resources/extensionsResources/{extensionsResourceName}";
171+
expect(
172+
isMockApiUriConsistentWithRoute(
173+
route,
174+
"/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/test-rg/providers/Azure.ResourceManager.Resources/extensionsResources/extension",
175+
),
176+
).toBe(true);
177+
expect(
178+
isMockApiUriConsistentWithRoute(
179+
route,
180+
"/providers/Azure.ResourceManager.Resources/extensionsResources/extension",
181+
),
182+
).toBe(true);
183+
});
184+
});
185+
});

0 commit comments

Comments
 (0)