Skip to content

ci: typecheck the test tree too (C-03) - #404

Merged
jpr5 merged 1 commit into
mainfrom
c03-typecheck-tests
Sep 2, 2026
Merged

ci: typecheck the test tree too (C-03)#404
jpr5 merged 1 commit into
mainfrom
c03-typecheck-tests

Conversation

@jpr5

@jpr5 jpr5 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

What

tsconfig.json excludes src/__tests__, so the 180-file / 5,638-test suite has never been type-checked. The typecheck CI job added in #402 covers src/ and scripts/ only.

Removing the exclude line does not work. The root project pins rootDir: "src", and tests import across into scripts/ — that produces 11 TS6059 rootDir violations. This needed its own project, as the plan said.

The tsconfig split

New tsconfig.test.json:

  • extends: "./tsconfig.json" — same strict, same target/module/resolution, so the tests are held to the same bar as src/.
  • rootDir: "." and declaration: false — both are emit-shape settings, and this project is --noEmit. Dropping rootDir is what removes the 11 TS6059s without weakening a single check.
  • include: ["src", "vitest.config.ts", "vitest.config.drift.ts"], exclude: ["node_modules", "dist"] — the tests are in, and the two vitest configs get covered for free.

pnpm typecheck now runs tsc -p tsconfig.json && tsc -p tsconfig.test.json && tsc -p scripts/tsconfig.json. No workflow change — the existing typecheck job already runs pnpm typecheck.

Program membership, not just a green exit: tsc -p tsconfig.test.json --listFiles | grep -c src/__tests__ = 223 (vs 0 for the root project).

Measured, not carried forward

plan measured on 3a1b1de
errors 126 115
files ~40 29

By code: 38×TS2345, 21×TS2352, 17×TS2531, 8×TS18048, 6×TS2339, 6×TS2322, 5×TS2769, 4×TS2683, 4×TS2554, 3×TS2741, 1×TS5097, 1×TS2353, 1×TS18047.

TS6059 shows as 0 here only because this config drops rootDir. The naive fix still hits all 11 — see proof 1b below.

No suppressions

No as any, no @ts-ignore, no @ts-expect-error, no new eslint-disable, no widened exclude, no file relocation. git diff | grep -E '^\+.*(as any|@ts-ignore|@ts-expect-error|eslint-disable)' → empty.

Real defects the gate caught:

  • isFamilyStillReferenced takes one argument; four calls in models.drift.ts passed two. The second argument was silently discarded.
  • recorder-path-traversal.test.ts set endpoint: "/v1/chat/completions", which is not in the FixtureMatch["endpoint"] union at all. The value is "chat".
  • HandlerDefaults.replaySpeed is required and 24 test defaults literals omitted it.
  • createMockRes().writeHead modelled only one of Node's two overloads — it would have thrown away headers on a (status, statusMessage, headers) call.
  • competitive-matrix.test.ts imported a .ts extension; every other test uses .js, which resolves the same under NodeNext.

The rest are honest assertions where the type is genuinely nullable (JournalEntry.body), narrowing that never happened (let x: T | undefined = await new Promise(...) — the promises are now explicitly generic), or missing required fields (AGUIMessage.id, AGUIRunAgentInput.threadId/runId, SSEChoice.logprobs). Two as AGUIRunAgentInput casts are deleted because the literals became valid.

Where a cast was genuinely needed, it is one documented single-site helper (looseEntry in fix-drift.test.ts, isRestorableSpy in fixture-loader.test.ts) rather than a scatter of double casts.

Red-green proof

1 — RED, the real state. origin/main @ 3a1b1de in a clean worktree, tsconfig.test.json dropped in, nothing else changed:

RED RC=2
115 errors
29 files

1b — the structural blocker is real. On the same pristine tree, the naive fix (delete src/__tests__ from exclude, keep rootDir):

$ npx tsc -p tsconfig.naive.json --noEmit 2>&1 | grep -c "error TS6059"
11
scripts/drift-report-collector.ts(41,8): error TS6059: File '.../scripts/drift-types.ts' is not under 'rootDir' '.../src'.

Exactly the 11 the plan predicted.

2 — GREEN on the real tree.

$ pnpm typecheck ; echo $?
> tsc -p tsconfig.json --noEmit && tsc -p tsconfig.test.json --noEmit && tsc -p scripts/tsconfig.json --noEmit
0

3 — the gate FAILS on a deliberate error. Appended to src/__tests__/jsonrpc.test.ts:

const deliberateTypeError: number = "C-03 gate must catch this";
$ pnpm typecheck ; echo $?
src/__tests__/jsonrpc.test.ts(517,7): error TS2322: Type 'string' is not assignable to type 'number'.
 ELIFECYCLE  Command failed with exit code 2.
2

4 — removed, green again.

$ pnpm typecheck ; echo $?
0

Gates

gate exit
prettier --check . 0
npx eslint . 0
pnpm typecheck 0
npx vitest run 0
npx commitlint --from origin/main --to HEAD 0

Suite unchanged against baseline: 179 passed / 1 skipped (180 files), 5592 passed / 46 skipped (5638 tests). No workflow touched, so no actionlint. No dependency change, so no lockfile drift.

Nothing left unfixed

All 115 are fixed. Zero remainder.

`tsconfig.json` excludes `src/__tests__`, so the 180-file suite has never
been type-checked. Removing that exclude does not work: the root project
pins `rootDir: "src"`, and tests import across into `scripts/`
(`adoption-wall.test.ts` -> `scripts/update-adoption-wall.ts`), which
yields 11 TS6059 rootDir violations.

Add `tsconfig.test.json` instead: extends the root project, drops
`rootDir` (meaningless under `noEmit`) and `declaration`, and includes
`src` with the tests plus the two vitest configs. `pnpm typecheck` now
runs it between the root and scripts projects, so the existing
`typecheck` CI job covers the suite with no workflow change.

Then fix the 115 errors across 29 files it surfaces. No suppressions —
no `as any`, no `@ts-ignore`, no widened excludes:

- `HandlerDefaults.replaySpeed` is required; test `defaults` literals
  omitted it.
- `createMockRes().writeHead` only modelled one of the two real
  overloads; it now handles `(status, statusMessage, headers)` as well.
- `JournalEntry.body` is nullable; assertions now say so.
- `let x: T | undefined = await new Promise(...)` never narrowed —
  the promises are now explicitly generic.
- `AGUIRunAgentInput` requires `threadId`/`runId` and `AGUIMessage`
  requires `id`; test inputs now supply them, and two `as
  AGUIRunAgentInput` casts are gone because the literals are now valid.
- `isFamilyStillReferenced` takes one argument; four calls passed two.
- `recorder-path-traversal` used `endpoint: "/v1/chat/completions"`,
  which is not in the `FixtureMatch["endpoint"]` union — it is `"chat"`.
- `competitive-matrix.test.ts` imported a `.ts` extension; every other
  test uses `.js`, which resolves the same under NodeNext.
- Remaining cast-shaped errors are fixed by reading through the type
  that already exists (`SSEChunk.system_fingerprint`, the
  `ResponsesSSEEvent` index signature) or by one documented,
  single-site widening helper (`looseEntry`, `isRestorableSpy`).

Suite unchanged: 5592 passed / 46 skipped, 180 files.
@pkg-pr-new

pkg-pr-new Bot commented Sep 2, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@404

commit: f202e41

@jpr5
jpr5 merged commit 79d51e6 into main Sep 2, 2026
29 checks passed
@jpr5
jpr5 deleted the c03-typecheck-tests branch September 2, 2026 00:54
jpr5 added a commit that referenced this pull request Sep 8, 2026
…eturns

The merge with main reddened `pnpm run typecheck`, which type-checks the TEST
tree too (tsconfig.test.json, added in #404 after this branch was cut) —
`tsc -p tsconfig.json` alone stays green, which is why this survived the local
check that only ran the src config.

`readRecordedFixture` read `FixtureFile.fixtures[0]` and declared it a runtime
`Fixture`. Those are different types: `FixtureFile.fixtures` is
`FixtureFileEntry[]`, the on-disk JSON shape, and `Fixture` is what the server
holds after `entryToFixture` converts it. The helper was asserting the
conversion had already happened.

Return the entry type it actually reads, and convert at the two places that
need a runtime fixture — the `createServer` call and the `validateFixtures`
assertions — with `entryToFixture`, which is exactly what the real load path
does (`handleControlAPI` converts before validating). That also makes the
"a fixture the recorder writes must pass load-time validation" assertion
stronger than it was: it now runs the same convert-then-validate sequence the
loader runs, instead of validating a value mistyped as already-converted.

pnpm run typecheck (all three configs) exit 0; full suite 183 files /
5690 tests passing; eslint and prettier clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant