Skip to content

Emit TypeScript transfer type converters and operation type info - #124

Merged
bergundy merged 10 commits into
mainfrom
ts-transfer-type-encoding
Aug 21, 2026
Merged

Emit TypeScript transfer type converters and operation type info#124
bergundy merged 10 commits into
mainfrom
ts-transfer-type-encoding

Conversation

@bergundy

@bergundy bergundy commented Aug 14, 2026

Copy link
Copy Markdown
Member

Nine commits, best reviewed one at a time. The headline is the TypeScript output: a model's companion becomes a TransferTypeConverter instance, and every generated operation carries the converter for each non-void side, so a protocol integration can apply the conversion without the caller wiring it by hand. Behind it, three fixes to the emitted-name layer — it had to learn the new identifiers, and teaching it surfaced two pre-existing holes — and then three more on multi-input correctness, where checking that the new converter identifiers were scoped properly turned up naming and file-layout bugs that emit code the target's own compiler rejects.

The first commit, 85a8ba1 ("Reject a root type name that collides with a $defs name"), is separate self-contained work that was sitting on local main unpushed; it rides along here rather than being split out. Review starts at 3d12399.

TypeScript: mappers become transfer type converters

Implements the contract from nexus-rpc/sdk-typescript#40.

// before
export class UserMapper {
  fromIntermediate(value: unknown): User {  }
  toIntermediate(value: User): unknown {  }
}
new UserMapper().fromIntermediate(raw)

// after
export const userTransferTypeConverter = new class implements TransferTypeConverter<User> {
  fromTransferType(value: unknown): User {  }
  toTransferType(value: User): unknown {  }
}()
userTransferTypeConverter.fromTransferType(raw)

Method bodies are unchanged: TransferTypeConverter<T> defaults its transfer type to unknown, which already matched the emitted signatures. models.ts gains a type-only import type { TransferTypeConverter } from "nexus-rpc".

The identifier is the model's resolved type identifier lower-camel-cased, read back through the name manifest wherever a reference is emitted — the plan hands operations their own type copies, which carry the derived name, not the resolved one — so an x-ts-name override moves the type, its converter, and the operation that names them, together.

Two consequences for load-time naming, both of which used to be silent bad output:

  • TransferTypeConverter is now imported into every models.ts, so it joins TypeScript's reserved runtime identifiers: a $defs type of that name rejects at load with a fix-it instead of emitting an import/local conflict.
  • The derived converter identifiers enter the per-module P15 namespace, because lower-camel-casing folds names the type namespace keeps apart — HTTPError and HttpError both yield httpErrorTransferTypeConverter. A fold, or a service whose x-ts-name lands on a converter name, is a P15 load reject instead of a module that declares one const twice. The parser owns the derivation, as it owns the rest of the per-language naming policy, so the collision pass and the emitters cannot drift apart.

Operation type info

Each non-void side of nexus.operation now carries inputType/outputType = { transferTypeConverter: … } naming the I/O model's converter. A void side carries neither field: there is no value to convert, and an empty TypeInfo would assert a conversion that does not exist. Cross-module I/O imports the converter as a value from its declaring module.

JSON-Schema input only — the WIT tier has no converters and its output is byte-identical.

The nexus-rpc shim

The published nexus-rpc 0.0.2 predates the type-info API, and upstream has no prepare script, so a git dependency installs unbuilt. Runtime is unaffected (operation() spreads its options through untouched), so both sample projects carry a types-only augmentation under shims/ to bridge typechecking until the release lands. It comes out when that release ships.

Emitted-name fixes

  • A model's x-<lang>-name override was invisible to every other input file that referenced it, in all four languages. Each leaf built its manifest from its own declarations only, so a consuming module emitted the pre-override identifier: a dangling operation generic, model import and converter import, a dangling field type for a cross-file $ref property, and — in Go's flat package — a reference to a type name that is never declared. EmittedNameResolutionPass now builds one manifest over the whole tree and rewrites every emitted name from it, references included (routed through ApiSpec::map_names, so operation I/O and record field types are reached). Collision scoping is untouched: build_name_manifest groups by module key, so a foreign model never joins a local module's namespace. Cross-module $refs inside a model's schema are resolved by the backends, which see one leaf at a time, so the pass records the foreign identifiers on PlannedSpecData::cross_module_model_names and each backend seeds its $ref registry with them — replacing the Go backend's approximation, which keyed entries by the pre-override name.
  • An x-<lang>-name on a property did not move two identifiers synthesized from it, so a collision on either was rejected with a fix-it the author could not act on. TypeScript's DEFAULT_<FIELD> and Go's closed-value type <Type><Field> derived from the JSON key rather than the emitted member identifier: retryCount and retry_count collide on DEFAULT_RETRY_COUNT, and the rejection's own remedy moved the members apart while leaving both constants on the colliding name — the only escape left was renaming the JSON property, i.e. changing the wire contract. Both now derive from the emitted member identifier, in the emitters and in the manifest that predicts them, which also puts Go's synthesized type name in agreement with Java's nested value class. The governing rule — a name synthesized from a member moves with that member; a name synthesized from a position stays with the position — now sits in PRINCIPLES §15, which previously claimed the override moved every name synthesized from the property.
  • TypeScript's <FIELD>_CONST bindings never entered the P15 namespace, so two could coincide and both be emitted. They coincide through the model-name disambiguator: A.kind is prefixed because kind is not unique, giving A_KIND_CONST, which is exactly what a unique C.aKind produces unprefixed — const A_KIND_CONST twice in one models.ts, a TypeScript SyntaxError emitted without a diagnostic. Now a reject with a fix-it naming both origins. (default.md and const.md both claimed TypeScript synthesizes no identifier for a const; it synthesizes no named type, but it does emit this value binding, and both now say so.)

Generated samples move only where the TypeScript converters do: no checked-in schema puts a name override on a cross-file model, or on a default- or const-bearing property.

Multi-input correctness

Three later commits, all about generating from a directory of schema files rather than a single file. They surfaced while checking that the new converter identifiers were scoped correctly, and each one turned out to be a pre-existing hole that produces code the target's own compiler rejects. No checked-in sample changes for any of them — see the end of this section for why that is expected rather than reassuring.

0b9ba68 — collision scope per target

Two scope errors, plus a third found while fixing them:

  • Services never entered the pass in multi-input mode. They were inserted only when the module key was empty — the root module, which in multi-input mode no file occupies. A service clashing with a model in its own file rejected when that file was the sole input, and was silently accepted when the same file sat in a directory.
  • Go was scoped per module, but flattens every module into one package. Two input files each declaring a Page emitted type Page struct twice into that package, with no diagnostic. Confirmed with the Go type checker: Page redeclared in this block.
  • Making Go's scope package-wide was not enough on its own. A collision is detected by identifier, and two models named Page in different modules produced the identical origin string type `Page` — which the namespace read as one declaration re-inserted rather than a clash. Origins now carry the module-qualified name, so the dedupe fires only for what is genuinely one declaration seen twice, and the diagnostic names both sides (a/page#Page and b/page#Page).
  • A TypeScript service identifier was derived as a type name while the generator emits a lower-camel const. That produced a false rejection — chatService and ChatService are distinct identifiers and generate cleanly — and missed the clash TypeScript really has: a service whose lower-camel form lands on a model's <model>TransferTypeConverter. Widening service insertion while the derivation was wrong would have spread the false positive into every module, so it had to come along. Manifest-only; no emitted name changes.

931f5d2 — TypeScript and Python resolve run-wide too

The same cross-module defect as Go, by a different route: both emit a root barrel that lifts every module's top-level names into one namespace — index.ts re-exporting each module with export *, __init__.py re-exporting them by name. Two input files each declaring a Page collide there.

Left to the target, TypeScript emits a barrel its own compiler rejects (TS2308, and the model's pageTransferTypeConverter collides alongside the interface). Python is worse: from .a import Page followed by from .b import Page raises nothing, binds the second, and lists the name once in __all__ — so from pkg import Page quietly resolves to the wrong model, the silent incorrectness P7 exists to prevent.

Java and .NET are genuinely unaffected, and this was checked against both generators rather than inferred from the directory layout: each module lands in its own sub-package/namespace (com.example.api.a.page, Nexgen.Generated.A.Page) and neither emits an aggregating barrel, so two Page types stay distinct.

The scope is now chosen by a documented predicate over the emitted layout instead of a Go special case, since the reason differs per target. PRINCIPLES.md §15, generated-file-layout.md, and ref.md all asserted the superseded rule and are corrected — ref.md had claimed one package-wide namespace for every target.

3a3575a — a $ref'd type is emitted only by its declaring module

A module re-emitted the types it only $refd from another input file whenever that module declared no types of its own — a service file whose every operation type is a $ref elsewhere. Every target emitted a second copy:

symptom
Go Page redeclared in this block (go build)
Java duplicate Page.java, build fails with a generated-file conflict
TypeScript module both imports and redeclares the name (TS2440), barrel re-exports it twice (TS2308)
Python two classes; barrel imports both and silently binds one (P7)
.NET duplicate class

The cause was in reachability pruning, which decides whether to drop declarations another module owns. It inferred "this front end does not scope declarations by module" from the module owning nothing — a state indistinguishable from "this module declares nothing" while the flag is a bool, so a service-only JSON module looked like a WIT spec and kept everything it could reach. A declaration now records Foreign distinctly from Unscoped, which is the distinction the prune was actually reaching for.

Dropping the re-emitted copies leaves such a module with an empty models.ts, and a barrel re-exporting a file with no exports is itself an error (TS2306), so TypeScript now emits neither the file nor the ./models re-export.

Why no sample churn. Every checked-in sample service module happens to declare at least one inline operation type, which suppresses the bug entirely — adding one unrelated inline type to a broken module makes it vanish. That is why this went unnoticed, and it was pinned down by bisecting from the working kb sample down to the trigger. Each of the four new regression tests was confirmed to fail without the fix, and the repaired output was checked with the Go compiler, javac, tsc, and a Python import rather than by inspection.

Also

unique_temp_dir in the test suite derived its path from the wall clock alone, while six tests pass write_oneof_descriptor one fixed name — two threads reading the same timestamp shared a directory and fs::write the same api.bin concurrently, so a third could read it truncated mid-write. The clock's granularity, not the test, decided whether the run was green. A process-wide counter now joins the path.

@bergundy
bergundy requested a review from a team as a code owner August 14, 2026 22:06
@bergundy

Copy link
Copy Markdown
Member Author

./scripts/validate.sh passes at 35ca5c6 (all Rust tests plus the Go/TypeScript/Python/Java sample suites; regeneration leaves the tree clean).

Comment thread src/spec.rs Outdated
Comment thread CHANGELOG.md Outdated
Comment thread src/planning/type_planning.rs
Comment thread src/generator/json_schema/mod.rs
@bergundy
bergundy requested review from a team and tconley1428 August 18, 2026 15:46
Comment thread src/planning/mod.rs Outdated
@bergundy
bergundy requested a review from tconley1428 August 19, 2026 18:42
@bergundy
bergundy force-pushed the ts-transfer-type-encoding branch from 37f1416 to c10a974 Compare August 19, 2026 19:08
Comment thread src/lib.rs Outdated
Comment thread src/planning/mod.rs Outdated
Comment thread src/planning/emitted_names.rs Outdated
@bergundy
bergundy force-pushed the ts-transfer-type-encoding branch from c10a974 to 86701e8 Compare August 20, 2026 20:38
A file whose root schema derived the name of one of its own `$defs`
entries silently collapsed the two into a single type: the external-type
map is keyed by the model's identity (the derived name), so the second
insert was dropped by `or_insert_with`. The root's shape won, the `$defs`
shape's members vanished, and every `$ref` at the lost entry retargeted
at the root — compilable but wrong output in all four languages (`thing.yaml`
with `$defs.Thing` emitted one self-referencing `Thing`).

The coincidence is now a load reject for every target (P15), naming the
identifier and both origins with the renames that resolve it; an
`x-<lang>-name` override cannot, because it moves the emitted identifier
and not the identity. A name synthesized for an inline shape is held to
the same rule and reported at the position the shape was written in, for
which hoisted definitions now carry their authored origin. Behind both,
`insert_json_external_type` rejects any two distinct schemas that reach
one identity rather than dropping one.
Replace the TypeScript output's mapper concept with nexus-rpc's
TransferTypeConverter contract (nexus-rpc/sdk-typescript#40) and wire it
into every generated operation.

A model's companion is now an exported instance rather than a class:
`class <Type>Mapper` with fromIntermediate/toIntermediate becomes
`export const <type>TransferTypeConverter = new class implements
TransferTypeConverter<Type> { … }()` with
fromTransferType/toTransferType. Call sites drop the construction. The
identifier is the model's resolved type identifier lower-camel-cased,
read back through the name manifest wherever a reference is emitted —
the plan hands operations their own type copies, which carry the derived
name, not the resolved one — so an x-ts-name override moves the type,
its converter, and the operation that names them together. Method bodies
are unchanged — TransferTypeConverter<T> defaults its transfer type to
`unknown`, which already matched the emitted signatures.

Each non-void side of nexus.operation now carries
inputType/outputType = { transferTypeConverter: … } naming the I/O
model's converter, so a protocol integration can apply the conversion
without the caller wiring it by hand. A void side carries neither field:
there is no value to convert, and an empty TypeInfo would assert a
conversion that does not exist. Cross-module I/O imports the converter as
a value from its declaring module. JSON-Schema input only — the WIT tier
has no converters and its output is byte-identical.

Since TransferTypeConverter is imported into every models.ts, it joins
TypeScript's reserved runtime identifiers: a $defs type of that name now
rejects at load with a fix-it instead of emitting an import/local
conflict. The derived converter identifiers enter that same per-module
namespace, because lower-camel-casing folds names the type namespace
keeps apart — HTTPError and HttpError both yield
httpErrorTransferTypeConverter. A fold, or a service whose x-ts-name
lands on a converter name, is now a P15 load reject instead of a module
that declares one const twice. The parser owns the derivation, as it owns
the rest of the per-language naming policy, so the collision pass and the
emitters cannot drift apart.

The published nexus-rpc 0.0.2 predates the type-info API and upstream has
no prepare script, so a git dependency installs unbuilt. Runtime is
unaffected (operation() spreads its options through untouched), so both
sample projects carry a types-only augmentation under shims/ to bridge
typechecking until the release lands.
A model's `x-<lang>-name` override was invisible to every other input file
that referenced it. Each leaf built its name manifest from its own
declarations only, so a consuming module emitted the pre-override
identifier in all four languages: a dangling operation generic, model
import and TypeScript converter import, a dangling field type for a
cross-file `$ref` property, and -- in Go's flat package -- a reference to
a type name that is never declared.

`EmittedNameResolutionPass` now builds one manifest over the whole tree
and rewrites every emitted name from it: the leaf's declarations, every
reference (routed through `ApiSpec::map_names`, so operation I/O and
record field types are reached too), and the names `module_imports` is
rendered from. Collision scoping is untouched -- `build_name_manifest`
groups the models it is handed by module key, so a foreign model never
joins a local module's namespace (P15).

Cross-module `$ref`s inside a model's schema are resolved by the
backends, which see one leaf at a time, so the pass records the foreign
identifiers on `PlannedSpecData::cross_module_model_names` and each
backend seeds its `$ref` registry with them. That replaces the Go
backend's approximation of the same map, which keyed entries by the
pre-override name.

Generated samples are byte-identical: no sample declares a cross-file
name override.
An `x-<lang>-name` override on a property did not reach two of the
identifiers synthesized from that property, so a collision on either was
rejected with a fix-it the author could not act on.

The TypeScript `DEFAULT_<FIELD>` constant and the Go closed-value type
`<Type><Field>` (with its value constants) derived from the JSON key
rather than the emitted member identifier. Two default-bearing members
that recase alike — `retryCount` and `retry_count` — collide on
`DEFAULT_RETRY_COUNT`, and the rejection's own remedy, "disambiguate with
an `x-ts-name` override", moved the members apart while leaving both
constants on the colliding name. Applying the override to one member, then
to both, rejects identically; the only escape left was renaming the JSON
property, which changes the wire contract. The Go closed-value type
misfired the same way against a declared type name, and disagreed with
Java, whose nested value class was already named off the emitted member.

Both now derive from the emitted member identifier, in the emitters and in
the P15 manifest that has to predict them, so the documented escape hatch
resolves the clash and Go and Java agree on the synthesized type's name.

The rule this follows — a name synthesized from a member moves with that
member; a name synthesized from a position stays with the position — now
sits in PRINCIPLES §15, which previously claimed the override moved every
name synthesized from the property. `properties.md` carried the same
overstatement while separately (and correctly) documenting the position
exception. An inline object hoisted to `<Model><Property>` is
position-derived and is still renamed by authoring it in `$defs`.

No generated sample moves: no checked-in schema puts a name override on a
`default`- or `const`-bearing property.
A `const`-bearing member emits a module-level `<FIELD>_CONST` holding the
fixed wire value, named `<MODEL>_<FIELD>_CONST` when the member identifier
is not unique across the module's models. That binding never entered the
P15 namespace, so two of them could coincide and both be emitted.

They coincide through the model-name disambiguator: `A.kind` is prefixed
because `kind` is not unique, giving `A_KIND_CONST`, which is exactly what
a unique `C.aKind` produces unprefixed. Before this change that schema
generated `const A_KIND_CONST` twice in one `models.ts` — a TypeScript
SyntaxError, emitted without a diagnostic. It now rejects with a fix-it
naming both origins, and an `x-ts-name` on either member resolves it.

Unexported is not out of scope: a module-level `const` is a module-scope
binding, and a redeclaration is an error whether or not it is exported.

`default.md` and `const.md` both claimed TypeScript synthesizes no
identifier for a `const`. It synthesizes no named *type* — the type closes
to an inline literal — but it does emit this value binding, and both now
say so.
`add_rpc_uses_the_same_oneof_variant_rendering` failed intermittently
under the full suite and passed when run alone.

`unique_temp_dir` derived the directory from the wall clock only, and
`write_oneof_descriptor` passes one fixed name while six tests call it.
Two threads reading the same timestamp share a directory and `fs::write`
the same `api.bin` concurrently, so a third can read the file truncated
mid-write. The clock's granularity, not the test, decided whether the run
was green.

Add a process-wide counter to the path. Five consecutive runs of the suite
pass, and `scripts/validate.sh` is green.
Two scope errors let multi-input runs accept schemas that generate
uncompilable code.

Services were entered into the namespace only when the module key was
empty — the root module, which in multi-input mode no file occupies. A
service clashing with a model in its own file therefore rejected when that
file was the sole input and was silently accepted when the same file sat in
a directory. Services now enter the namespace of the module that declares
them.

Every target was also scoped per module, but Go flattens every module into
one package. Two input files each declaring a `Page` emitted the type twice
into that package — `Page redeclared in this block`, confirmed with the Go
type checker — and the pass said nothing. Go's scope is now the whole input
closure; the other three targets keep per-module scoping, where separate
modules genuinely keep the names apart.

Making Go's scope package-wide was not enough on its own: a collision is
detected by identifier, and two models named `Page` in different modules
produced the identical origin string `type `Page``, which the namespace
read as one declaration re-inserted rather than a clash. Origins now carry
the module-qualified name, so the diagnostic names both sides
(`a/page#Page` and `b/page#Page`) and the dedupe only fires for what is
genuinely the same declaration seen twice.

A third defect surfaced while fixing these: a TypeScript service
identifier was derived as a type name, though the generator emits a
lower-camel `const`. The pass rejected a service and a model of the same
name, which generate cleanly as `chatService` and `ChatService`, and missed
the clash TypeScript does have — a service whose lower-camel form lands on
a model's `<model>TransferTypeConverter`. Widening service insertion while
that derivation was wrong would have spread the false positive across every
module, so it is fixed here. The manifest feeds only the collision pass, so
no emitted name changes.

`generated-file-layout.md` already claimed cross-file type collisions were
validated in Go, and `services.md` already put service bindings in the
model namespace; both were ahead of the implementation. Their text now
distinguishes the module scope from Go's package scope, and PRINCIPLES §15
defines what a scope is per target.

No generated sample moves.
TypeScript and Python had the same cross-module defect Go did, by a
different route. Both emit a root barrel that lifts every module's
top-level names into a single namespace — `index.ts` re-exports each
module with `export *`, `__init__.py` re-exports them by name — so two
input files each declaring a `Page` collide there, even though the two
modules are separate scopes on their own.

Left to the target, TypeScript emitted a barrel its own compiler rejects
(TS2308, "Module './a' has already exported a member named 'Page'"), and
the model's `pageTransferTypeConverter` collided alongside the interface.
Python was worse: `from .a import Page` followed by `from .b import Page`
raises nothing, binds the second, and lists the name once in `__all__`, so
`from pkg import Page` quietly resolved to the wrong model — the silent
incorrectness P7 exists to prevent. Both now reject at load with the same
module-qualified diagnostic Go gives.

Java and .NET keep per-module scoping. Each module lands in its own
sub-package/namespace (`com.example.api.a.page`, `Nexgen.Generated.A.Page`)
and neither emits an aggregating barrel, so two `Page` types stay distinct;
this was checked against both generators, not assumed from the layout.

The scope is now chosen by a documented predicate over the emitted layout
rather than a Go special case, since the reason differs per target: Go
flattens to one package, TS and Python re-aggregate through a barrel.

Three specs asserted the superseded rule and are corrected: PRINCIPLES §15,
generated-file-layout.md, and ref.md, which had claimed one package-wide
namespace for every target.
A module re-emitted the types it only `$ref`d from another input file,
instead of importing them, whenever that module declared no types of its
own — a service file whose every operation type is a `$ref` elsewhere.

Every target produced a second copy. Go put two `type Page struct` in its
flat package (`Page redeclared in this block`, from the Go compiler);
Java wrote `Page.java` twice and failed the build with a generated-file
conflict; TypeScript emitted a module that both imports and redeclares the
name (TS2440) and a barrel that re-exports it twice (TS2308); Python
emitted two classes and a barrel importing both, silently binding one and
dropping the other off the package surface, which is the P7 case.

The cause was in reachability pruning, which decides whether to drop
declarations another module owns. It inferred "this front end does not
scope declarations by module" from the module owning nothing — a state
indistinguishable from "this module declares nothing" while the flag is a
bool, so a service-only module looked like a WIT spec and kept everything
it could reach. A declaration now records `Foreign` distinctly from
`Unscoped`, which is the distinction the prune was actually reaching for.

Dropping the re-emitted copies leaves a module that declares nothing with
an empty models file, so TypeScript no longer emits one, and its barrel no
longer re-exports `./models`: re-exporting a file with no exports is itself
an error (TS2306).

The checked-in samples do not change, because every sample service module
happens to declare at least one inline operation type — which is why this
went unnoticed. Each new test was confirmed to fail without the fix, and
the repaired output was checked with the Go compiler, javac, tsc, and a
Python import rather than by inspection.
@bergundy
bergundy force-pushed the ts-transfer-type-encoding branch from ee984d2 to 7c5984c Compare August 21, 2026 21:06
@bergundy
bergundy merged commit 8500376 into main Aug 21, 2026
4 checks passed
@bergundy
bergundy deleted the ts-transfer-type-encoding branch August 21, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants