[NS 1] Add sub-module mounts support + codegen by aasoni · Pull Request #5167 · clockworklabs/SpacetimeDB

aasoni · 2026-06-02T08:48:05Z

Description of Changes

Add a new recursive "mounts" field to add submodules to module def. Handle code generation for each language.
Handle identifiers with "/" or "." in the name to handle namespaced reducers (e.g. lib/reducer) and namespaced tables (lib.table).

Handle code generation for the recursive type. This needed some special handling in code generation for typescript and c++.
Typescript codegen in particular is quite complex as it tries to handle circular dependency generically. C++ on the other hand is a lot simpler because it hard-codes a special handling of the V10 definition but doesn't solve circular dependencies in general.
I would advice against solving circular dependencies in a generic way for C++ however we could consider modifying the typescript code gen to just have special handling for the V10 recursive definition which would simplify the code quite a lot. I went down the rabbit hole of handling this generically and came out on the other side, but if there is strong opinion to keep the codegen code simple, I am happy to revisit and align to the C++ way.

API and ABI breaking changes

The change is purely additive and newer host versions will accept older module defs. However older host versions will not accept new module defs.

Expected complexity level and risk

5 - While this specific PR is maybe a 4, the overall namespace change is definitely 5. This is a pretty significant change. It's a large diff which touches the module def and changes code that hasn't been touched in a long time (e.g. Identifier).

Testing

Beyond the rust tests defined in this PR, the following tests were done on the full PR sequence once the entire namespace feature was implemented for typescript:
Feature Test Checklist

Module:

root module can import another module and mount it with a namespace under its schema
root module and submodule can have the same function and table names without conflicting
ctx.db.lib.lib_table is readable/writable inside root module reducer
ctx.db.lib.lib_table is readable/writable inside root module procedure withTx block
library_reducer(ctx.as.lib) is callable inside root module reducer

Client

Client can subscribe to lib.library_table
Client can subscribe to lib.library_view
Client can call lib/library_reducer
Client can call lib/library_procedure
Client can subscribe to lib.sublib.sublib_table
Client can subscribe to lib.sublib.sublib_view
Client can call lib/sublib/sublib_reducer
Client can call lib/sublib/sublib_procedure

CLI

CLI can subscribe to lib.library_table
CLI can subscribe to lib.library_view
CLI can call lib/library_reducer
CLI can call lib/library_procedure
CLI can subscribe to lib.sublib.sublib_table
CLI can subscribe to lib.sublib.sublib_view
CLI can call lib/sublib/sublib_reducer
CLI can call lib/sublib/sublib_procedure

Migration

Module migrates without issue from having a submodule to not having a submodule
Module migrates without issue from not having a submodule to having a submodule
Module migrates without issue when having a submodule and root module change occurs (change reducer signature, add table, add column with default, change reducer function body, change index)
Module migrates without issue when having a submodule and a submodule change occurs (change reducer signature, add table, add column with default, change reducer function body, change index)

Commit Log

Module loads fine from commit log
Module snapshot is created without issue
Module loads fine from snapshot

Co-authored-by: Jason Larabie <jason@clockworklabs.io> Signed-off-by: Alessandro Asoni <alessandro@clockworklabs.io>

gefjon

I'm really unhappy with the codegen changes in both TypeScript and C++ here. The TypeScript changes seem very complicated and finnickey, and the C++ changes are an obviously brittle special case. I don't understand why we don't just move to generating a single file for the whole module, if generating individual files and importing across them is causing so many problems.

gefjon · 2026-06-04T17:43:25Z

+        r#"// THIS FILE IS AUTOMATICALLY GENERATED BY SPACETIMEDB. EDITS TO THIS FILE
+// WILL NOT BE SAVED. MODIFY TABLES IN YOUR MODULE SOURCE CODE INSTEAD.
+
+// This was generated using spacetimedb codegen.


This line seems redundant.

gefjon · 2026-06-04T17:45:50Z

+        // (1) its `namespace` field is a C++ keyword; (2) its `module` field creates a
+        // circular include chain through RawModuleDefV10 → RawModuleDefV10Section.
+        // We break both with a forward declaration and shared_ptr.
+        if name.to_string() == "RawModuleMountV10" {


This feels brittle and gross. I'm really not a fan. For one thing, we're assigning special semantics to the name RawModuleMountV10 but not doing any work to reserve that name or ensure users don't provide it. For another thing, everything else about this.

gefjon · 2026-06-04T17:46:54Z

+
+SPACETIMEDB_INTERNAL_PRODUCT_TYPE(RawModuleMountV10) {
+    std::string namespace_;                                          // renamed: 'namespace' is a C++ keyword
+    std::shared_ptr<SpacetimeDB::Internal::RawModuleDefV10> module; // shared_ptr breaks infinite-size recursion


How is it possible that the type definition works in Rust, but you have to manually insert this shared_ptr in the C++ definition?

The Rust definition has the RawModuleMountV10 in a Vec, which I assume we'd generate as std::vector, so additional indirection shouldn't be necessary.

gefjon · 2026-06-04T17:53:31Z

+/// BFS to find all type refs reachable from `start` in the type-dependency graph.
+fn reachable_from(
+    typespace: &spacetimedb_schema::type_for_generate::TypespaceForGenerate,
+    start: AlgebraicTypeRef,
+) -> BTreeSet<AlgebraicTypeRef> {
+    let mut visited = BTreeSet::new();
+    let mut stack = vec![start];
+    while let Some(r) = stack.pop() {
+        if !visited.insert(r) {
+            continue;
+        }
+        if let Some(def) = typespace.get(r) {
+            for neighbor in direct_refs_of_def(def) {
+                stack.push(neighbor);
+            }
+        }
+    }
+    visited
+}
+
+/// Get all strongly connected components within the provided ModuleDef types.
+/// Used to compute circular dependencies within the provided ModuleDef.
+fn algebraic_type_scc(module: &ModuleDef) -> BTreeSet<AlgebraicTypeRef> {
+    let Some(at_ref) = iter_types(module)
+        .find(|ty| type_ref_name(module, ty.ty) == "AlgebraicType")
+        .map(|ty| ty.ty)
+    else {
+        return BTreeSet::new();
+    };
+
+    let typespace = module.typespace_for_generate();
+    let from_at = reachable_from(typespace, at_ref);
+    from_at
+        .iter()
+        .filter(|&&r| reachable_from(typespace, r).contains(&at_ref))
+        .copied()
+        .collect()
+}


If we're going to have to compute SCCs to do codegen (which I really hope we don't), then I think we should at least be doing it in the schema crate in a way that's well-tested and reused across all our codegen languages, rather than ad-hoc here. And we should be doing Tarjan's, Kosaraju the third one (I forget what it's called, path-based or something) once for the module, rather than running it for every type.

gefjon · 2026-06-04T17:53:48Z

+/// Converts an `AlgebraicTypeUse` to a TypeScript type expression for use in explicit type aliases.
+/// Used when generating `export type Foo = { ... }` for recursive types.


Why do recursive types require generating type aliases? Once you have the SCCs, can't you just put all of the defs from each component together?

gefjon · 2026-06-04T18:02:37Z

+/// Check that no two modules in the mount tree claim the same lifecycle reducer.
+///
+/// The host assigns exactly one reducer per lifecycle slot; if both the consumer
+/// and a mounted submodule (or two sibling mounts) declare `__init__` (etc.), the
+/// module must be rejected at publish time.


I'm surprised that mounts are allowed to claim lifecycle reducers at all, if we're not going to allow multiple hooks on the same lifecycle event.

gefjon · 2026-06-04T18:03:10Z

+    if mount.namespace.len() > 63 {
+        return Err(ValidationErrors::from(ValidationError::NamespaceTooLong {


Why are we limiting the length of namespace names?

it was part of the proposal

gefjon · 2026-06-04T18:03:45Z

    })
 }

+fn validate_mount(mount: RawModuleMountV10) -> Result<(String, ModuleDef)> {


This could use a doc comment describing what about the mount it validates.

gefjon · 2026-06-04T18:05:25Z

+        }));
+    }
+
+    Ok((mount.namespace, validate(mount.module)?))


We should combine errors from validating the mounted module with the above two checks.

gefjon · 2026-06-04T18:13:33Z

+        // Flatten mounted modules into the root table/reducer/procedure lists and typespace.
+        // Each mount's typespace is appended to the merged types with all AlgebraicTypeRef
+        // indices shifted by the current length, keeping internal references valid.
+        let root_anon_view_count = views.values().filter(|v| v.is_anonymous).count() as u32;
+        let root_non_anon_view_count = views.values().filter(|v| !v.is_anonymous).count() as u32;
+        let mut flat_tables: Vec<RawTableDefV9> = to_raw(tables);
+        let mut flat_reducers: Vec<RawReducerDefV9> = reducers.into_iter().map(|(_, def)| def.into()).collect();
+        let mut flat_misc: Vec<RawMiscModuleExportV9> = column_defaults
+            .into_iter()
+            .chain(procedures.into_iter().map(|(_, def)| def.into()))
+            .chain(views.into_iter().map(|(_, def)| def.into()))
+            .collect();
+        let mut merged_types = typespace.types;
+        let mut anon_view_offset = root_anon_view_count;
+        let mut view_offset = root_non_anon_view_count;
+        collect_v9_mounts(
+            &mounts,
+            "",
+            &mut flat_tables,
+            &mut flat_reducers,
+            &mut flat_misc,
+            &mut merged_types,
+            &mut anon_view_offset,
+            &mut view_offset,
+        );


This shouldn't be necessary. Notice that, for all other features that post-date the V9 module def format, this method just ignores them. I don't see why mounts would be any different.

yeah true, I can remove this

aasoni changed the base branch from master to alessandro/namespace-base-branch June 2, 2026 08:51

aasoni changed the title ~~Add sub-module mounts support + codegen~~ [NS 1] Add sub-module mounts support + codegen Jun 2, 2026

aasoni requested review from coolreader18 and gefjon June 2, 2026 08:51

aasoni assigned JasonAtClockwork and unassigned JasonAtClockwork Jun 2, 2026

aasoni requested a review from JasonAtClockwork June 2, 2026 11:35

Add sub-module mounts support + codegen

c8cac1d

aasoni force-pushed the alessandro/sub-module-mounts branch from a5dfaf5 to c8cac1d Compare June 3, 2026 08:00

JasonAtClockwork reviewed Jun 3, 2026

View reviewed changes

Comment thread crates/codegen/src/cpp.rs

Update crates/codegen/src/cpp.rs

f9b867e

Co-authored-by: Jason Larabie <jason@clockworklabs.io> Signed-off-by: Alessandro Asoni <alessandro@clockworklabs.io>

gefjon reviewed Jun 4, 2026

View reviewed changes

		/// Converts an `AlgebraicTypeUse` to a TypeScript type expression for use in explicit type aliases.
		/// Used when generating `export type Foo = { ... }` for recursive types.

		if mount.namespace.len() > 63 {
		return Err(ValidationErrors::from(ValidationError::NamespaceTooLong {

Conversation

aasoni commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of Changes

API and ABI breaking changes

Expected complexity level and risk

Testing

Uh oh!

Uh oh!

gefjon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

aasoni commented Jun 2, 2026 •

edited

Loading