Enable multiple Node-API hosting JS engines / runtimes to share the global Node-API functions #329

kraenhansen · 2025-11-10T21:48:26Z

This is my suggestion for adding "multi-host" support to weak-node-api, enabling multiple engines implementing Node-API to co-exist and share the Node-API function namespace. While not needed specifically for bringing Node-API to React Native adding this could make weak-node-api more applicable in other scenarios where multiple engines implementing Node-API share a single process.

I'm proposing adding mechanisms to "wrap" the opaque pointers of specific Node-API implementors with references to the NodeApiHost object to enable deferring implementation of Node-API functions based on the napi_env, node_api_basic_env, napi_threadsafe_function or napi_async_cleanup_hook_handle passed.

classDiagram
    class NodeApiHost {
        <<interface>>
    }

    class NodeApiMultiHost {
        -vector~unique_ptr~WrappedEnv~~ envs
        +wrap(napi_env, weak_ptr~NodeApiHost~) napi_env
        +static napi_* methods...
    }

    class WrappedEnv {
        +napi_env value
        +weak_ptr~NodeApiHost~ host
        -vector~unique_ptr~WrappedThreadsafeFunction~~ threadsafe_functions
        -vector~unique_ptr~WrappedAsyncCleanupHookHandle~~ async_cleanup_hook_handles
        +wrap(napi_threadsafe_function, WrappedEnv*, weak_ptr~NodeApiHost~) napi_threadsafe_function
        +wrap(napi_async_cleanup_hook_handle, WrappedEnv*, weak_ptr~NodeApiHost~) napi_async_cleanup_hook_handle
    }

    class WrappedThreadsafeFunction {
        +napi_threadsafe_function value
        +WrappedEnv* env
        +weak_ptr~NodeApiHost~ host
    }

    class WrappedAsyncCleanupHookHandle {
        +napi_async_cleanup_hook_handle value
        +WrappedEnv* env
        +weak_ptr~NodeApiHost~ host
    }

    NodeApiMultiHost --|> NodeApiHost : inherits
    NodeApiMultiHost *-- "0..*" WrappedEnv : owns
    WrappedEnv *-- "0..*" WrappedThreadsafeFunction : owns
    WrappedEnv *-- "0..*" WrappedAsyncCleanupHookHandle : owns
    WrappedThreadsafeFunction --> WrappedEnv : references
    WrappedAsyncCleanupHookHandle --> WrappedEnv : references
    WrappedEnv --> NodeApiHost : weak reference
    WrappedThreadsafeFunction --> NodeApiHost : weak reference
    WrappedAsyncCleanupHookHandle --> NodeApiHost : weak reference
    WrappedEnv ..|> enable_shared_from_this : implements

    note for NodeApiMultiHost "Manages multiple Node-API host implementations"
    note for WrappedEnv "Wraps napi_env with ownership tracking for threadsafe functions and async cleanup hook handles"

WrappedEnv, WrappedThreadsafeFunction and WrappedAsyncCleanupHookHandle objects can then be passed around like their respective opaque pointers and are "unwrapped" in the internal implementation of the "multi host" implementation of Node-API functions.

Wrapped objects are created calling one of the wrap instance methods on NodeApiMultiHost or WrappedEnv, which are called internally in napi_create_threadsafe_function and napi_add_async_cleanup_hook too.

Usage

Create a NodeApiMultiHost (providing functions to register modules and handling a fatal error)
Inject it as the global host
Get an env from the actual Node-API implementatin
Wrap the env with using the NodeApiMultiHost (calling multi_host.wrap(original_env, host);)
Pass the wrapped env to Node-API functions to delegate as needed

react-native-node-api/packages/weak-node-api/tests/test_multi_host.cpp

Lines 18 to 52 in f79a30b

    
           static size_t foo_calls = 0; 
        
           auto host_foo = std::shared_ptr<WeakNodeApiHost>(new WeakNodeApiHost{ 
        
               .napi_create_object = [](napi_env env, 
        
                                        napi_value *result) -> napi_status { 
        
                 foo_calls++; 
        
                 return napi_status::napi_ok; 
        
               }}); 
        
           static size_t bar_calls = 0; 
        
           auto host_bar = std::shared_ptr<WeakNodeApiHost>(new WeakNodeApiHost{ 
        
               .napi_create_object = [](napi_env env, 
        
                                        napi_value *result) -> napi_status { 
        
                 bar_calls++; 
        
                 return napi_status::napi_ok; 
        
               }}); 
        
           // Create and inject a multi host and wrap two envs 
        
           WeakNodeApiMultiHost multi_host{nullptr, nullptr}; 
        
           inject_weak_node_api_host(multi_host); 
        
           auto foo_env = multi_host.wrap(napi_env{}, host_foo); 
        
           auto bar_env = multi_host.wrap(napi_env{}, host_bar); 
        
           napi_value result; 
        
           REQUIRE(foo_calls == 0); 
        
           REQUIRE(bar_calls == 0); 
        
           REQUIRE(napi_create_object(foo_env, &result) == napi_ok); 
        
           REQUIRE(foo_calls == 1); 
        
           REQUIRE(bar_calls == 0); 
        
           REQUIRE(napi_create_object(bar_env, &result) == napi_ok); 
        
           REQUIRE(foo_calls == 1); 
        
           REQUIRE(bar_calls == 1);

TODO

Provide a way to "delete" a WrappedEnv (besides deleting the entire NodeApiMultiHost)
Delete WrappedThreadsafeFunction (in some napi_*_threadsafe_function function?) and WrappedAsyncCleanupHookHandle (in napi_remove_async_cleanup_hook).

Open questions

Should we use std::function instead of raw function pointers for all (or some) of the WeakNodeApiHost members? This would allow capturing lambdas, making it much easier to provide a meaningful implementation of for example napi_module_register.

Generated code

Below are samples from the generated code:

`napi_create_object` implementation

napi_status NodeApiMultiHost::napi_create_object(napi_env arg0,
                                                 napi_value *arg1) {
  auto wrapped = reinterpret_cast<WrappedEnv *>(arg0);
  if (auto host = wrapped->host.lock()) {
    if (host->napi_create_object == nullptr) {
      fprintf(stderr, "Node-API function 'napi_create_object' called on a host "
                      "which doesn't provide an implementation\n");
      return napi_status::napi_generic_failure;
    }

    return host->napi_create_object(wrapped->value, arg1);

  } else {
    fprintf(stderr, "Node-API function 'napi_create_object' called after host "
                    "was destroyed.\n");
    return napi_status::napi_generic_failure;
  }
};

`napi_create_threadsafe_function` and `napi_add_async_cleanup_hook` implementations

Notice the calls to wrap, wrapping their opaque "out" pointers.

napi_status NodeApiMultiHost::napi_create_threadsafe_function(
    napi_env env, napi_value func, napi_value async_resource,
    napi_value async_resource_name, size_t max_queue_size,
    size_t initial_thread_count, void *thread_finalize_data,
    napi_finalize thread_finalize_cb, void *context,
    napi_threadsafe_function_call_js call_js_cb,
    napi_threadsafe_function *result) {
  auto wrapped = reinterpret_cast<WrappedEnv *>(env);
  if (auto host = wrapped->host.lock()) {
    if (host->napi_create_threadsafe_function == nullptr) {
      fprintf(stderr,
              "Node-API function 'napi_create_threadsafe_function' called on a "
              "host which doesn't provide an implementation\n");
      return napi_status::napi_generic_failure;
    }

    auto status = host->napi_create_threadsafe_function(
        wrapped->value, func, async_resource, async_resource_name,
        max_queue_size, initial_thread_count, thread_finalize_data,
        thread_finalize_cb, context, call_js_cb, result);
    if (status == napi_status::napi_ok) {
      *result = wrapped->wrap(*result, wrapped, wrapped->host);
    }
    return status;

  } else {
    fprintf(stderr, "Node-API function 'napi_create_threadsafe_function' "
                    "called after host was destroyed.\n");
    return napi_status::napi_generic_failure;
  }
};

napi_status NodeApiMultiHost::napi_add_async_cleanup_hook(
    node_api_basic_env env, napi_async_cleanup_hook hook, void *arg,
    napi_async_cleanup_hook_handle *remove_handle) {
  auto wrapped = reinterpret_cast<WrappedEnv *>(env);
  if (auto host = wrapped->host.lock()) {
    if (host->napi_add_async_cleanup_hook == nullptr) {
      fprintf(stderr, "Node-API function 'napi_add_async_cleanup_hook' called "
                      "on a host which doesn't provide an implementation\n");
      return napi_status::napi_generic_failure;
    }

    auto status = host->napi_add_async_cleanup_hook(wrapped->value, hook, arg,
                                                    remove_handle);
    if (status == napi_status::napi_ok) {
      *remove_handle = wrapped->wrap(*remove_handle, wrapped, wrapped->host);
    }
    return status;

  } else {
    fprintf(stderr, "Node-API function 'napi_add_async_cleanup_hook' called "
                    "after host was destroyed.\n");
    return napi_status::napi_generic_failure;
  }
};

RobinWuu · 2025-11-11T14:05:20Z

Wow, I really need this capability. And I like the "wrap" mechanisms; they're very elegant for this scenario.

Should we use std::function instead of raw function pointers for all (or some) of the WeakNodeApiHost members? This would allow capturing lambdas, making it much easier to provide a meaningful implementation of for example napi_module_register.

From my perspective, using std::function for napi_module_register indeed provides greater flexibility. If napi_module_register is a std::function, it can enable module isolation between multiple Node-API hosting JS engines/runtimes within the same process. Would this be meaningful in certain scenarios?

I suggest the Wrapped objects be owned by the WeakNodeApiMultiHost object (at least for now), but could we make memory management more efficient to release the Wrapped before the WeakNodeApiMultiHost deletion: In napi_remove_async_cleanup_hook or some of the napi_*_threadsafe_function functions?

I agree that releasing the corresponding Wrapped<T> in napi_remove_async_cleanup_hook or napi_release_threadsafe_function would be appropriate and feasible.

Copilot

Pull Request Overview

This PR adds multi-host support to weak-node-api, enabling multiple Node-API implementations to coexist in a single process. The implementation wraps opaque Node-API pointers (napi_env, napi_threadsafe_function, napi_async_cleanup_hook_handle) with metadata tracking which host owns them, allowing proper delegation of API calls to the correct implementation.

Key changes:

Introduces WeakNodeApiMultiHost class with wrapper mechanism for opaque Node-API types
Generates multi-host header and source files alongside existing weak-node-api files
Adds comprehensive test coverage for multi-host scenarios including host lifecycle and wrapped pointer handling

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`packages/weak-node-api/tests/test_multi_host.cpp`	Comprehensive test suite validating multi-host injection, call routing, host lifecycle, and wrapped opaque pointer handling
`packages/weak-node-api/tests/CMakeLists.txt`	Adds new multi-host test file to the build configuration
`packages/weak-node-api/scripts/generators/multi-host.ts`	Code generator for multi-host header and implementation, including wrapper creation and call delegation logic
`packages/weak-node-api/scripts/generate-weak-node-api.ts`	Updates generator to produce multi-host files with documentation headers
`packages/weak-node-api/CMakeLists.txt`	Includes generated multi-host source and header files in the build
`packages/weak-node-api/.gitignore`	Simplifies ignore pattern to cover all generated files

packages/weak-node-api/scripts/generators/multi-host.ts

packages/weak-node-api/scripts/generate.ts

kraenhansen self-assigned this Nov 10, 2025

kraenhansen added the weak-node-api label Nov 10, 2025

kraenhansen force-pushed the kh/weak-node-api-multi-host branch from 6f9356d to f79a30b Compare November 10, 2025 21:53

kraenhansen marked this pull request as ready for review November 11, 2025 15:18

kraenhansen requested a review from Copilot November 11, 2025 15:18

Copilot AI reviewed Nov 11, 2025

View reviewed changes

packages/weak-node-api/scripts/generators/multi-host.ts Outdated Show resolved Hide resolved

packages/weak-node-api/scripts/generate.ts Show resolved Hide resolved

kraenhansen mentioned this pull request Nov 13, 2025

More refactoring of the weak-node-api generator #330

Open

kraenhansen force-pushed the kh/weak-node-api-multi-host branch from f79a30b to 4d78b71 Compare November 13, 2025 10:32

kraenhansen changed the base branch from main to kh/weak-node-api-refactored-generator-3 November 13, 2025 10:32

kraenhansen force-pushed the kh/weak-node-api-multi-host branch from 4d78b71 to 5ff0a5e Compare November 13, 2025 11:20

kraenhansen added 2 commits November 13, 2025 12:23

Remove incorrect comments

cf5e056

Implement WeakNodeApiMultiHost generator

daf6686

kraenhansen force-pushed the kh/weak-node-api-multi-host branch from 5ff0a5e to daf6686 Compare November 13, 2025 11:24

kraenhansen force-pushed the kh/weak-node-api-refactored-generator-3 branch from cf5e056 to bd952a6 Compare November 13, 2025 13:58

kraenhansen added Apple 🍎 Anything related to the Apple platform (iOS, macOS, Cocoapods, Xcode, XCFrameworks, etc.) Android 🤖 Anything related to the Android platform (Gradle, NDK, Android SDK) labels Nov 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable multiple Node-API hosting JS engines / runtimes to share the global Node-API functions #329

Enable multiple Node-API hosting JS engines / runtimes to share the global Node-API functions #329

Uh oh!

kraenhansen commented Nov 10, 2025 •

edited

Loading

Uh oh!

RobinWuu commented Nov 11, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	static size_t foo_calls = 0;
	auto host_foo = std::shared_ptr<WeakNodeApiHost>(new WeakNodeApiHost{
	.napi_create_object = [](napi_env env,
	napi_value *result) -> napi_status {
	foo_calls++;
	return napi_status::napi_ok;
	}});

	static size_t bar_calls = 0;
	auto host_bar = std::shared_ptr<WeakNodeApiHost>(new WeakNodeApiHost{
	.napi_create_object = [](napi_env env,
	napi_value *result) -> napi_status {
	bar_calls++;
	return napi_status::napi_ok;
	}});

	// Create and inject a multi host and wrap two envs
	WeakNodeApiMultiHost multi_host{nullptr, nullptr};
	inject_weak_node_api_host(multi_host);

	auto foo_env = multi_host.wrap(napi_env{}, host_foo);
	auto bar_env = multi_host.wrap(napi_env{}, host_bar);

	napi_value result;

	REQUIRE(foo_calls == 0);
	REQUIRE(bar_calls == 0);

	REQUIRE(napi_create_object(foo_env, &result) == napi_ok);
	REQUIRE(foo_calls == 1);
	REQUIRE(bar_calls == 0);

	REQUIRE(napi_create_object(bar_env, &result) == napi_ok);
	REQUIRE(foo_calls == 1);
	REQUIRE(bar_calls == 1);

Enable multiple Node-API hosting JS engines / runtimes to share the global Node-API functions #329

Are you sure you want to change the base?

Enable multiple Node-API hosting JS engines / runtimes to share the global Node-API functions #329

Uh oh!

Conversation

kraenhansen commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Usage

TODO

Open questions

Generated code

napi_create_object implementation

napi_create_threadsafe_function and napi_add_async_cleanup_hook implementations

Uh oh!

RobinWuu commented Nov 11, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kraenhansen commented Nov 10, 2025 •

edited

Loading

`napi_create_object` implementation

`napi_create_threadsafe_function` and `napi_add_async_cleanup_hook` implementations