Proposal: Registering a "node" URI scheme

[SMotaal]

I wanted to propose registering a "provisional" URI scheme "node" which reflects a custom resolution extension based on the transport scheme. I don't think it is time to lock in any particular specifications aside from the fact that it depends upon known transport protocols like 'file' and is intended to provide practical URI mappings and improved interoperability with other applications like web browsers.

Let's try to focus this thread on the minimal provisional RFC and as I am sure many have thought about this and want to get into details, I recommend creating a parallel issue mentioning this issue so we can all converge.

[ljharb]

This seems like it would conflict a bit with the idea that many believe that node shouldn't by default add support for importing from URLs. What's the use case you have in mind?

[SMotaal]

I am not sure that there is a concrete use case in mind, aside from the fact that I find my self (and surely others) often relying on the fact that module specifiers are simply derivations for URLs.

As far as URI Schemes go, they are obviously well defined for browser contexts in the form of HTTP* WS* *FTP and FILE, ie the transport & resolution schemes, but start getting a little messy for other contexts (like GIT... etc.)

For node, one can almost say that node specifiers are node+file: because they adopt file:'s transport and relative resolution behaviours but also introduce node: resolutions behaviours that relate to bare specifiers (and maybe scopes) that take precedence over file.

Since such resolutions only take place when modules are being resolved by node, this node+file: is omitted since the entry point is node+file: and all modules that are resolved to a file: URL are also within the node+file: scheme, whereas some modules like "modules" or "path" which are also resolved by node: but do not rely on any transport protocol (like for instance blob:) as such modules are aliases to resources that are simply referenced internally.

Minor Edits

[devsnek]

i wouldn't imagine node: going to anywhere except maybe the builtin modules (which the current loader actually already does internally)

[demurgos]

I am working on implementing code coverage for ESM using V8's native profiler. One of the issues is to differentiate user's scripts from internal ones. At the moment, user scripts have absolute paths (file:// for ESM, simple FS path for CJS) and internal scripts have relative paths. Using node:// for internal scripts would be a stronger signal that the scripts cannot be just read using fs.readFile.

Note: V8 reports the effective URL of the script, after all resolution was performed so it's either immediately a path to a file or an internal module.

[SMotaal]

@devsnek 👍 this is most likely the seed that inspires this thread. At times when I am experimenting with NW.js but more so with Electron (since it has a more open schemes API) it becomes very easy to see how one needs to assume node resources are always 'node:' if only implicitly and outside the scope of the actual resources themselves.

[SMotaal]

@demurgos that's probably the URL parameter that is passed to V8 via the ModuleWrap constructor

[demurgos]

@SMotaal
Exactly. I've looked into the related source code and relative/abs path may be enough to differentiate them but I am not sure if it's something that can be relied for long term (I did not find it documented).

[hybrist]

Would a protocol like commonjs:// be more clear? The disadvantage of a protocol solution (last time I tried it) was that it becomes weird for non-absolute things. commonjs://./foo.node looks really weird and isn't quite how "normal" URLs would work.

[SMotaal]

Let's get into the details over at #169 (Discussion about an implicit (or explicit) node scheme)

[ljharb]

You can already use isCore from https://npmjs.com/resolve, for example, to determine if it's a builtin module or not - the implementation would be able to be simpler if the specifier contained that info (like, for example, if it was under a scope) but it's basically a solved problem.

[SMotaal]

@ljharb I'm thinking less about implementation, or at the very least, assuming NPM packages are not an accessible luxury, from the premise that a loader that circularly depends on what it loads (unless ESM will always depend on CJS) is paradoxical at best.

[ljharb]

A loader might always have to depend on what’s loadable in its absence, otherwise it wouldn’t be able to have any external modularity. If this is running in node, installed npm packages will always be accessible because require/import will be, of course but never necessary if installation is unavailable - just like now.