Strategic initiative to move to cmake for building Node.js?

[mhdawson]

@bnoordhuis had done some experimentation to move Node.js to use cmake but I think we need more consensus/momentum to be able to make a move like that.

I'd like to restart discussion in the context of: nodejs/node-addon-api#422. Before we start to experiment with building addons with a different tool like cmake, it would be good to better understand if it might align with what we using in Node.js in the future.

[ofrobots]

This seems related to #642.

For some background there is some related discussion in #464.

To keep the discussion clear and focussed, I think this is about how Node.js-core should be built rather than ecosystem native addons, right? The constraints are quite different between the two problem spaces, so let clarify that this is not about native addons.

Before we can have a strategic initiative for building with cmake, I think we will need to explicitly agree that cmake is the build system we are going to use.

At the core, the choice of build tool is about modularity. C++, unlike Go, rust, and JavaScript, does not have a standard "assemble-dependencies-and-build" system. Whatever choice of build system we have will have to deal with the fact that we depend on some fairly large projects (e.g. V8) and that we have to make those projects buildable with whatever build tool we chose for building Node.js itself. Most of our other C/C++ dependencies are simple enough that the main challenge we need to solve is how our build tool choice will affect how we incorporate V8. It is not a matter of migrating V8 to cmake once. We have to maintain this build system, and V8 notoriously moves very fast.

For diagnostics we (I?) want to pick up the Perfetto project as a dependency to replace the Trace Events infrastructure that we are maintaining in V8 and Node. This and other future dependencies, and how we consume them, will be affected by our choice of build tool.

I am not necessary against cmake, I just haven't seen consensus around it yet.

[gabrielschulhof]

FWIW there's some momentum in nodejs/node-addon-api#422 to move to cmake. Do we need to agree to move the whole project to CMake before we deploy cmake on our CI? I'd like to ask in @nodejs/build that CMake be added to the tools available on the CI (if not already there).

[mhdawson]

@ofrobots that is exactly the discussion I wanted to get re-started "how Node.js-core should be built. @bnoordhuis for some reason I was thinking that you might have been building everything except v8 with cmake in your work, but I may not be remembering correctly.

[mhdawson]

@gabrielschulhof I mentioned nodejs/node-addon-api#422 above as I wanted to see if there was any consensus on what we should do for Node.js core as that would make the choice for add-ons easier. I don't think it's a prereq but I think its worth exploring as it would be good to avoid having too many different build systems required.

[ofrobots]

As a point of comparison, from another notable V8 embedder, deno seems to be using GN to build.

[mhdawson]

I think when we asked the Google team last time they specifically said GN was not "supported" for use in a case like ours and recommended other options.

[ofrobots]

Adding @nodejs/v8 for clarity about GN and level of support. It is ultimately the V8 team's choice to use GN that puts this cost on their embedders like us.

[mhdawson]

@ofrobots thanks, its good to re-confirm on that front. Things may even have changed over time.

[mhdawson]

While not the same topic I think it would be good to add to the discussion whether the build system for Node.js core and add-ons should be the same? I wanted to restart the discussion in core to have as good as data as possible for nodejs/node-addon-api#422 but if the concerns can be/should be separated it's good to figure that out as well.

[joyeecheung]

It is possible to build v8 with gn but use cmake for the rest of the project? V8’s fork has been using the configure option —build-v8-with-gn to trigger an action and build v8 directly with gn, if we can somehow host gn in-tree and bootstrap by compiling it with cmake (if gn is not available in PATH) then we won’t take on the burden of porting v8 build files to cmake?

[Trott]

As a point of comparison, from another notable V8 embedder, deno seems to be using GN to build.

Same for electron. (Not saying we should use GN. Just another data point.)

[ryzokuken]

I had been compiling a list of pros/cons regarding build systems in our case, but had been reluctant to spark off this discussion myself since there's a lot of conflicting opinions.

Some objective truths that are important to consider IMO:

1. If we want to build using GN, we will not only have make all contributors install depot_tools, we'd also have to use them for various other purposes including fetching dependencies (which is something electron does as well). Important things to consider: another step for new and returning contributors as well as atleast a minor loss in productivity while everyone gets used to the new workflow.

2. Right now, @targos and I take turns at updating our gypfiles for building V8 whenever canary-base is broken, which on bad days is nearly daily. While CMake is great in the way that we do not have to maintain it by ourselves (Refael has taken up the burden to maintain gyp pretty much all by himself recently, but I wouldn't mention him since he's staying away from the project right now) we will have to do the same for CMake, porting the changes in the latest GN configs to the CMake format.

Pros of CMake/Cons of gyp:

1. It's easily one of the most widely used build systems out there. Since there's a ton of resources, documentation and knowledge regarding CMake out there in the open, we'd get rid of the current situation where apart from a small group of collaborators, nobody really understands how gyp really works etc.

2. It's well maintained, and not going anywhere anytime soon.

Pros of gyp/Cons of CMake:

1. There's a small upfront cost in porting all our gyp configs to CMake, but that should probably be the least of our worries.

2. CMake is completely unsupported by the V8 team, so it would be futile to expect for a lot.

3. Nobody in the ecosystem (either upstream at V8, nor downstream at Electron, for example) really uses CMake. We make everyone else's lives hard (including people who help embed Node.js into Electron, /cc @codebytere).

Game changer: Over at Electron, the exact opposite is required to be done (porting our changes in gyp to GN), and they've hacked up a clever script that automatically does most of the work, if not all for them. I've wanted to write a similar script for us vs V8 for quite some time, which should take care of ~70-80% of minor changes to V8's build. If we do manage to cover that many changes, it would be a serious pro in favor of gyp.

(Note: A similar script could also be written down for CMake, tbh. The only difference is that we would not have much to refer to.)

If the official support (neither of the above two are, really) seems to be a major issue, we have two officially supported alternatives (these would cut down the amount of work we need to put in on a near daily basis): GN and Bazel.

Last I discussed this with @hashseed, the Chromium team was not very happy with any outside project using GN, but that might have changed: https://gn.googlesource.com/gn/.

They seemed to be (atleast at that time) more interested in the idea of us using Bazel since Google uses Blaze internally anyway and it was likely that they could share the currently internal build files with us, making our lives easier.

Ofcourse, the GN build files are already public so the sky's the limit for that one.

EDIT: Removed way too many occurrences of the phrase "unless I'm wrong". Please note that I'm by far not an authority in any of these matter and may be colossally wrong in multiple points.

[gengjiawen]

I want list some pros of CMake:

CMake has better IDE support

I think CMake is very friendly to new cpp users. Both clion and visual studio support cmake.
With great IDE support we can attract more developers contribute to nodejs.

I have been use https://github.com/lev-kazakov/node-cmake-generator to generate CMake files for me.
Pretty pleasant developer exerience with clion.
If we can go to other build system in the end, maybe still provide an option to generate cmake.

CMake has better cpp tools integration

With CMake, it will be easier for us to integrate tools like clang-format, clang-tidy.
I recently see https://github.com/bugwelle/cpp-starter-project.

[refack]

@gengjiawen in that respect GYP has two redeeming qualities. It can generate CMake files (specifically CMakeLists.txt), and it can generate compile_commands.json.

[gengjiawen]

@refack Yeap, that's node-cmake-generator internally using. But it doesn't support the latest cmake.