Build failing with heap out of memory error, gatsby@>=2.11.0

[wKovacs64]

Description

Adding gatsby-plugin-use-dark-mode (my plugin) causes the Gatsby build to fail with a JavaScript heap out of memory error, as other people have reported with other plugins (e.g. gatsby-plugin-netlify-cms), specifically when using npm (not yarn) and Node.js prior to v12.

I suspect this is happening due to node_modules being run through Babel now and this plugin depends on Terser, which is a fairly hefty import. If that is the case, do plugin authors need to be conscious of this and include modifications to the Gatsby Webpack config to remove the dependencies rule like gatsby-plugin-netlify-cms did?

cc @wardpeet

Similar issues:

• [email protected] Build failure (JavaScript heap out of memory) #15190
• Build fail with heap out of memory error in gatsby 2.11.6 #15256
• JavaScript heap out of memory wKovacs64/gatsby-plugin-use-dark-mode#23

Also related:

• feat(gatsby): enable babel-loader for all dependencies #14111 (original change)
• Making babel node_modules transpilation better #15213 (game plan)
• fix(*): enable separate config for node_modules transpilation #15270 (partial fix?)

Steps to reproduce

1. Create fresh Gatsby site with default starter
2. Add gatsby-plugin-use-dark-mode with npm
3. Build on Node.js v10 (active LTS)

Example project:

https://github.com/wKovacs64/gpudm-test

Expected result

Successful build.

Actual result

Failed build due to JavaScript heap out of memory error:

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF75AB9C6AA v8::internal::GCIdleTimeHandler::GCIdleTimeHandler+4506
 2: 00007FF75AB77416 node::MakeCallback+4534
 3: 00007FF75AB77D90 node_module_register+2032
 4: 00007FF75AE9189E v8::internal::FatalProcessOutOfMemory+846
 5: 00007FF75AE917CF v8::internal::FatalProcessOutOfMemory+639
 6: 00007FF75B077F94 v8::internal::Heap::MaxHeapGrowingFactor+9620
 7: 00007FF75B06EF76 v8::internal::ScavengeJob::operator=+24550
 8: 00007FF75B06D5CC v8::internal::ScavengeJob::operator=+17980
 9: 00007FF75B076317 v8::internal::Heap::MaxHeapGrowingFactor+2327
10: 00007FF75B076396 v8::internal::Heap::MaxHeapGrowingFactor+2454
11: 00007FF75B1A0637 v8::internal::Factory::NewFillerObject+55
12: 00007FF75B21D826 v8::internal::operator<<+73494
13: 000000B733BDC5C1

Environment

As mentioned previously, this only seems to present itself when using npm (not yarn) and Node.js version prior to v12.

Reproduced in multiple environments (CI and local), but current local is:

  System:
    OS: Windows 10
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
  Binaries:
    Yarn: 1.17.3 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
    npm: 6.9.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: 44.18362.1.0
  npmPackages:
    gatsby: ^2.13.54 => 2.13.56
    gatsby-image: ^2.2.8 => 2.2.8 
    gatsby-plugin-manifest: ^2.2.5 => 2.2.5 
    gatsby-plugin-offline: ^2.2.5 => 2.2.5 
    gatsby-plugin-react-helmet: ^3.1.3 => 3.1.3 
    gatsby-plugin-sharp: ^2.2.10 => 2.2.10 
    gatsby-plugin-use-dark-mode: 1.1.2 => 1.1.2 
    gatsby-source-filesystem: ^2.1.9 => 2.1.9 
    gatsby-transformer-sharp: ^2.2.6 => 2.2.6 

Workarounds

1. Downgrade and pin gatsby to 2.10.5
2. Use yarn (maybe?)
3. Use Node.js v12

[chris-hammond]

I'm seeing the same sort of memory leak, starting in 2.11.0, but I'm not using that plugin and I am using yarn. In 2.10.5, my builds take ~450 MB; in 2.11.0+, my builds skyrocket to the memory limit and fail.

I'm currently using Node.js v10.15.1.

Is there anything else we can do to help get this fixed? This is a pretty major problem.

[wKovacs64]

Yeah, it's not about that specific plugin, that was just one example. Maybe see if you can get someone's attention in #15213?

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[LekoArts]

Please create an up-to-date reproduction if you're still seeing this issue. The linked issue was fixed with #15213 (comment) - thanks!