Cap maxWorkers and bump heap to bound Fantom Metro memory (#56529)

Summary:
Pull Request resolved: #56529

Fantom runs every test through a single shared Metro server (started in
`globalSetup`). Jest's default `maxWorkers` is `numCpus - 1`, so on a
high-core box (e.g. 176 CPUs) ~175 workers fire bundle requests at Metro
concurrently. Each in-flight request makes Metro materialize a full
dependency `Graph` (transformed modules, source maps, inverse-deps,
file-watcher subscription), which is hundreds of MB.

The per-test `DELETE` eviction added in D101652820 only releases that
memory after the bundle response completes, so the simultaneous in-flight
set still blows past the previous Node `--max-old-space-size=8192` ceiling
in `scripts/fantom.sh` — the Metro process aborts with
`FATAL ERROR: Ineffective mark-compacts near heap limit` after just a
handful of test suites.

Two coordinated changes that balance throughput and safety:

- `scripts/fantom.sh`: bump the Node heap from 8 GB to 16 GB so we have
  headroom over the observed steady-state peak.
- `private/react-native-fantom/config/jest.config.js`: cap `maxWorkers`
  at `min(numCpus - 1, 16)`. With 8 workers the heap peaked at ~3 GB
  (~400 MB / worker), so 16 workers fits comfortably under a 16 GB cap
  with ~40% headroom.

This is intentionally a balance rather than a hard worker cap — bumping
the heap alone would still leave 100+ in-flight graphs racing GC, and
capping workers alone leaves throughput on the table on big machines.

Changelog: [Internal]

Reviewed By: andrewdacenko

Differential Revision: D101791795

fbshipit-source-id: 035c7235c32303f7b7f1eb05698b2a5cba90edc9

Author: rubennorte

private/react-native-fantom/config/jest.config.js

@@ -11,10 +11,32 @@
 'use strict';
 
 const baseConfig = require('../../../jest.config');
+const os = require('os');
 const path = require('path');
 
+// Every Fantom worker shares a single Metro server. With unbounded
+// parallelism, concurrent bundle builds pile up faster than the
+// per-test DELETE eviction can free their dependency graphs. Cap
+// concurrency so the number of in-flight graphs stays bounded on
+// high-core machines, while still using all available CPUs on smaller
+// ones. The cap pairs with the Node heap size set in scripts/fantom.sh
+// (16 GB) to leave comfortable headroom over the observed peak.
+function getNumCpus() /*: number */ {
+  if (typeof os.availableParallelism === 'function') {
+    return os.availableParallelism();
+  }
+  const cpus = os.cpus();
+  return cpus != null ? cpus.length : 1;
+}
+
+const FANTOM_MAX_WORKERS /*: number */ = Math.max(
+  1,
+  Math.min(getNumCpus() - 1, 16),
+);
+
 module.exports = {
   displayName: 'fantom',
+  maxWorkers: FANTOM_MAX_WORKERS,
   rootDir: path.resolve(__dirname, '../../..') /*:: as string */,
   roots: [
     '<rootDir>/packages/react-native',

scripts/fantom.sh

@@ -16,7 +16,13 @@ else
   export FANTOM_FORCE_OSS_BUILD=1
 fi
 
-export NODE_OPTIONS='--max-old-space-size=8192'
+# The Fantom Jest process hosts a single shared Metro server. Each test
+# triggers a fresh Metro bundle build whose dependency graph (transformed
+# modules, source maps, inverse-deps, file-watcher subscription) lives in
+# memory until the per-test DELETE evicts it. With the maxWorkers cap in
+# jest.config.js (~16 in-flight bundles at peak), 16 GB gives ~40% heap
+# headroom over the observed steady state of ~6 GB.
+export NODE_OPTIONS='--max-old-space-size=16384'
 
 # Parse arguments to extract custom flags
 ARGS=()