Conversation
|
Review requested:
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #65810 +/- ##
==========================================
+ Coverage 90.01% 90.26% +0.25%
==========================================
Files 785 790 +5
Lines 269282 271742 +2460
Branches 51288 51865 +577
==========================================
+ Hits 242381 245296 +2915
+ Misses 17401 16942 -459
- Partials 9500 9504 +4
🚀 New features to boost your workflow:
|
|
The CI is possibly hitting the same bug addressed in #65814 as this means SEA is looking for a |
Failed to start CI- Validating Jenkins credentials ✔ Jenkins credentials valid - Querying data for job/node-test-pull-request/77180/ SyntaxError: Unexpected token '<', ..." https://github.com/nodejs/node/actions/runs/34502846196 |
Failed to start CI- Validating Jenkins credentials ✔ Jenkins credentials valid - Querying data for job/node-test-pull-request/77180/ SyntaxError: Unexpected token '<', ..." https://github.com/nodejs/node/actions/runs/34683737619 |
0a39ff7 to
a317623
Compare
When "vfsArchive" names a ZIP archive in the SEA configuration (with "useVfs": true), --build-sea embeds the archive verbatim as one reserved asset, and at runtime the SEA virtual file system mounts the existing ZipProvider over a zero-copy view of the embedded archive instead of the SEAProvider. The main script is injected into the in-memory archive index as a stored entry, so the mounted tree looks the same as with "assets". The archive can be produced with any ZIP tool or with the ZIP support in node:zlib; no ZIP serialization logic is added to the build side. This trades asset read speed (entries are inflated when opened) for a substantially smaller executable when the assets are compressible. Also make FindSingleExecutableResource() return the deserialized SeaResource by reference instead of by value: the copy included the whole assets map, and getAsset() calls it once per asset read, so every fs operation served by the SEA virtual file system paid a cost linear in the number of bundled assets - with 8192 assets, reading each of them took seconds instead of milliseconds. Signed-off-by: Matteo Collina <hello@matteocollina.com>
a317623 to
3bcd0ef
Compare
Failed to start CI�[36m⠋�[39m Getting reviews from nodejs/node/pull/65810 �[36m⠋�[39m Getting commits from nodejs/node/pull/65810 �[36m⠙�[39m Validating Jenkins credentials �[36m⠙�[39m Validating Jenkins credentials ✔ Jenkins credentials valid �[36m⠹�[39m Getting comments from nodejs/node/pull/65810 �[36m⠸�[39m Querying data for job/node-test-pull-request/77180/ �[36m⠸�[39m Querying data for job/node-test-pull-request/77180/ �[36m⠸�[39m Querying API for job/node-test-pull-request/77180/ SyntaxError: Unexpected token '<', ..." https://github.com/nodejs/node/actions/runs/35060560795 |
Failed to start CI�[36m⠋�[39m Getting reviews from nodejs/node/pull/65810 �[36m⠋�[39m Getting commits from nodejs/node/pull/65810 �[36m⠙�[39m Validating Jenkins credentials �[36m⠙�[39m Validating Jenkins credentials ✔ Jenkins credentials valid �[36m⠹�[39m Getting comments from nodejs/node/pull/65810 �[36m⠸�[39m Querying data for job/node-test-pull-request/77180/ �[36m⠸�[39m Querying data for job/node-test-pull-request/77180/ �[36m⠸�[39m Querying API for job/node-test-pull-request/77180/ SyntaxError: Unexpected token '<', ..." https://github.com/nodejs/node/actions/runs/35071280576 |
Failed to start CI�[36m⠋�[39m Getting reviews from nodejs/node/pull/65810 �[36m⠋�[39m Getting commits from nodejs/node/pull/65810 �[36m⠙�[39m Validating Jenkins credentials �[36m⠙�[39m Validating Jenkins credentials ✔ Jenkins credentials valid �[36m⠹�[39m Getting comments from nodejs/node/pull/65810 �[36m⠸�[39m Querying data for job/node-test-pull-request/77180/ �[36m⠸�[39m Querying data for job/node-test-pull-request/77180/ �[36m⠸�[39m Querying API for job/node-test-pull-request/77180/ SyntaxError: Unexpected token '<', ..." https://github.com/nodejs/node/actions/runs/35185994844 |
Yesterday's fix was `.cjs`, on the reasoning that the stub is injected at the root of this package's own bundle and this package says "type": "module", so a .js there is an ES module and `require` is undefined in it. All true, and the wrong conclusion: "mainFormat": "module" says the same thing from the other direction, and then the stub is an ES module on purpose rather than a CommonJS one by extension. Which is the better shape. Every other file in that bundle is ESM, the launcher it loads is ESM, and importing it directly beats requiring an ES module through interop. The stub also stops needing a promise chain: top-level await and a try/catch say the same thing in fewer moving parts. Worth recording what does *not* work, since the name suggests it should: "mainFormat": "commonjs" does not make a .js stub CommonJS. Format detection inside the mount follows the file system rules, so the package.json at the root wins, and the only route to a CommonJS main there is the .cjs extension. .js + CJS syntax, mainFormat unset fails .js + CJS syntax, mainFormat commonjs fails — the type: module still wins .js + ESM syntax, mainFormat module works .cjs + CJS syntax, mainFormat unset works The vfsArchive this depends on is nodejs/node#65810, which is on the deploy branch but not merged upstream; useVfs itself (#65675) is merged. Docs are left alone for now — that status is a week from changing.
Run the synchronous SEA build and execution inside the promise callback. The callback is then counted before buildSEA can skip the test when signing tools are unavailable. Signed-off-by: Matteo Collina <hello@matteocollina.com> Assisted-by: Pi
Commit Queue failedThis pull request has multiple commits, but no landing policy was selected. Add
commit-queue-squash
The pull request was removed from the Commit Queue and labeled
commit-queue-failed
Full Commit Queue output |
This adds a
"vfsArchive"option to the SEA configuration that serves the bundled assets from a ZIP archive, and fixes a performance bug in the SEA asset lookup found while benchmarking it."vfsArchive"Instead of listing individual
"assets", the configuration can point at a prebuilt ZIP archive:{ "main": "main.js", "output": "app", "useVfs": true, "vfsArchive": "assets.zip" }--build-seaembeds the archive verbatim as a single reserved asset (only aPKsignature sanity check happens at build time — no ZIP serialization logic is added to the build side). At runtime the SEA virtual file system mounts the existingZipProviderover a zero-copy view of the embedded archive instead of theSEAProvider, and the main script is injected into the in-memory archive index as a stored entry, so the mounted tree looks exactly like plain"useVfs":__dirname-relative reads, relativerequire(), and bare specifier lookups are unchanged.The archive can be produced with any ZIP tool or with the ZIP support in
node:zlib(zlib.zipFiles()), which is also what the test does."vfsArchive"requires"useVfs": trueand cannot be combined with"assets". Since only the archive is embedded,sea.getAsset()/sea.getAssetAsBlob()do not serve the individual files; they are read through the fs APIs instead.Perf fix: return the SEA resource by reference
FindSingleExecutableResource()returned the deserializedSeaResourceby value, copying the whole assets map on every call.getAsset()calls it once per asset read, making every fs operation served by the SEA VFS pay a cost linear in the number of bundled assets: with 8192 assets, reading each of them took ~7.9s instead of ~270ms (~30x). Fixed in the first commit by returning a reference to the cached resource.Size / speed tradeoff
Measured with a 75% text / 25% random asset mix in 16KB files (Linux x64, warm cache, medians of 15 runs; scenarios: start the binary touching no asset, read one small asset, read every asset):
internal/zipmachinery and parsing the central directory); nothing is inflated until a file is opened.SEAProvidermemcpy path; per-file stat cost is identical (VFS dispatch dominates).Rule of thumb: the archive is a clear win above roughly 30MB of compressible assets when a run reads a subset of them; plain
"useVfs"remains better for small bundles or workloads that repeatedly read large assets.——
AI written, humanly reviewed.