Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs-site/guide/bytecode.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,16 @@ pkg --no-bytecode --public-packages "*" --public index.js

`--public` additionally exposes the **top-level project** sources (i.e. your own code) as plain text.

## Fallback to source on failure

When bytecode generation fails for a specific file (e.g. during cross-compilation without QEMU), `pkg` logs a warning and **skips the file** — it won't be available at runtime. If you'd rather ship the affected files as plain source instead of skipping them, pass `--fallback-to-source`:

```sh
pkg --fallback-to-source -t node22-linux-arm64 index.js
```

Files that compile successfully still ship as bytecode; only the ones that fail are included as plain JavaScript. A warning is emitted for each file that falls back.

## SEA mode

SEA mode **never uses bytecode**. Source is always plaintext in a SEA binary. This is a deliberate trade-off — see [SEA vs Standard](/guide/sea-vs-standard).
Expand Down
37 changes: 19 additions & 18 deletions docs-site/guide/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,24 +111,25 @@ pkg .

## CLI reference

| Flag | Short | Description |
| -------------------------- | ----- | ------------------------------------------------------------------------------------------------ |
| `--help` | `-h` | Show usage |
| `--version` | `-v` | Print pkg version |
| `--targets <list>` | `-t` | Comma-separated target list, e.g. `node22-linux-x64` — see [Targets](/guide/targets) |
| `--config <path>` | `-c` | Path to `package.json` or any JSON file with a top-level `pkg` config |
| `--output <path>` | `-o` | Output file name (single-target builds only) |
| `--out-path <dir>` | | Output directory for multi-target builds |
| `--debug` | `-d` | Verbose packaging log — see [Output & debug](/guide/output) |
| `--build` | `-b` | Compile base binaries from source instead of downloading — see [Build](/guide/build) |
| `--public` | | Speed up packaging and disclose top-level sources |
| `--public-packages <list>` | | Force listed packages to be treated as public — see [Bytecode](/guide/bytecode) |
| `--no-bytecode` | | Skip V8 bytecode compilation, embed source as plain JS — see [Bytecode](/guide/bytecode) |
| `--no-native-build` | | Skip building native addons |
| `--no-dict <list>` | | Ignore bundled dictionaries for listed packages (`*` disables all) |
| `--options <list>` | | Bake V8 options into the executable — see [CLI options](/guide/options) |
| `--compress <algo>` | `-C` | Compress the embedded filesystem with `Brotli` or `GZip` — see [Compression](/guide/compression) |
| `--sea` | | Use Node.js SEA instead of the patched base binary — see [SEA mode](/guide/sea-mode) |
| Flag | Short | Description |
| -------------------------- | ----- | -------------------------------------------------------------------------------------------------------------------- |
| `--help` | `-h` | Show usage |
| `--version` | `-v` | Print pkg version |
| `--targets <list>` | `-t` | Comma-separated target list, e.g. `node22-linux-x64` — see [Targets](/guide/targets) |
| `--config <path>` | `-c` | Path to `package.json` or any JSON file with a top-level `pkg` config |
| `--output <path>` | `-o` | Output file name (single-target builds only) |
| `--out-path <dir>` | | Output directory for multi-target builds |
| `--debug` | `-d` | Verbose packaging log — see [Output & debug](/guide/output) |
| `--build` | `-b` | Compile base binaries from source instead of downloading — see [Build](/guide/build) |
| `--public` | | Speed up packaging and disclose top-level sources |
| `--public-packages <list>` | | Force listed packages to be treated as public — see [Bytecode](/guide/bytecode) |
| `--no-bytecode` | | Skip V8 bytecode compilation, embed source as plain JS — see [Bytecode](/guide/bytecode) |
| `--fallback-to-source` | | Ship files as plain source when bytecode generation fails instead of skipping them — see [Bytecode](/guide/bytecode) |
| `--no-native-build` | | Skip building native addons |
| `--no-dict <list>` | | Ignore bundled dictionaries for listed packages (`*` disables all) |
| `--options <list>` | | Bake V8 options into the executable — see [CLI options](/guide/options) |
| `--compress <algo>` | `-C` | Compress the embedded filesystem with `Brotli` or `GZip` — see [Compression](/guide/compression) |
| `--sea` | | Use Node.js SEA instead of the patched base binary — see [SEA mode](/guide/sea-mode) |

Run `pkg --help` at any time for the live list of options.

Expand Down
4 changes: 3 additions & 1 deletion docs-site/guide/targets.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,8 @@ Tracked in [#87](https://github.com/yao-pkg/pkg/issues/87) and [#181](https://gi

1. **Switch to SEA** — `pkg . --sea`. Avoids the V8 bytecode step entirely.
2. **Disable bytecode** — `pkg . --no-bytecode --public-packages "*" --public`. Keeps Standard mode, stores source as plaintext.
3. **Target Node 24** — the regression is gone on `node24-*` targets.
3. **Fallback to source** — `pkg . --fallback-to-source`. Keeps bytecode for files that compile successfully and ships the rest as plain source. See [Bytecode → Fallback to source](/guide/bytecode#fallback-to-source-on-failure).
4. **Target Node 24** — the regression is gone on `node24-*` targets.

:::

Expand All @@ -95,6 +96,7 @@ Regardless of the bug above, the V8 bytecode fabricator in Standard mode needs t
- **macOS** — you can build `x64` on `arm64` with Rosetta 2, but not the opposite
- **Windows** — you can build `x64` on `arm64` with x64 emulation, but not the opposite
- Or disable bytecode generation entirely with `--no-bytecode --public-packages "*" --public`
- Or use `--fallback-to-source` to ship only the failing files as plain source while keeping bytecode for the rest

Enhanced SEA doesn't have this limitation when the host and target share the same Node major: pkg uses `process.execPath` to generate the SEA blob, so no target-arch interpreter is needed. Cross-major SEA builds (e.g. building `node22-*` targets on a Node 24 host) still require an interpreter for the downloaded target binary.

Expand Down
1 change: 1 addition & 0 deletions lib/help.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ export default function help() {
--public-packages force specified packages to be considered public
--no-bytecode skip bytecode generation and include source files as plain js
--no-native-build skip native addons build
--fallback-to-source if bytecode generation fails for a file, ship it as plain source instead of skipping it
--no-dict comma-separated list of packages names to ignore dictionaries. Use --no-dict * to disable all dictionaries
-C, --compress [default=None] compression algorithm = Brotli or GZip
--sea (Experimental) compile give file using node's SEA feature. Requires node v20.0.0 or higher and only single file is supported
Expand Down
2 changes: 2 additions & 0 deletions lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ export async function exec(argv2: string[]) {
'native-build',
'd',
'debug',
'fallback-to-source',
'h',
'help',
'public',
Expand Down Expand Up @@ -692,6 +693,7 @@ export async function exec(argv2: string[]) {
symLinks,
doCompress,
nativeBuild,
fallbackToSource: argv['fallback-to-source'],
});

if (target.platform !== 'win' && target.output) {
Expand Down
1 change: 1 addition & 0 deletions lib/packer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ interface PackerOptions {

export interface Stripe {
snap: string;
skip?: boolean;
store: number;
file?: string;
buffer?: Buffer;
Expand Down
44 changes: 23 additions & 21 deletions lib/producer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,7 @@ interface ProducerOptions {
symLinks: SymLinks;
doCompress: CompressType;
nativeBuild: boolean;
fallbackToSource?: boolean;
}

/**
Expand Down Expand Up @@ -368,6 +369,7 @@ export default function producer({
symLinks,
doCompress,
nativeBuild,
fallbackToSource,
}: ProducerOptions) {
return new Promise<void>((resolve, reject) => {
if (!Buffer.alloc) {
Expand Down Expand Up @@ -443,7 +445,7 @@ export default function producer({
}

if (count === 2) {
if (prevStripe) {
if (prevStripe && !prevStripe.skip) {
const { store } = prevStripe;
let { snap } = prevStripe;
snap = snapshotify(snap, slash);
Expand All @@ -464,33 +466,33 @@ export default function producer({
const snap = snapshotify(stripe.snap, slash);
const sourceBuffer = stripe.buffer;

// Fall back to shipping source for this file (as if it had
// been packed with --no-bytecode). The previous behaviour was
// to emit an empty stripe, which left the VFS entry with
// neither STORE_BLOB nor STORE_CONTENT and blew up at runtime
// with "Error: UNEXPECTED-20" (#87, #181).
const fallbackToContent = (reason: string) => {
log.warn(
`Failed to generate V8 bytecode for ${
stripe.file ?? snap
}. Shipping source instead. Cause: ${reason}`,
);
stripe.store = STORE_CONTENT;
stripe.buffer = sourceBuffer;
return cb(
null,
pipeMayCompressToNewMeter(intoStream(sourceBuffer)),
);
};

return fabricateTwice(
bakes,
target.fabricator,
snap,
sourceBuffer,
(error, buffer) => {
if (error) {
return fallbackToContent(error.message);
const file = stripe.file ?? snap;

if (fallbackToSource) {
log.warn(
`Failed to generate V8 bytecode for ${file}. Shipping source instead. Cause: ${error.message}`,
);
stripe.store = STORE_CONTENT;
stripe.buffer = sourceBuffer;
return cb(
null,
pipeMayCompressToNewMeter(intoStream(sourceBuffer)),
);
}

log.warn(
`Failed to generate V8 bytecode for ${file}. Cause: ${error.message}. ` +
`Use --fallback-to-source to include the file as plain source instead.`,
);
stripe.skip = true;
return cb(null, intoStream(Buffer.alloc(0)));
}

cb(
Expand Down
22 changes: 9 additions & 13 deletions prelude/bootstrap.js
Original file line number Diff line number Diff line change
Expand Up @@ -955,19 +955,15 @@ function payloadFileSync(pointer) {
if (entityBlob) {
return cb2(null, Buffer.from('source-code-not-available'));
}
// why return empty buffer?
// otherwise this error will arise:
// Error: UNEXPECTED-20
// at readFileFromSnapshot (e:0)
// at Object.fs.readFileSync (e:0)
// at Object.Module._extensions..js (module.js:421:20)
// at Module.load (module.js:357:32)
// at Function.Module._load (module.js:314:12)
// at Function.Module.runMain (e:0)
// at startup (node.js:140:18)
// at node.js:1001:3

return cb2(new Error('UNEXPECTED-20'));
return cb2(
new Error(
'[pkg] UNEXPECTED-20: no source or bytecode for ' +
path_ +
'. This usually means V8 bytecode generation failed during ' +
'packaging (e.g. cross-compilation without QEMU). Rebuild with ' +
'--fallback-to-source, --no-bytecode, or --sea to fix this.',
),
);
}

fs.readFileSync = function readFileSync(path_, options_) {
Expand Down
Loading