Exp 234: zero-copy blob parameter transfer via TransferableTypedData

[danReynolds]

Hypothesis

The parameter-encoding-and-binding direction carried an open question: are there blob-heavy parameter shapes where the parameter hop, not SQLite stepping, dominates? The text and numeric bind paths were tightened by earlier experiments, but the blob arm was never measured.

Tracing a Uint8List blob write param, it is copied three times before it reaches a database page: (1) main → writer isolate, when SendPort.send(ExecuteRequest) copies the request graph — including the blob bytes — via the VM's object-graph copy (Dart SDK runtime/vm/object_graph_copy.cc), landing the payload on the shared GC heap; (2) writer → native param arena, in allocateParams; (3) arena → SQLite page, inside sqlite3_step. Copies (2) and (3) are unavoidable. Copy (1) is the target: TransferableTypedData carries bytes across a SendPort by ownership transfer of a malloc'd buffer instead of a graph copy onto the heap.

Exp 005 rejected a Dart binary codec for structured map results — but that lost on encoding structure in Dart. A raw blob parameter has no structure to encode, so it is a distinct question exp 005 never answered.

Approach

New internal helper lib/src/writer/blob_param_transfer.dart:

• wrapBlobParams(params) — on the main isolate, replaces any Uint8List with length >= 256 KB by TransferableTypedData.fromList([blob]). Returns the input list unchanged, with no allocation, when nothing qualifies — so non-blob and small-blob writes are structurally untouched.
• unwrapBlobParams(params) — on the writer isolate, materializes any TransferableTypedData back to a Uint8List view before binding. Also a no-op when nothing was wrapped.

Wrapping is wired into the ExecuteRequest and QueryRequest constructors and the coalescing pump's MultiExecuteRequest build; unwrapping into _handleExecute, _handleTxQuery, and _handleMultiExecute on the writer. fromList copies rather than takes — it does not neuter the caller's list — so the public contract (the caller keeps ownership of the blob it passed) holds, verified by boundary round-trip tests (256 KB − 1 direct, 256 KB wrapped, 512 KB). The threshold is internal (not exported) and matches the read-side sacrificeByteThreshold (256 KB) by design. Deliberately conservative: the batch (executeBatch) path stays on the direct route for now. Full detail in experiments/234-blob-param-transfer.md.

Review-pass fixes

A recall-biased multi-angle review of the branch surfaced and fixed three defects before merge:

1. Writer-isolate death on failing large-blob queries (critical). A failing tx.select with a wrapped blob embedded the already-materialized (spent) TransferableTypedData in its exception; sending that exception back threw from inside the catch clause, killed the writer isolate, and hung the caller forever. Exceptions now embed the unwrapped params, writerEntrypoint's reply-send falls back to a stripped copy if a payload is ever unsendable, and a regression test asserts the error propagates as ResqliteQueryException with the writer surviving.
2. Coalesced writes bypassed the optimization. Concurrent standalone blob writes ride the exp 180 coalescing pump (MultiExecuteRequest), which didn't wrap — so the win silently vanished under the exact burst shape the pump exists for. Wrapping now happens at the drain-site record build (no extra allocation in the no-blob case) with matching unwrap in the handler, covered by a concurrent large-blob round-trip test.
3. FIFO desync window. Writer._request enqueued its reply completer before constructing the request; a constructor throw (now possible — fromList allocates native memory) would strand the completer and shift every later reply off by one. The request is now built first.

The review also fixed the writeup's decision heading (Outcome → Decision, which the docs generator's why-accepted parser actually recognizes) and tightened the boundary test to cover 256 KB − 1 and exactly 256 KB.

Results

Single-row BLOB INSERT, median µs/INSERT, two order-flipped passes (lanes toggled in one process, sampled interleaved so drift indicts both equally):

Size	Δ P1 (base→cand)	Δ P2 (cand→base)	Read
64 KB (control)	+4.2%	+3.3%	direct path — noise
128 KB (control)	−2.9%	+4.1%	direct path — noise
256 KB	−16.4%	−22.7%	wrapped — reproduced win
512 KB	−10.4%	−11.8%	wrapped — reproduced win
1 MB	+0.8%	−4.0%	WAL-write dominated — neutral

For 256 KB–512 KB single-row blob INSERTs — thumbnails, documents, serialized payloads, small media — the wrapped route is a ~15–20% end-to-end speedup, reproduced same-direction in every order-flipped leg across ~7 passes (12+/12 negative at 256 KB, 8/8 at 512 KB), while the sub-threshold controls flip sign across the order flip (the exp 177 drift signature), confirming the wins clear the noise floor.

Mechanism (why it wins — attributed per claim)

A follow-up deep-dive (blob_param_mechanism_proof.dart, blob_param_gc_split.dart, plus VM source) corrected the intuitive "avoids the serializer deep copy" story. Both routes copy the payload exactly once, on the main isolate — send(blob)'s synchronous wall scales linearly with size (proving the sender-side copy; there is no receive-side rebuild since Dart 2.15 isolate groups), fromList is one linear memcpy, send(ttd) is constant-time at every size, and materialize() is ~1 µs flat (a view). The win is where the copy lands:

• Direct: every in-flight blob becomes live data on the shared GC heap. On the real INSERT path (isolated per-lane processes under --verbose_gc, 300 × 256 KB inserts) that costs 29 GCs / 8.6 ms of pause vs 20 GCs / 1.2 ms for the wrapped lane — the scavenger must evacuate live payloads, and each collection safepoints the whole isolate group, including the writer mid-step. Blobs too large for the copier's fast-path new-space allocation additionally abort onto a slow path that redoes the copy in 100 KB safepoint-polled chunks (~2× a plain memcpy at ≥ 1 MB).
• Wrapped: the buffer is malloc'd external memory the GC never traces, and the writer's arena setRange from the materialized view is equal-or-faster — no cost shifted to the writer.

This also explains the size boundaries: below 256 KB the graph copy's fast path is near-free and wrap bookkeeping loses (+23% at 64 KB transport — hence the floor); at ≥ 1 MB the WAL write dominates and huge payloads skip new space (neutral, never negative — hence no ceiling).

Outcome

Accepted (in review). A contained, threshold-gated ~15–20% win on moderate-large single-row blob INSERTs, with the mechanism attributed per claim and the boundaries measured. Would extend to the batch/coalesced blob paths if a blob-heavy executeBatch shows the same hop fraction; would revisit the 256 KB threshold on a differing production blob-size profile.

Test plan

• dart analyze --fatal-infos on lib/, harnesses, and tests (clean)
• dart test test/database_test.dart -j 1 (55/55, incl. a new test that a >256 KB blob round-trips byte-identically through the wrapped path on both execute and tx.select)
• Focused end-to-end A/B (blob_param_write_ab.dart), interleaved + order-flipped, ~7 passes
• Transport microbenchmark (blob_param_transport_ab.dart)
• Mechanism attribution (blob_param_mechanism_proof.dart per-call table ×2; blob_param_gc_split.dart per-lane GC attribution on the real path)
• dart run benchmark/finalize_experiment.dart --experiment=experiments/234-blob-param-transfer.md (green)

🤖 Generated with Claude Code