From 2a032478fe1b9191c902ebbf98b9d5cc80359a61 Mon Sep 17 00:00:00 2001 From: Dan Reynolds Date: Tue, 23 Jun 2026 14:07:11 -0400 Subject: [PATCH] experiment pipeline: make derived + accumulated files bot-owned (phases 1+2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Removes the routine merge-conflict churn on experiment PRs. The conflicts came from experiment branches hand-maintaining shared aggregate files that the post-merge bot also rewrites, so every merge re-staled every open PR. Phase 1 — generated docs are bot-owned, never committed on a branch: - check_generated_data.dart now verifies the *sources* build cleanly (running the generators, which keeps every validity assert) instead of diffing the committed docs/*.json. A branch no longer has to carry a fresh copy. - finalize_experiment.dart no longer writes history.json; it only validates. - New CI guard `guard-generated-docs` fails any PR that modifies docs/experiments/history.json or docs/benchmarks/devices.json. - The Update Docs bot remains the sole owner of those files on main. Phase 2 (signal entries) — the 67-entry experiments{} map in signals.json is split into per-experiment fragments so two experiments never touch the same file: - experiments/signals/base.json — schema meta + per-direction synthesis - experiments/signals/entries/NNN.json — one file per experiment - New generate_signals.dart assembles signals.json from those (entries sorted numerically). signals.json becomes a generated, bot-owned aggregate. - check_experiment_signals.dart validates the assembled-from-source map. - Round-trip verified: regenerated signals.json is byte-for-byte semantically identical to the prior file (same 67 entries, same content; entry order normalized to ascending). Deferred: generating the README experiment tables. README rows are currently a *source* for generate_history.dart, so generating them requires reworking the history parser first — a bigger, riskier change for a lower-frequency conflict. Tracked as a follow-up. Co-Authored-By: Claude Opus 4.8 --- .claude/skills/resqlite-experiment/SKILL.md | 62 +- .github/workflows/ci.yml | 26 + .github/workflows/update-experiments.yml | 11 +- benchmark/check_experiment_signals.dart | 30 +- benchmark/check_generated_data.dart | 127 ++-- benchmark/finalize_experiment.dart | 15 +- benchmark/generate_signals.dart | 93 +++ experiments/RUNNER_INSTRUCTIONS.md | 28 +- experiments/signals.json | 718 ++++++++++++++------ experiments/signals/base.json | 341 ++++++++++ experiments/signals/entries/088.json | 13 + experiments/signals/entries/090.json | 13 + experiments/signals/entries/099.json | 13 + experiments/signals/entries/100.json | 12 + experiments/signals/entries/101.json | 12 + experiments/signals/entries/103.json | 12 + experiments/signals/entries/105.json | 12 + experiments/signals/entries/108.json | 12 + experiments/signals/entries/109.json | 12 + experiments/signals/entries/110.json | 15 + experiments/signals/entries/111.json | 15 + experiments/signals/entries/112.json | 15 + experiments/signals/entries/113.json | 15 + experiments/signals/entries/114.json | 15 + experiments/signals/entries/115.json | 15 + experiments/signals/entries/116.json | 13 + experiments/signals/entries/117.json | 14 + experiments/signals/entries/118.json | 14 + experiments/signals/entries/119.json | 15 + experiments/signals/entries/120.json | 15 + experiments/signals/entries/121.json | 18 + experiments/signals/entries/122.json | 15 + experiments/signals/entries/125.json | 14 + experiments/signals/entries/126.json | 14 + experiments/signals/entries/134.json | 15 + experiments/signals/entries/136.json | 18 + experiments/signals/entries/142.json | 16 + experiments/signals/entries/143.json | 16 + experiments/signals/entries/144.json | 17 + experiments/signals/entries/145.json | 16 + experiments/signals/entries/146.json | 15 + experiments/signals/entries/147.json | 16 + experiments/signals/entries/148.json | 16 + experiments/signals/entries/149.json | 16 + experiments/signals/entries/150.json | 16 + experiments/signals/entries/151.json | 16 + experiments/signals/entries/158.json | 19 + experiments/signals/entries/159.json | 16 + experiments/signals/entries/161.json | 17 + experiments/signals/entries/164.json | 17 + experiments/signals/entries/167.json | 16 + experiments/signals/entries/169.json | 16 + experiments/signals/entries/170.json | 16 + experiments/signals/entries/171.json | 15 + experiments/signals/entries/172.json | 17 + experiments/signals/entries/173.json | 16 + experiments/signals/entries/174.json | 17 + experiments/signals/entries/175.json | 18 + experiments/signals/entries/176.json | 16 + experiments/signals/entries/177.json | 15 + experiments/signals/entries/178.json | 15 + experiments/signals/entries/179.json | 15 + experiments/signals/entries/180.json | 17 + experiments/signals/entries/181.json | 16 + experiments/signals/entries/182.json | 18 + experiments/signals/entries/183.json | 17 + experiments/signals/entries/184.json | 15 + experiments/signals/entries/185.json | 17 + experiments/signals/entries/186.json | 17 + experiments/signals/entries/187.json | 16 + experiments/signals/entries/188.json | 16 + experiments/signals/entries/189.json | 16 + experiments/signals/entries/190.json | 17 + experiments/signals/entries/191.json | 16 + experiments/signals/entries/192.json | 16 + experiments/signals/entries/193.json | 16 + experiments/signals/entries/195.json | 17 + 77 files changed, 2122 insertions(+), 363 deletions(-) create mode 100644 benchmark/generate_signals.dart create mode 100644 experiments/signals/base.json create mode 100644 experiments/signals/entries/088.json create mode 100644 experiments/signals/entries/090.json create mode 100644 experiments/signals/entries/099.json create mode 100644 experiments/signals/entries/100.json create mode 100644 experiments/signals/entries/101.json create mode 100644 experiments/signals/entries/103.json create mode 100644 experiments/signals/entries/105.json create mode 100644 experiments/signals/entries/108.json create mode 100644 experiments/signals/entries/109.json create mode 100644 experiments/signals/entries/110.json create mode 100644 experiments/signals/entries/111.json create mode 100644 experiments/signals/entries/112.json create mode 100644 experiments/signals/entries/113.json create mode 100644 experiments/signals/entries/114.json create mode 100644 experiments/signals/entries/115.json create mode 100644 experiments/signals/entries/116.json create mode 100644 experiments/signals/entries/117.json create mode 100644 experiments/signals/entries/118.json create mode 100644 experiments/signals/entries/119.json create mode 100644 experiments/signals/entries/120.json create mode 100644 experiments/signals/entries/121.json create mode 100644 experiments/signals/entries/122.json create mode 100644 experiments/signals/entries/125.json create mode 100644 experiments/signals/entries/126.json create mode 100644 experiments/signals/entries/134.json create mode 100644 experiments/signals/entries/136.json create mode 100644 experiments/signals/entries/142.json create mode 100644 experiments/signals/entries/143.json create mode 100644 experiments/signals/entries/144.json create mode 100644 experiments/signals/entries/145.json create mode 100644 experiments/signals/entries/146.json create mode 100644 experiments/signals/entries/147.json create mode 100644 experiments/signals/entries/148.json create mode 100644 experiments/signals/entries/149.json create mode 100644 experiments/signals/entries/150.json create mode 100644 experiments/signals/entries/151.json create mode 100644 experiments/signals/entries/158.json create mode 100644 experiments/signals/entries/159.json create mode 100644 experiments/signals/entries/161.json create mode 100644 experiments/signals/entries/164.json create mode 100644 experiments/signals/entries/167.json create mode 100644 experiments/signals/entries/169.json create mode 100644 experiments/signals/entries/170.json create mode 100644 experiments/signals/entries/171.json create mode 100644 experiments/signals/entries/172.json create mode 100644 experiments/signals/entries/173.json create mode 100644 experiments/signals/entries/174.json create mode 100644 experiments/signals/entries/175.json create mode 100644 experiments/signals/entries/176.json create mode 100644 experiments/signals/entries/177.json create mode 100644 experiments/signals/entries/178.json create mode 100644 experiments/signals/entries/179.json create mode 100644 experiments/signals/entries/180.json create mode 100644 experiments/signals/entries/181.json create mode 100644 experiments/signals/entries/182.json create mode 100644 experiments/signals/entries/183.json create mode 100644 experiments/signals/entries/184.json create mode 100644 experiments/signals/entries/185.json create mode 100644 experiments/signals/entries/186.json create mode 100644 experiments/signals/entries/187.json create mode 100644 experiments/signals/entries/188.json create mode 100644 experiments/signals/entries/189.json create mode 100644 experiments/signals/entries/190.json create mode 100644 experiments/signals/entries/191.json create mode 100644 experiments/signals/entries/192.json create mode 100644 experiments/signals/entries/193.json create mode 100644 experiments/signals/entries/195.json diff --git a/.claude/skills/resqlite-experiment/SKILL.md b/.claude/skills/resqlite-experiment/SKILL.md index f4e3c94b..6efe18c8 100644 --- a/.claude/skills/resqlite-experiment/SKILL.md +++ b/.claude/skills/resqlite-experiment/SKILL.md @@ -113,14 +113,15 @@ Run this mental (or literal) checklist: - [ ] `git status --short benchmark/results/` — is there an untracked result file? - [ ] Does that file's filename timestamp match `grep "^**Date:**" experiments/NNN-*.md`? - [ ] Is the experiment listed in `experiments/README.md` (Accepted or Rejected section)? +- [ ] Is the signal entry in its own file, `experiments/signals/entries/NNN.json` + (not hand-edited into the generated `signals.json`)? - [ ] Does the experiment doc have the headings the parser expects? (`Problem`, `Hypothesis`, `Approach` or `What We Built`, `Results`, `Decision` or `Why Accepted` / `Why Rejected`) -- [ ] Did `finalize_experiment.dart` (or `generate_history.dart`) run **last**? - If you touched any experiment / doc / `signals.json` / result / fixture - file *after* finalizing, re-run it — otherwise the committed - `history.json` is stale and the freshness check fails (post-merge, if the - PR auto-merged before CI re-ran). +- [ ] Did `finalize_experiment.dart` pass? It only *validates* sources — it does + not write the generated aggregates, and you must not commit + `docs/experiments/history.json`, `docs/benchmarks/devices.json`, or + `experiments/signals.json` (the bot regenerates them on `main`). The generator's section extraction tolerates a few heading variants; see `_extractSection` in `generate_history.dart` for the full list. @@ -306,12 +307,18 @@ worktree off `origin/main` (steps below), leaving the current tree untouched: git fetch origin ``` -Every experiment PR rewrites the same three shared files — -`experiments/README.md`, `experiments/signals.json`, and the generated -`docs/experiments/history.json` — and CI's `check_generated_data.dart` fails -any PR whose `history.json` predates a merge. So concurrent runs that grab the -same number, or do the same work, collide and stale each other (exp 168 was -claimed by three PRs; exp 175 by two runs shipping the *same* follow-up). +Experiment PRs used to collide on shared generated files; that is now designed +out. The generated aggregates — `docs/experiments/history.json`, +`docs/benchmarks/devices.json`, and `experiments/signals.json` — are +**bot-owned on `main` and never committed on a branch** (CI's +`guard-generated-docs` job blocks them, and `check_generated_data.dart` only +checks that the *sources* build). Each experiment's signal data lives in its +own file, `experiments/signals/entries/NNN.json`, so two experiments never +touch the same one. The only files a normal run still shares are +`experiments/README.md` (rows append) and `experiments/signals/base.json` (the +per-direction synthesis). You must still claim your number — concurrent runs +that grab the same number, or ship the same follow-up, still collide (exp 168 +was claimed by three PRs; exp 175 by two runs shipping the *same* follow-up). **1. Claim the number atomically.** A plain "check open PRs, then pick the next free" *races*: two runs check, both see N free, both take N. That is exactly @@ -363,24 +370,33 @@ which case expect to regenerate `history.json` on the later one. ## Resolving a stale derived-file conflict -If a PR did fall behind `main`, the conflict is mechanical — only -generated/narrative files collide: +Generated aggregates no longer conflict: `docs/experiments/history.json`, +`docs/benchmarks/devices.json`, and `experiments/signals.json` are bot-owned +and never committed on a branch, so a stale branch just takes `main`'s copy +with a one-sided auto-merge. If a PR falls behind `main`, the only files that +can really conflict are hand-edited *sources*: + +- `experiments/README.md` — rows append; usually auto-merges. If two + experiments inserted at the same spot, fix the row order by hand. +- `experiments/signals/entries/NNN.json` — one file per experiment, so a true + collision only happens when two runs claimed the same number `N`. That's a + numbering bug: renumber, never overwrite a prior experiment's entry. +- `experiments/signals/base.json` — two experiments editing the same + direction's `currentRead` / `notesForExperimenters` narrative is a real + weave; keep both contributions. ```bash git merge origin/main -dart run benchmark/generate_history.dart # rebuilds docs/experiments/history.json -dart run benchmark/generate_devices.dart # rebuilds docs/benchmarks/devices.json -# hand-reconcile experiments/signals.json: keep BOTH sides' entries -dart run benchmark/check_generated_data.dart # must print "up to date" -dart run benchmark/check_experiment_signals.dart +# resolve any of the source files above by hand, then: +dart run benchmark/check_generated_data.dart # sources build cleanly +dart run benchmark/check_experiment_signals.dart # signal map valid git add -A && git commit --no-edit ``` -`experiments/README.md` usually auto-merges (rows append). `signals.json` -needs care: two experiments editing the same `currentRead` / -`notesForExperimenters` narrative is a real weave, and two experiments -claiming the same `experiments.` key is a number collision — fix the -numbering, never overwrite a prior experiment's entry. +Do **not** regenerate or `git add` the generated aggregates. If your branch +somehow carries changes to them, revert: `git checkout origin/main -- +docs/experiments/history.json docs/benchmarks/devices.json +experiments/signals.json`. ## Post-merge diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 839d8cdd..2390793d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -33,6 +33,32 @@ jobs: exit 1 fi + guard-generated-docs: + # history.json and devices.json are generated aggregates owned by the + # "Update Docs Data" bot on main. Experiment branches must NOT commit them, + # so a merge (or a post-merge bot commit) never re-conflicts open PRs on + # them — the routine churn this whole convention exists to remove. See + # experiments/RUNNER_INSTRUCTIONS.md ("Generated files are bot-owned"). + name: Guard against committing generated docs + runs-on: ubuntu-latest + if: github.event_name == 'pull_request' + steps: + - uses: actions/checkout@v6.0.2 + with: + fetch-depth: 0 + - name: Check PR diff for bot-owned generated files + run: | + base="origin/${{ github.base_ref }}" + changed=$(git diff --name-only "$base...HEAD" \ + -- 'docs/experiments/history.json' 'docs/benchmarks/devices.json' || true) + if [ -n "$changed" ]; then + echo "::error::Generated docs are bot-owned; do not commit them on a branch." + echo "Revert them so the PR carries no change to these files:" + printf ' %s\n' $changed + echo "Run: git checkout origin/${{ github.base_ref }} -- docs/experiments/history.json docs/benchmarks/devices.json" + exit 1 + fi + test: name: Tests runs-on: macos-latest diff --git a/.github/workflows/update-experiments.yml b/.github/workflows/update-experiments.yml index 3d796e34..8ba24bcc 100644 --- a/.github/workflows/update-experiments.yml +++ b/.github/workflows/update-experiments.yml @@ -5,6 +5,8 @@ on: branches: [main] paths: - 'experiments/*.md' + - 'experiments/signals/**' + - 'benchmark/generate_signals.dart' - 'benchmark/results/*.md' - 'benchmark/results/*.json' - 'benchmark/HARDWARE_RESULTS.md' @@ -39,6 +41,9 @@ jobs: - name: Generate history JSON run: dart run benchmark/generate_history.dart + - name: Generate signals JSON + run: dart run benchmark/generate_signals.dart + - name: Generate devices JSON run: dart run benchmark/generate_devices.dart @@ -51,13 +56,13 @@ jobs: - name: Check for changes id: diff run: | - git diff --quiet docs/ && echo "changed=false" >> "$GITHUB_OUTPUT" || echo "changed=true" >> "$GITHUB_OUTPUT" + git diff --quiet docs/ experiments/signals.json && echo "changed=false" >> "$GITHUB_OUTPUT" || echo "changed=true" >> "$GITHUB_OUTPUT" - name: Commit and push if: steps.diff.outputs.changed == 'true' run: | git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - git add docs/ - git commit -m "Auto-update docs (experiments + devices + blog + releases)" + git add docs/ experiments/signals.json + git commit -m "Auto-update docs (experiments + signals + devices + blog + releases)" git push diff --git a/benchmark/check_experiment_signals.dart b/benchmark/check_experiment_signals.dart index 57c8f6df..355615ff 100644 --- a/benchmark/check_experiment_signals.dart +++ b/benchmark/check_experiment_signals.dart @@ -1,6 +1,7 @@ -import 'dart:convert'; import 'dart:io'; +import 'generate_signals.dart' as generate_signals; + const _experimentsDir = 'experiments'; const _readmePath = '$_experimentsDir/README.md'; const _signalsPath = '$_experimentsDir/signals.json'; @@ -115,26 +116,21 @@ int? _numericExperimentId(String id) { return match == null ? null : int.tryParse(match.group(0)!); } +/// Assembles the signal map from its sources (`experiments/signals/base.json` + +/// `experiments/signals/entries/NNN.json`) and validates that, rather than the +/// committed `signals.json` — which is a generated, bot-owned aggregate that an +/// experiment branch never has to keep fresh. A malformed fragment surfaces +/// here as an assembly failure. Map? _readSignals(List<_ValidationError> errors) { - final file = File(_signalsPath); - if (!file.existsSync()) { - _signalError(errors, 'Missing $_signalsPath.'); - return null; - } - - Object? decoded; try { - decoded = json.decode(file.readAsStringSync()); - } on FormatException catch (error) { - _signalError(errors, 'signals.json is not valid JSON: ${error.message}'); - return null; - } - - if (decoded is! Map) { - _signalError(errors, 'signals.json must contain a top-level JSON object.'); + final data = generate_signals.buildSignalsData( + signalsSourceDir: Directory('$_experimentsDir/signals'), + ); + return data.cast(); + } catch (error) { + _signalError(errors, 'signals sources do not assemble: $error'); return null; } - return decoded.cast(); } void _checkSignals( diff --git a/benchmark/check_generated_data.dart b/benchmark/check_generated_data.dart index e19e6a4f..8914e71f 100644 --- a/benchmark/check_generated_data.dart +++ b/benchmark/check_generated_data.dart @@ -1,108 +1,65 @@ -import 'dart:convert'; import 'dart:io'; import 'generate_devices.dart' as generate_devices; import 'generate_history.dart' as generate_history; +import 'generate_signals.dart' as generate_signals; +/// Verifies that the generated-docs sources are well-formed enough that the +/// generators run to completion. +/// +/// This does **not** compare against the committed `docs/.../*.json` or +/// `experiments/signals.json`. Those are generated artifacts owned by the +/// post-merge "Update Docs Data" bot, not by experiment branches — see +/// experiments/RUNNER_INSTRUCTIONS.md ("Generated files are bot-owned"). A +/// branch never has to carry a fresh copy, which is what used to force every +/// open experiment PR to re-resolve the same mechanical conflict on those +/// files. The job here is to fail fast on a *source* that can't be generated +/// (a malformed signals fragment, a missing run declaration, a broken +/// experiment <-> benchmark-run mapping) — every one of those checks lives +/// inside the build functions below and throws on violation. Future main() async { - final mismatches = []; - - await _checkJsonFile( - label: 'devices.json', - currentPath: 'docs/benchmarks/devices.json', - buildExpected: (generatedAt) => generate_devices.buildDevicesData( + final builders = { + 'devices.json': () => generate_devices.buildDevicesData( hardwareResultsMarkdown: File( 'benchmark/HARDWARE_RESULTS.md', ).readAsStringSync(), resultsDir: Directory('benchmark/results'), - generatedAt: generatedAt, + generatedAt: null, ), - mismatches: mismatches, - ); - - await _checkJsonFile( - label: 'history.json', - currentPath: 'docs/experiments/history.json', - buildExpected: (generatedAt) => generate_history.buildHistoryData( + 'history.json': () => generate_history.buildHistoryData( resultsDir: Directory('benchmark/results'), experimentsDir: Directory('experiments'), - generatedAt: generatedAt, + generatedAt: null, + ), + 'signals.json': () => generate_signals.buildSignalsData( + signalsSourceDir: Directory('experiments/signals'), + generatedAt: null, ), - mismatches: mismatches, - ); + }; + + final failures = []; + for (final entry in builders.entries) { + try { + entry.value(); + print('${entry.key}: generators build cleanly.'); + } catch (error, stack) { + failures.add(entry.key); + stderr.writeln('::error::${entry.key} failed to generate: $error'); + stderr.writeln(stack); + } + } - if (mismatches.isNotEmpty) { + if (failures.isNotEmpty) { stderr.writeln(''); stderr.writeln( - 'Benchmark-generated docs are stale. Re-run the generators and commit the updated files:', - ); - stderr.writeln(' dart run benchmark/generate_devices.dart'); - stderr.writeln(' dart run benchmark/generate_history.dart'); - stderr.writeln('For experiment writeups, prefer:'); - stderr.writeln( - ' dart run benchmark/finalize_experiment.dart --experiment=experiments/NNN-short-slug.md', + 'Generated-docs sources do not build: ${failures.join(', ')}. ' + 'Fix the offending source (experiment doc, benchmark result, or ' + 'signals fragment) — you do NOT need to commit the regenerated ' + 'docs/*.json or signals.json; the bot owns those on main.', ); exitCode = 1; return; } - print('Benchmark-generated docs are up to date.'); -} - -Future _checkJsonFile({ - required String label, - required String currentPath, - required Object Function(String? generatedAt) buildExpected, - required List mismatches, -}) async { - final currentFile = File(currentPath); - if (!currentFile.existsSync()) { - throw StateError( - 'Missing generated docs artifact: $currentPath. ' - 'Re-run the generator and commit the updated file.', - ); - } - - final currentText = currentFile.readAsStringSync(); - final currentJson = json.decode(currentText); - if (currentJson is! Map) { - throw StateError('$currentPath does not contain a top-level JSON object.'); - } - - final expected = buildExpected(currentJson['generated']?.toString()); - const encoder = JsonEncoder.withIndent(' '); - final currentComparable = '${encoder.convert(currentJson)}\n'; - final expectedText = '${encoder.convert(expected)}\n'; - - if (currentComparable == expectedText) { - print('$label is current.'); - return; - } - - mismatches.add(label); - stderr.writeln('::error file=$currentPath::$label is stale.'); - - final tempDir = await Directory.systemTemp.createTemp( - 'resqlite-generated-data-diff_', - ); - try { - final normalizedCurrentFile = File( - '${tempDir.path}/current_${currentFile.uri.pathSegments.last}', - )..writeAsStringSync(currentComparable); - final expectedFile = File( - '${tempDir.path}/${currentFile.uri.pathSegments.last}', - )..writeAsStringSync(expectedText); - final diff = await Process.run('diff', [ - '-u', - normalizedCurrentFile.path, - expectedFile.path, - ]); - if ((diff.stdout as String).trim().isNotEmpty) { - stderr.writeln(diff.stdout); - } else if ((diff.stderr as String).trim().isNotEmpty) { - stderr.writeln(diff.stderr); - } - } finally { - await tempDir.delete(recursive: true); - } + print('All generated-docs sources build cleanly.'); } diff --git a/benchmark/finalize_experiment.dart b/benchmark/finalize_experiment.dart index 68b6723c..46892d6a 100644 --- a/benchmark/finalize_experiment.dart +++ b/benchmark/finalize_experiment.dart @@ -7,8 +7,12 @@ const _usage = ''' Usage: dart run benchmark/finalize_experiment.dart --experiment=experiments/NNN-short-slug.md -Regenerates docs/experiments/history.json and runs the experiment postflight -checks that should pass before committing an experiment record. +Runs the experiment postflight checks that should pass before committing an +experiment record. It does NOT write docs/experiments/history.json, +docs/benchmarks/devices.json, or experiments/signals.json — those are +generated aggregates owned by the post-merge bot, not committed on branches +(see experiments/RUNNER_INSTRUCTIONS.md). The checks below verify the sources +generate cleanly and the signal map is valid. '''; Future main(List args) async { @@ -35,12 +39,7 @@ Future main(List args) async { final commands = [ _Command( - label: 'generate experiment history', - executable: Platform.resolvedExecutable, - args: const ['run', 'benchmark/generate_history.dart'], - ), - _Command( - label: 'check generated docs data', + label: 'check generated docs sources build cleanly', executable: Platform.resolvedExecutable, args: const ['run', 'benchmark/check_generated_data.dart'], ), diff --git a/benchmark/generate_signals.dart b/benchmark/generate_signals.dart new file mode 100644 index 00000000..44b72572 --- /dev/null +++ b/benchmark/generate_signals.dart @@ -0,0 +1,93 @@ +// ignore_for_file: avoid_print +import 'dart:convert'; +import 'dart:io'; + +/// Assembles `experiments/signals.json` from its hand-edited sources: +/// +/// experiments/signals/base.json — schema metadata + the per-direction +/// research synthesis (`directions[]`) +/// experiments/signals/entries/NNN.json — one file per experiment, holding +/// that experiment's +/// {directions, outcomeClass, +/// changedBeliefs, nextSignals} +/// +/// `signals.json` is a *generated* aggregate owned by the post-merge "Update +/// Docs Data" bot — see experiments/RUNNER_INSTRUCTIONS.md. Experiments add an +/// entry fragment (their own file, so two concurrent experiments never touch +/// the same one) and, when a direction's synthesis changes, edit `base.json`. +/// They never hand-edit `signals.json`. +Map buildSignalsData({ + required Directory signalsSourceDir, + String? generatedAt, +}) { + final baseFile = File('${signalsSourceDir.path}/base.json'); + if (!baseFile.existsSync()) { + throw StateError('Missing signals source: ${baseFile.path}'); + } + final base = json.decode(baseFile.readAsStringSync()); + if (base is! Map) { + throw StateError('${baseFile.path} must be a top-level JSON object.'); + } + if (base.containsKey('experiments')) { + throw StateError( + '${baseFile.path} must not contain "experiments"; per-experiment entries ' + 'live in ${signalsSourceDir.path}/entries/NNN.json.', + ); + } + + final entriesDir = Directory('${signalsSourceDir.path}/entries'); + final entryFiles = entriesDir.existsSync() + ? entriesDir + .listSync() + .whereType() + .where((f) => f.path.endsWith('.json')) + .toList() + : []; + + final ids = []; + final byId = {}; + for (final file in entryFiles) { + final id = file.uri.pathSegments.last.replaceFirst(RegExp(r'\.json$'), ''); + final decoded = json.decode(file.readAsStringSync()); + if (decoded is! Map) { + throw StateError('${file.path} must be a JSON object.'); + } + if (byId.containsKey(id)) { + throw StateError('Duplicate signals entry for experiment $id.'); + } + byId[id] = decoded; + ids.add(id); + } + ids.sort(_compareExperimentIds); + + final experiments = {for (final id in ids) id: byId[id]}; + + // Append in the canonical position (after the synthesis), preserving the + // base key order. + return {...base, 'experiments': experiments}; +} + +int _compareExperimentIds(String a, String b) { + final ma = RegExp(r'^(\d+)(.*)$').firstMatch(a); + final mb = RegExp(r'^(\d+)(.*)$').firstMatch(b); + final na = ma == null ? (1 << 30) : int.parse(ma.group(1)!); + final nb = mb == null ? (1 << 30) : int.parse(mb.group(1)!); + if (na != nb) return na.compareTo(nb); + final sa = ma == null ? a : ma.group(2)!; + final sb = mb == null ? b : mb.group(2)!; + return sa.compareTo(sb); +} + +Future main() async { + final data = buildSignalsData( + signalsSourceDir: Directory('experiments/signals'), + ); + const encoder = JsonEncoder.withIndent(' '); + File('experiments/signals.json').writeAsStringSync('${encoder.convert(data)}\n'); + final experiments = data['experiments'] as Map; + final directions = data['directions'] as List; + print( + 'Wrote experiments/signals.json ' + '(${experiments.length} experiment entries, ${directions.length} directions).', + ); +} diff --git a/experiments/RUNNER_INSTRUCTIONS.md b/experiments/RUNNER_INSTRUCTIONS.md index cb911951..81df983b 100644 --- a/experiments/RUNNER_INSTRUCTIONS.md +++ b/experiments/RUNNER_INSTRUCTIONS.md @@ -212,8 +212,12 @@ When finished: what was measured, and what would make the area interesting again — a "rejected, no signal" record is worth less than a "rejected because X, would reopen if Y" record. -- update [`signals.json`](signals.json) if the run changes how future agents - should interpret an area. +- record the run's signal in its own file, + `experiments/signals/entries/NNN.json` (directions, outcomeClass, + changedBeliefs, nextSignals) — never the generated `signals.json`, and never + a shared file, so two concurrent runs can't collide on it. When the run + changes how future agents should read a whole *direction*, also update that + direction's synthesis in [`signals/base.json`](signals/base.json). - add to [`JOURNAL.md`](JOURNAL.md) only when the run surfaced a *transferable* lesson — something a future runner could reapply to a different direction or could waste time relearning. Per-direction state goes in `signals.json`, not @@ -221,21 +225,23 @@ When finished: - do not edit [`../doc/stories/`](../doc/stories/) as part of an experiment run. Story posts are updated on maintainer request, not per experiment. - run the experiment finalizer after the writeup, README row, and - `signals.json` entry are in place: + `experiments/signals/entries/NNN.json` are in place: ```bash dart run benchmark/finalize_experiment.dart \ --experiment=experiments/NNN-short-slug.md ``` - This regenerates `docs/experiments/history.json`, verifies generated docs, - and checks that the experiment is indexed in both the README and signal map. - **Run it as the very LAST step.** If you edit *any* tracked file afterward — - the writeup, README, `signals.json`, a result artifact, a relocated fixture — - re-run it, or you commit a stale `history.json`. The freshness check then - fails *post-merge* (it can slip through if the PR auto-merges before CI - re-runs), and only the Update-Docs bot saves you — a runner without that bot - ships a red `main`. This is exactly how exp 177 briefly reddened the main tip. + This verifies the generated-docs **sources** build cleanly and the signal map + is valid. It does **not** write `docs/experiments/history.json`, + `docs/benchmarks/devices.json`, or `experiments/signals.json` — those are + generated aggregates owned by the post-merge Update-Docs bot. **Never commit + them on your branch** (CI's `guard-generated-docs` job blocks it). You commit + only sources: the writeup, the README row, your signal fragment, benchmark + result files, and any code; the bot regenerates the aggregates on `main` + after merge. This is what keeps a stale branch from re-conflicting on + generated files the way the old "regenerate + commit `history.json` on every + branch" rule did. - run focused validation plus the relevant repo checks - open a PR when the local experiment package is coherent enough for review diff --git a/experiments/signals.json b/experiments/signals.json index 7a724990..7d592522 100644 --- a/experiments/signals.json +++ b/experiments/signals.json @@ -23,10 +23,42 @@ { "id": "stream-rerun-dispatch", "status": "active", - "subsystems": ["streaming", "dispatch", "reader-pool", "invalidation"], + "subsystems": [ + "streaming", + "dispatch", + "reader-pool", + "invalidation" + ], "currentRead": "Stream fan-out performance is shaped by rerun scheduling, reader-pool admission, completion-side churn, writer/request residual, and dependency precision. Queue changes have produced both strong wins and sharp regressions. Exp 120 closed the upstream over-dispatch in StreamEngine._flushQueue (parked_total drops 3,590 -> 0 on A11c overlap and 1,198 -> 0 on keyed-PK; max_parked 46 -> 0), and exp 122 removed the remaining stream-admission async boundary by constructing StreamEngine with a concrete ReaderPool. Exp 121 ruled out invalidation traversal as the active implementation target: overlap invalidation is 10-15% of wall, column intersection is 2.5-5.7%, and the structural ceiling is at the per-benchmark decision-threshold edge. Exp 134 proved row-level dirty precision can halve keyed-PK writer-burst wall for a narrow `WHERE id = ?` proof, but its internal SQL recognizer is rejected; revive that area only through explicit API/design or real workload evidence. Exp 136 ships the completion-side reader-handler counter: on A11c overlap the reader worker port handler is 28.57% of total wall (burst + drain) at ~18 us per call across 4,228 calls/burst, and subscriber-fanout emit is only 0.35% of the chain. Exp 147 split writer-side burst wall from SQLite-facing writer calls: on A11c overlap, SQLite is 15.7 ms / 166.8 ms (9.4%), invalidation is 18.8%, and residual writer/request wall is 71.8%; keyed-PK shows the same shape (18.1% SQLite, 18.7% invalidation, 63.3% residual). Exp 148 tested the natural reader-reply batching follow-up and rejected it: the profile smoke cut A11c overlap completion callbacks 4,527 -> 1,425 and completion wall 109.6 ms -> 55.6 ms, but Tracelite measured elapsed stayed neutral/slower (+5.18% high-cardinality, +3.28% many-streams, +13.5% keyed-PK). Exp 151 tested synchronous writer response resolution against the residual writer/request bucket and rejected it: high-cardinality fanout stayed neutral (+2.92%), keyed-PK trended slower with too-noisy evidence (+18.5%), and many-streams writer throughput trended slower (+14.0%). Exp 170 tested the matching request-side variant — `Mutex.tryLock` plus a non-`async` `Writer.execute` / `executeBatch` to drop the uncontended `await _mutex.lock()` microtask hop — and rejected it: the primary Single Inserts (100 sequential) / sequential-awaited (2000 writes) lanes stayed within ±2 % (wrong direction) across paired runs, while the only positive signal (-7.8 % on Concurrent Single Inserts) is a row exp 159 already drives at -58 % to -61 %. SQLite-step tuning, stream admission, invalidation traversal, plain worker-side reader-reply batching, synchronous writer response resolution, synchronous writer request acquisition, SQL-recognizer-based keyed-PK precision, allocation-only `_flushQueue` cleanup, and standalone residual-split profiling are not active targets on currently-measured workloads. Exp 159 attacked the residual structurally: persistent writer reply port + cached SendPort + sync FIFO completion remove fixed per-round-trip scheduling cost, and releasing the write lock at send time pipelines concurrent standalone writes through the worker port FIFO; the focused concurrent-burst benchmark improved 36-45%, exp 147 residual_us dropped on all four audit workloads, and stream-dispatch Tracelite guardrails were neutral on the clean order-flipped pass. Exp 161 closes the release-suite gap by promoting the concurrent-burst shape into `benchmark/suites/writes.dart` as a paired Single Inserts (100 sequential) / Concurrent Single Inserts (100 concurrent) row pair, so exp 159's pipelining win and future writer-scheduling experiments are evaluable on a public release lane (resqlite concurrent median ~1.1 ms vs ~2.9 ms sequential). Exp 171 then tried to apply exp 159's `_sendPort` cache pattern one layer up — a sync-readable `_resolvedRuntime` field on `Database` so post-open hot paths skip the `await _runtime` microtask hop — and rejected it: two order-flipped passes on writer_pipelining.dart produced alternating-sign deltas inside per-round variance (sequential-awaited -2.3%/+2.5%, transaction-guardrail -7.5%/+6.1%), so a ~1-2 us per-call hop sits at or below the harness floor. Database-layer microtask hop trimming is now off the candidate list; the next sequential-write reduction must reduce round-trip count (group commit) or change transport.", - "keyPriors": ["120", "121", "134", "136", "147", "148"], - "archive": ["045", "075", "083", "084", "085", "100", "105", "106", "114", "115", "118", "119", "122", "145", "151", "164", "170", "171", "182"], + "keyPriors": [ + "120", + "121", + "134", + "136", + "147", + "148" + ], + "archive": [ + "045", + "075", + "083", + "084", + "085", + "100", + "105", + "106", + "114", + "115", + "118", + "119", + "122", + "145", + "151", + "164", + "170", + "171", + "182" + ], "interestingIf": [ "a change reduces duplicate reruns before reader-pool admission", "a workload shows row-level precision is worth explicit API or metadata design outside a SQL text recognizer", @@ -55,10 +87,33 @@ { "id": "parameter-encoding-and-binding", "status": "active", - "subsystems": ["ffi", "params", "writer", "allocation"], + "subsystems": [ + "ffi", + "params", + "writer", + "allocation" + ], "currentRead": "Parameter work can still matter when it removes meaningful native allocation, copying, or repeated SQLite work. Smaller Dart-side allocation cleanups and two-parameter batch-list flattening changes usually measured flat, but exp 113 found a clear wide-row batch signal by avoiding the temporary flat Dart parameter list entirely. Exp 116 promotes the 10,000-row x 20-parameter mixed batch shape into release-suite coverage so width regressions are visible outside the focused script. Exp 125 then showed that large wide ASCII-heavy batches still had removable per-string UTF-8 list allocation inside the matrix encoder: direct ASCII payload packing improved focused 10k x20 from 17.199 ms to 12.760 ms and release Wide Batch Insert from 18.201 ms to 13.031 ms. Exp 126 extended that same allocation-removal shape to non-ASCII wide batches with direct UTF-8 payload writing: focused Unicode 10k x20 improves 21.945 ms to 18.988 ms and emoji 10k x20 improves 24.187 ms to 17.458 ms while release write-suite guardrails stay neutral. Exp 142 retested direct single-row text parameter encoding under Tracelite on chat-sim and narrow-batch-insert; it did not clear the primary gate and trended slower (+6.86% and +16.4%). Exp 146 tested lowering the ASCII batch-packing threshold to 2 params / 64 total params with a Tracelite A/B run over narrow-batch-insert; it produced no primary improvement (resqlite +1.45%, neutral) and a noisy sqlite_async guardrail, so small/narrow writes should stay on the generic path. Exp 149 found the middle ground with Tracelite profile merge rounds: repeated six-parameter ASCII merge batches improve executeBatch p50 88 -> 75 us and writer SQLite time 87,895 -> 75,947 us when admitted at 6 params / 600 total params. Exp 150 fixed the first-row-null blind spot inside that same guard: nullable ASCII 10k x8 improves 13.552 -> 11.152 ms and 10k x20 improves 25.738 -> 21.723 ms, while existing ASCII and Unicode wide guardrails stayed neutral in the focused pass. Exp 186 then closed exp 179's named revisit condition: added a focused single-row large-text-bind workload (single_row_large_text_bind.dart, 1 KB to 1 MB sequential INSERT shapes) and ran archive/exp-179's direct-ASCII allocateParams rewrite against it. The encoder savings are immaterial at 1 KB but become decisive once the bound text crosses the mid-tens-of-KB range — focused medians improve -15.4% / -11.1% at 16 KB, -17.3% / -18.7% at 64 KB, -32.3% / -32.1% at 256 KB, and -26.7% / -28.5% at 1 MB across two order-flipped passes, with encoder-isolation deltas reproducing exp 179's -45% / -58% / -37% on the synthetic micro. Small-payload release-suite lanes (Parameterized Queries, Single Inserts, Concurrent Single Inserts) stay neutral, so exp 179's small-bind finding stands — the encoder is now the right default for the single-row path because we have a representative workload where it matters, not because the small case stopped being flat. Exp 187 consumes exp 186's UTF-8-heavy follow-up by adding byte-matched CJK rows to the same harness and reusing the batch direct-UTF-8 writer for non-ASCII single-row strings; CJK improves roughly 31-39% from 16 KB through 1 MB across the order-flipped pair while the exp 186 all-ASCII fast path stays intact.", - "keyPriors": ["109", "125", "126", "149", "186", "187"], - "archive": ["028", "076", "077", "095", "096", "112", "116", "142", "146", "150"], + "keyPriors": [ + "109", + "125", + "126", + "149", + "186", + "187" + ], + "archive": [ + "028", + "076", + "077", + "095", + "096", + "112", + "116", + "142", + "146", + "150" + ], "interestingIf": [ "the idea changes ownership/layout enough to remove native allocation or copying", "a benchmark shows parameter encoding as a material part of write wall time", @@ -75,10 +130,23 @@ { "id": "long-text-stream-hashing", "status": "settled", - "subsystems": ["streaming", "hashing", "text"], + "subsystems": [ + "streaming", + "hashing", + "text" + ], "currentRead": "Native hashing is valuable, and long TEXT/BLOB cells now have four representative unchanged-fanout signals: exp 110's 4 KB TEXT shape, exp 172's mixed 32 KB TEXT + 32 KB BLOB release row, exp 173's 32 KB long-text focused harness, and exp 181's one-reader single-stream 64 KB TEXT + 64 KB BLOB harness. Exp 173 showed the unrolled 16-byte FNV body was +4.5% and +12.1% versus exp 110's 8-byte fold on the pool-of-4 32 KB fanout; exp 181 removed reader-pool parallelism and still measured flat across an order-flipped pair (baseline 2.771/2.777 ms, candidate 2.763/2.792 ms). The 8-byte fold remains the correct implementation. Further FNV loop unrolling is settled off unless a production profile or direct resqlite_query_hash microbenchmark proves the byte fold itself, not SQLite value access, reader dispatch, or reply delivery, is again dominant.", - "keyPriors": ["075", "099", "110", "172", "173", "181"], - "archive": ["033"], + "keyPriors": [ + "075", + "099", + "110", + "172", + "173", + "181" + ], + "archive": [ + "033" + ], "interestingIf": [ "a production profile shows long TEXT/BLOB stream hashing remains hot after chunked folding", "a workload isolates the hash loop from reader-pool parallelism (single-stream long-payload, direct FFI microbenchmark)", @@ -96,10 +164,23 @@ { "id": "sqlite-version-and-build-config", "status": "watch", - "subsystems": ["sqlite", "sqlite3mc", "build", "planner"], + "subsystems": [ + "sqlite", + "sqlite3mc", + "build", + "planner" + ], "currentRead": "SQLite compile/config changes have produced real wins, but version bumps need audit discipline because planner and text-format behavior can shift under the library. Exp 144 bumped sqlite3mc 2.3.2 → 2.3.5 (SQLite 3.51.3 → 3.53.2) once SQLite shipped its `.2` point release and sqlite3mc cut a tracking release, satisfying exp 090's revisit trigger exactly. Tests stayed green including the embedded-NUL and Unicode bind regression suite. The single-pass release-suite A/B swung between 19 wins / 18 regressions / 124 neutral on the canonical run and 30 wins / 2 regressions / 129 neutral on the rerun — a wide spread characteristic of single-pass noise at sub-ms granularity. The only metric flagged consistently across reruns is Concurrent Reads 8× wall median (+~20% on a sub-ms metric); the 4× concurrency case wins on the same baseline, so it is not a generic read-pool slowdown. Soak window is the right place to confirm whether this is a real 3.53.x reader-pool interaction or single-run tail noise. The 3.53.0 FP-rounding default change (15→17 digits) was confirmed irrelevant because resqlite reads REAL via `sqlite3_column_double` and serialises with its own `snprintf(\"%.17g\", ...)` rather than `sqlite3_column_text`, so the proposed `SQLITE_DBCONFIG_FP_DIGITS=15` shim was skipped.", - "keyPriors": ["016", "044", "090", "144"], - "archive": ["020", "021"], + "keyPriors": [ + "016", + "044", + "090", + "144" + ], + "archive": [ + "020", + "021" + ], "interestingIf": [ "sqlite3mc tracks a stable SQLite point release with relevant planner or C API changes", "a platform-specific storage feature maps cleanly to resqlite defaults", @@ -124,10 +205,22 @@ { "id": "transaction-control-paths", "status": "low-current-signal", - "subsystems": ["writer", "transactions", "sqlite"], + "subsystems": [ + "writer", + "transactions", + "sqlite" + ], "currentRead": "Cached top-level transaction control statements won. Nested-transaction string/native helpers stayed flat even after exp 111 added a worst-case shallow-fan-out savepoint workload (50× SAVEPOINT/RELEASE per iteration): exp 102's cache pattern measured -9 % (within the ±17 % decision threshold). Exp 189 then tried the smaller savepoint naming-compression variant: reusing same-name cached SQL (`SAVEPOINT s`, `RELEASE s`, `ROLLBACK TO s`) produced best-case focused wins on empty fanout (~-6%), rollback fanout (-17% / -13%), and repeated deep chains (-7% / -8%), but the representative nested-write fanout failed to reproduce (-1.3%, then +21.6% slower). Per-isolate-round-trip cost dominates per-savepoint allocation/naming savings, so string/naming work is closed.", - "keyPriors": ["101", "102", "111", "189"], - "archive": ["027", "103"], + "keyPriors": [ + "101", + "102", + "111", + "189" + ], + "archive": [ + "027", + "103" + ], "interestingIf": [ "a change collapses multiple savepoint open/close operations into a single isolate round-trip (analogous to exp 009's read-side batching)", "a profile mode shows savepoint boundary round-trips, not string allocation, as a measurable spike under a realistic workload", @@ -153,10 +246,30 @@ { "id": "result-transfer-shape", "status": "watch", - "subsystems": ["results", "isolate-transfer", "api-shape"], + "subsystems": [ + "results", + "isolate-transfer", + "api-shape" + ], "currentRead": "The current ResultSet/Row shape is close to optimal for the shipped select() contract. Alternatives often move work rather than remove it, especially once main-isolate consumption is measured. Exp 158 found a narrow exception inside the existing shape: adding a schema-name identity fast path for schemas up to 32 columns plus private HashMap fallback for RowSchema.indexOf roughly halved focused row facade lookup and select_maps main-isolate full-consumption medians without changing transfer or public API, while point-query schema construction stayed neutral/noisy. Exp 167 rechecked closed exp 141's direct ResultSet.forEach override on a real SQLite-backed consumer lane after exp 158 and rejected it: a small first-pair win reversed on the longer confirmation pair, so no runtime code was kept. Exp 174 found a transport asymmetry: the reader 'sacrifice' path (Isolate.exit + reader respawn) is a real win for the rows path because it transfers already-built Dart objects with no re-copy, but it was applied by result size to selectBytes too, where the native JSON must be Uint8List.fromList-copied before Isolate.exit can transfer it — so sacrifice saved zero copies on bytes and only added a respawn, while the non-sacrifice bytes path copied twice. selectBytes now sends a Uint8List view over the connection's persistent json_buf and never sacrifices: -44% (~1.8x) on large (>256KB) byte reads by eliminating the respawn, -4% on small, at a bounded ~+15MB RSS high-water (readers no longer respawned). Exp 175 adds a named release-suite guard for that large-bytes path: `Large payload (~650KB) / resqlite selectBytes()` measured 0.323 ms wall / 0.000 ms main and is curated as `selectBytes() large bytes`, so the history no longer relies on the sub-256KB 1K-row metric to watch exp 174. Rows select() keeps sacrifice — there the zero-copy object transfer is real. Exp 176 closed a gap exp 158 left inside the same RowSchema index: Row.containsKey still hashed the key in the private HashMap on every call, bypassing the identity fast path its sibling operator[] already used, so it ran ~+3.6 ms slower than a LinkedHashMap on the focused containsKey lane. Routing it through a shared RowSchema.containsName (= indexOf(name) >= 0) improved that lane ~13.0 -> ~10.0 ms (-23%) with a flat hot-lookup control, flipping Row to at-parity, behavior-identical and no API change. The win is interned-key-specific (decoded schema names are not identical to user literals, so production probes generally fall through to the HashMap, same cost as before). Exp 193 rejected replacing Row.values' custom iterator with a fixed ListBase slice view: JIT row_map_facade values samples were unstable, and the AOT check showed the original _RowValueIterator faster (2.663-2.687 ms) than the list view (6.828-7.101 ms), so Row.values should keep the custom iterator unless a future Dart runtime changes compiled behavior. Exp 183 closed the remaining piece of exp 174's bounded RSS trade-off by quantifying it and reclaiming it: a new Diagnostics.readerJsonBufHighWaterBytes field exposes per-reader json_buf.cap, the focused json_buf_retention.dart audit confirms pathological retention is real (8 concurrent x 8 MB selectBytes pin 32 MB across the 4-reader pool for the rest of the connection), and a C-side reader-worker shrink fired after SendPort.send returns (gated by cap > 1 MB AND last_used_len < 256 KB) reclaims back to the 16 KB initial cap on subsequent small reads — post-burst settle 32 MB -> 64 KB, recurring-large 16 MB -> 64 KB, with neutral large_bytes_transfer.dart numbers. Exp 185 promotes that diagnostic into the release diagnostics suite: `SQLite Diagnostics / JSON buffer reclaim (8 large selectBytes + 64 small settles)` now records `jsonBufKiB` and fails if the post-settle high-water exceeds 512 KiB; the first focused suite run settled at 64.0 KiB with idle readers, so the exp 183 reclaim has public regression visibility. Exp 190 takes the encoder-side win inside the same direction: `write_json_to_buf` in `native/resqlite.c` now pre-builds each column's `\"col\":` / `,\"col\":` token once at first-row time into a per-query scratch buffer, so subsequent rows emit each column with one `buf_write` instead of comma + `json_write_string` (SWAR scan + escape walk) + colon. Focused `select_bytes_wide_cols.dart` measures -4% to -11% across two order-flipped passes on 10k-row x 8 / 20-col shapes; `large_bytes_transfer.dart` (exp 174's focused guard) also moves -8.7% / -8.2% on large/small lanes. Regression guards (1 row, 100 rows) stay in the sub-microsecond noise floor. Exp 192 closes the bounded headroom exp 023 left inside `write_json_to_buf`'s `SQLITE_INTEGER` arm: replacing the single-digit `fast_i64_to_str` body with a two-digit `[00..99]` lookup table (one `% 100` / `/ 100` and one 2-byte memcpy per digit pair) cuts focused integer-heavy selectBytes by −8 to −26 % across two order-flipped passes on `select_bytes_int_heavy.dart` — biggest win on 10k × 20 ~18-digit big ints (−24 to −26 %), where the digit-loop length is greatest. Mixed-cell and small-payload regression guards stay inside ±1 %. The release suite is not the right denominator (no lane is integer-heavy enough), so `select_bytes_int_heavy.dart` is the durable gate for future selectBytes integer-encode work. Exp 195 promotes exp 190's per-query `tokens_buf` scratch into the `resqlite_cached_stmt` entry, amortizing the per-query `buf_init(64)` + `free` pair and the first-row pre-encode walk across every re-execution of the same prepared SQL. Per-row inner loop and JSON output are byte-identical to exp 190. The exp 190 1-row regression guard sits at the millisecond-reporting harness floor and looked neutral; a new microsecond-precision focused harness `select_bytes_repeated_calls.dart` (1000 calls per sample) measures the predicted shape directly — 1-row × 20-col improves −9.2 % / −7.2 % and 10-row × 20-col improves −5.1 % / −2.7 % across two order-flipped passes, while 100/1000-row guards show sign reversal (drift-suspected per exp 177's classifier; per-query setup is < 0.2 % of wall there). Exp 190's `wide_cols.dart` 10k-row shapes also trend candidate-faster on every lane across both passes. Memory cost per cached statement is `O(col_count * 8 + name_byte_count)` capped by `STMT_CACHE_MAX = 32` per connection.", - "keyPriors": ["158", "174", "176", "183", "192", "195"], - "archive": ["008", "063", "066", "081", "082", "089", "190", "193"], + "keyPriors": [ + "158", + "174", + "176", + "183", + "192", + "195" + ], + "archive": [ + "008", + "063", + "066", + "081", + "082", + "089", + "190", + "193" + ], "interestingIf": [ "Dart adds new deeply immutable or transfer primitives that support typed data and lists", "a change preserves the lean API while removing end-to-end work", @@ -180,10 +293,32 @@ { "id": "measurement-system", "status": "active", - "subsystems": ["benchmarking", "profiling", "methodology"], + "subsystems": [ + "benchmarking", + "profiling", + "methodology" + ], "currentRead": "Several plausible optimizations failed because the benchmark did not stress the target path or because run noise hid small effects. Measurement work can be the highest-signal experiment when it unlocks a named implementation or rejection decision, but scheduled runners should first look for an instrument-and-implement path. Exp 119/121/136/147 are the recent stream-dispatch examples: exp 119 located surviving dispatch pressure in stream admission, exp 121 ruled out invalidation traversal as a wall-time target (10–15% of overlap wall, intersection ~3–6%), exp 136 added the completion-side reader-handler counter plus drain-aware audit snapshots and found 28.57% of A11c overlap total wall in the reader worker port handler chain at ~18 us per call, and exp 147 added the writer SQLite wall split and found SQLite-facing writer calls are not the active stream-fanout bottleneck on A11c overlap or keyed-PK. Exp 143 confirmed the pinned Tracelite profile path is useful because it captures dispatch floors, floor-subtracted work, memory diagnostics, allocation counters, source provenance, and graph data in one run; exp 169 consumes its interpretation gap by validating that `tracelite explain` emits workload-summary insight IDs for dispatch floors, work-bound operations, tail spread, RSS, allocation, and WAL signal before the profile wrapper completes. Exp 161 closes the matching release-suite gap for the writer side by promoting exp 159's concurrent-burst shape into `benchmark/suites/writes.dart` as a paired Single Inserts (sequential) / Concurrent Single Inserts (concurrent) row pair; the resqlite concurrent median (~1.1 ms) is now ~60% below the sequential median (~2.9 ms) on a public lane, and future writer-scheduling experiments can claim release-suite wins without depending on the focused `writer_pipelining.dart` script. Exp 177 mechanizes the JOURNAL's order-flipped drift check: `cvPct` + `classifyDriftFlag` in `benchmark/shared/stats.dart` and a `benchmark/ab_drift_check.dart` CLI classify a phase-ordered A/B regression flag as reproduced / drift-suspected / inconclusive from two order-flipped passes of per-run values, reproducing the manual verdicts on the recorded exp 159 (CV asymmetry) and exp 167 (sign reversal) flags. It is methodology tooling (exp 161 / 169 class), not a wall-time change, so future A/B runners can cite a deterministic verdict instead of re-deriving the CV-asymmetry rule each time. Exp 178 closes a silent pipeline gap on the experiment->chart linker itself: `generate_history.dart` already failed the build when an Accepted experiment linked a baseline-shaped run while a candidate existed (`_assertAcceptedExperimentsLinkToCandidates`), but it tolerated the more common case — a chartable experiment with NO linked run and NO `**Benchmark Run:**` opt-out, which silently drops off the chart when a runner forgets the result file or mismatches its date. A structural tally on main showed only ~5 of 23 null-run accepted/in-review experiments declared the opt-out. Exp 178 adds `_assertNewExperimentsLinkOrDeclareRun` (over a pure `findUndeclaredMissingRunExperiments` detector) that fails the build for accepted/in-review experiments numbered >= 178 with a null run and no opt-out declaration; pre-178 experiments are grandfathered via a cutoff constant (same pattern as `experimentEntriesRequiredFrom`). No runtime code, history.json unchanged.", - "keyPriors": ["136", "143", "147", "169", "177", "178"], - "archive": ["055", "088", "099", "102", "108", "113", "115", "116", "119", "121"], + "keyPriors": [ + "136", + "143", + "147", + "169", + "177", + "178" + ], + "archive": [ + "055", + "088", + "099", + "102", + "108", + "113", + "115", + "116", + "119", + "121" + ], "interestingIf": [ "a profiler or focused benchmark exposes a cost hidden by the release suite", "a control metric can identify noisy runs before results are trusted", @@ -205,7 +340,9 @@ ], "experiments": { "088": { - "directions": ["measurement-system"], + "directions": [ + "measurement-system" + ], "outcomeClass": "rejected_after_noise_check", "changedBeliefs": [ "Single-run tail wins need confirmation against control metrics before acceptance" @@ -216,7 +353,9 @@ ] }, "090": { - "directions": ["sqlite-version-and-build-config"], + "directions": [ + "sqlite-version-and-build-config" + ], "outcomeClass": "watch", "changedBeliefs": [ "sqlite3mc version bumps should wait for safer point releases when the newest SQLite release is a fresh .0" @@ -227,7 +366,10 @@ ] }, "099": { - "directions": ["long-text-stream-hashing", "measurement-system"], + "directions": [ + "long-text-stream-hashing", + "measurement-system" + ], "outcomeClass": "benchmark_gap", "changedBeliefs": [ "Byte-stream hash-loop changes need long-text stream workloads before they are meaningfully testable" @@ -237,7 +379,9 @@ ] }, "100": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_regression", "changedBeliefs": [ "Bounded rerun scheduling can harm high-cardinality stream fan-out even if it looks attractive for unrelated reads" @@ -247,7 +391,9 @@ ] }, "101": { - "directions": ["transaction-control-paths"], + "directions": [ + "transaction-control-paths" + ], "outcomeClass": "accepted", "changedBeliefs": [ "Top-level transaction control still had removable SQLite prepare/finalize work" @@ -257,7 +403,9 @@ ] }, "103": { - "directions": ["transaction-control-paths"], + "directions": [ + "transaction-control-paths" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "Native nested transaction helpers add complexity without a stable realistic-workload win" @@ -267,7 +415,9 @@ ] }, "105": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_regression", "changedBeliefs": [ "Increasing reader worker count can worsen writer throughput by increasing completion-side microtask churn" @@ -277,7 +427,9 @@ ] }, "108": { - "directions": ["measurement-system"], + "directions": [ + "measurement-system" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "Persistent selectBytes out-parameter slots are below the current benchmark and RSS signal" @@ -287,7 +439,9 @@ ] }, "109": { - "directions": ["parameter-encoding-and-binding"], + "directions": [ + "parameter-encoding-and-binding" + ], "outcomeClass": "accepted", "changedBeliefs": [ "Parameter encoding still has worthwhile headroom when the change removes per-text/blob native allocations and SQLite strlen work" @@ -297,7 +451,10 @@ ] }, "110": { - "directions": ["long-text-stream-hashing", "measurement-system"], + "directions": [ + "long-text-stream-hashing", + "measurement-system" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The missing long-text unchanged-fanout benchmark was the blocker for evaluating byte-stream hash-loop work", @@ -309,7 +466,10 @@ ] }, "111": { - "directions": ["transaction-control-paths", "measurement-system"], + "directions": [ + "transaction-control-paths", + "measurement-system" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "Even on a worst-case shallow-fan-out workload (50 SAVEPOINTs per iteration), the savepoint string allocation cost is below the per-benchmark decision threshold — per-isolate-round-trip cost dominates", @@ -321,7 +481,10 @@ ] }, "112": { - "directions": ["parameter-encoding-and-binding", "measurement-system"], + "directions": [ + "parameter-encoding-and-binding", + "measurement-system" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "Pre-sizing the temporary executeBatch flat parameter list is below the current benchmark signal", @@ -333,7 +496,10 @@ ] }, "113": { - "directions": ["parameter-encoding-and-binding", "measurement-system"], + "directions": [ + "parameter-encoding-and-binding", + "measurement-system" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The two-parameter batch insert shape is not a sufficient proxy for batch parameter overhead", @@ -345,7 +511,9 @@ ] }, "114": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "Reader-pool wake amplification (single shared completer waking every parked dispatcher) was a real but workload-dependent cost: -10 % to -32 % on streaming fan-out paths against pre-exp-106 main, then collapsed into noise once exp 106 polish landed and elided most stream re-queries on the writer side", @@ -358,7 +526,10 @@ ] }, "115": { - "directions": ["measurement-system", "stream-rerun-dispatch"], + "directions": [ + "measurement-system", + "stream-rerun-dispatch" + ], "outcomeClass": "accepted_measurement", "changedBeliefs": [ "The parked-dispatcher path inside `ReaderPool._dispatch` is now directly observable via `ProfileCounters.dispatcherParkedTotal`, `dispatcherWakeRetryTotal`, and `dispatcherMaxParkedConcurrent` — gated behind `kProfileMode` so release builds stay zero-cost", @@ -370,7 +541,10 @@ ] }, "116": { - "directions": ["parameter-encoding-and-binding", "measurement-system"], + "directions": [ + "parameter-encoding-and-binding", + "measurement-system" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The release write suite now covers parameter width directly instead of relying on the two-parameter batch insert as the only public batch proxy" @@ -380,7 +554,9 @@ ] }, "117": { - "directions": ["parameter-encoding-and-binding"], + "directions": [ + "parameter-encoding-and-binding" + ], "outcomeClass": "deferred", "changedBeliefs": [ "Named-parameter support is functionally viable but too invasive for the v0.x launch path unless real user demand justifies widening the public parameter API", @@ -392,7 +568,9 @@ ] }, "118": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The exp 115 counters convert exp 114's previously ambiguous FIFO waiter idea into a direct measurement: under overloaded reads, wake retries drop from 6/66/378 to 0 at concurrency 8/16/32 while max parked depth remains unchanged", @@ -404,7 +582,10 @@ ] }, "119": { - "directions": ["stream-rerun-dispatch", "measurement-system"], + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], "outcomeClass": "in_review_measurement", "changedBeliefs": [ "Post-FIFO app-shaped workloads keep `dispatcherWakeRetryTotal` at zero, so ReaderPool wake policy is no longer the active dispatch target", @@ -416,7 +597,9 @@ ] }, "120": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The 3,590-park / 46-max signal exp 119 measured on A11c overlap was upstream over-dispatch from `StreamEngine._flushQueue`, not reader-pool admission proper. Snapshotting `availableWorkerCount` once and decrementing per pop drops parking to zero on overlap and keyed-PK, with disjoint unchanged and high-cardinality fan-out (the exp-100 killer) within ±10% noise", @@ -429,7 +612,10 @@ ] }, "121": { - "directions": ["stream-rerun-dispatch", "measurement-system"], + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], "outcomeClass": "in_review_measurement", "changedBeliefs": [ "Under a corrected wall convention (stopwatch stops on the last write, not after a fixed drain), invalidation traversal is at the *edge* of the wall-time noise floor on A11c overlap (10–15% of wall, intersection 2.5–5.7%) and keyed-PK (13.5–13.9%, intersection ~4%). The earlier ~7%/~1.6% figures were biased low by an arbitrary 50 ms drain sleep included in the wall denominator", @@ -444,7 +630,10 @@ ] }, "122": { - "directions": ["stream-rerun-dispatch", "measurement-system"], + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "Exp 120 fixed the per-flush over-dispatch signal; exp 122 removes the remaining async handoff by constructing `StreamEngine` with a concrete `ReaderPool`, so `_flushQueue` stays synchronous and admitted reads reach pool dispatch without awaiting a pool future", @@ -455,8 +644,38 @@ "look for keyed/row-level invalidation or observer APIs to reduce keyed-PK miss-path work beyond what admission accuracy can solve" ] }, + "125": { + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Large wide ASCII-heavy batches still had removable per-string UTF-8 allocation after exp 113's matrix encoder", + "A guarded ASCII fast path can improve wide batch parameter packing without changing the public API or weakening Unicode fallback behavior" + ], + "nextSignals": [ + "watch release-suite Wide Batch Insert and narrow Batch Insert together", + "benchmark non-ASCII-heavy wide batches before broadening the fallback path" + ] + }, + "126": { + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Large wide non-ASCII batches have the same removable per-string allocation pattern exp 125 found for ASCII: direct UTF-8 payload writing improves focused Unicode 10k x20 by 13.5% and emoji 10k x20 by 27.8%", + "Dart-compatible surrogate-pair and replacement-character encoding can stay private to the guarded batch encoder while preserving embedded-NUL byte lengths through sqlite3_bind_text" + ], + "nextSignals": [ + "watch release-suite Wide Batch Insert and narrow Batch Insert together because the public suite is still ASCII-heavy", + "only pursue blob-heavy or broader embedded-NUL work with a workload that crosses the same large/wide guard" + ] + }, "134": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_preserved_potential", "changedBeliefs": [ "Row-level dirty precision can remove the keyed-PK miss-path cost: verified simple `WHERE id = ?` INTEGER PRIMARY KEY streams skipped when dirty rowids did not overlap", @@ -468,8 +687,28 @@ "revive row-level dependency precision only if a real workload shows keyed-PK miss writes dominate or the API/design can express watched row identity directly" ] }, + "136": { + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "On A11c overlap the reader-worker port handler accounts for 28.57% of total wall (burst + drain) at ~18 µs per call across 4,228 calls per burst — the largest remaining reachable main-isolate slice", + "Subscriber-fanout emit is < 1% of the completion chain on every measured workload; batching `controller.add` is not the optimization target", + "Most reader replies on A11c overlap short-circuit via `selectIfChanged` hash comparison (4,228 completions → 29 emits in the fresh pass); the per-call cost is handler bootstrap + Future resolution + hash check + `_flushQueue` admit, not real query result work", + "Most reader-completion wall fires AFTER the writer-burst stopwatch stops; the shared `audit_workloads.dart` now snapshots counters BOTH at burst-end and after the drain so future main-isolate audits can pick the right denominator. A11c drain switched from a fixed 50 ms wait to the same quiet-window pattern keyed-PK already uses" + ], + "nextSignals": [ + "evaluate reader-reply batching as the bounded implementation candidate that follows: must drop `completion_us / total_us` on A11c overlap AND stay neutral on A11c disjoint and keyed-PK (otherwise overall release-suite delta is workload-specific)", + "treat `stream_emit_us` as evidence-of-absence for subscriber-fanout optimization candidates until a workload with very many listeners per stream surfaces", + "future main-isolate counters should follow the exp 136 pattern (counter + post-drain snapshot in `audit_workloads.dart`) rather than redoing the harness wiring" + ] + }, "142": { - "directions": ["parameter-encoding-and-binding"], + "directions": [ + "parameter-encoding-and-binding" + ], "outcomeClass": "rejected_no_signal", "changedBeliefs": [ "Direct single-row text parameter encoding did not produce current Tracelite production evidence despite the old focused-harness win from PR #130", @@ -483,7 +722,9 @@ ] }, "143": { - "directions": ["measurement-system"], + "directions": [ + "measurement-system" + ], "outcomeClass": "in_review_measurement", "changedBeliefs": [ "Pinned Tracelite profile runs already produce decision-useful structured evidence: dispatch floors, floor-subtracted work, operation tails, memory deltas, allocation counters, source provenance, and validated graph data", @@ -497,7 +738,9 @@ ] }, "144": { - "directions": ["sqlite-version-and-build-config"], + "directions": [ + "sqlite-version-and-build-config" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "Once SQLite ships a `.2` point release and sqlite3mc cuts a tracking release, exp 090's vendoring-bump audit moves from `defer` to `do` without any further policy debate — the 3.53.x line is the canonical case", @@ -512,7 +755,9 @@ ] }, "145": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "`StreamEngine._flushQueue` helper allocation is not the active stream-dispatch target after exp 120 and exp 122; replacing `take(...).toList()` and `where(...).length` with inline loops kept dispatch counters at zero but produced mixed wall-time movement", @@ -525,8 +770,41 @@ "use the existing A11c/keyed-PK profile counter gates to reject stream-admission cleanups whose counters are already zero" ] }, + "146": { + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "rejected_no_signal", + "changedBeliefs": [ + "Lowering the ASCII batch-packing guard from 8 params / 8192 total params to 2 params / 64 total params did not improve the Tracelite narrow-batch-insert primary lane", + "The clean A/B run measured resqlite at +1.45% with neutral verdict and 13.2% max CV, while the sqlite_async guardrail was too noisy to add confidence", + "Small and narrow batch writes should stay on the generic path unless a future workload shows parameter encoding is a material part of wall time" + ], + "nextSignals": [ + "do not broaden the exp 125 ASCII fast path to small/narrow batches without a new workload and a Tracelite A/B decision that clears the primary gate", + "when experimenting in this direction, prefer the integrated Tracelite A/B wrapper so baseline/candidate histories, policy, decision, insights, and graph data stay together" + ] + }, + "147": { + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "SQLite-facing write calls are a minority of writer-side burst wall on active stream workloads: A11c overlap is 9.4% SQLite and keyed-PK is 18.1% SQLite", + "After subtracting SQLite-facing calls and stream invalidation, residual writer/request wall remains the largest bucket on A11c overlap and keyed-PK", + "Future stream-dispatch work should target completion-side scheduling, reply/request coordination, or dirty-set harvest rather than SQLite-step tuning, and should carry any narrow measurement inside the implementation branch" + ], + "nextSignals": [ + "reader-reply batching candidate should reduce exp 136 completion_us / total_us on A11c overlap", + "if residual writer/request detail is needed, gather only the split required by a concrete reduction candidate and remove temporary scaffolding before merge unless it is reusable" + ] + }, "148": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "Plain worker-side reader-reply batching reduces completion callback counters but does not produce a mergeable measured-elapsed win on the stream-dispatch Tracelite suite", @@ -540,7 +818,9 @@ ] }, "149": { - "directions": ["parameter-encoding-and-binding"], + "directions": [ + "parameter-encoding-and-binding" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "Tracelite profile merge rounds expose a material six-parameter batch shape between exp 146's rejected small/narrow guard and exp 125/126's large wide guard", @@ -554,7 +834,9 @@ ] }, "150": { - "directions": ["parameter-encoding-and-binding"], + "directions": [ + "parameter-encoding-and-binding" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The first-row string probe missed nullable generated-statement batches where row 0 carries NULL text columns and later rows carry strings", @@ -567,25 +849,10 @@ "if blob-heavy parameter shapes become interesting, benchmark them separately because this run targeted text-nullability rather than blob-only packing" ] }, - "158": { - "directions": ["result-transfer-shape"], - "outcomeClass": "in_review_accepted", - "changedBeliefs": [ - "A narrow private RowSchema lookup change can still improve full row consumption without changing the ResultSet/Row public API", - "The schema-name identity fast path plus HashMap fallback cut row_map_facade hot lookup from 10.750 -> 5.136 ms and keys+lookup from 20.602 -> 10.650 ms", - "A local width sweep showed the identity scan still ahead at 32 columns but behind by 48 columns, so the fast path is capped at 32 columns and unusually wide selects fall straight through to the HashMap", - "Focused select_maps main-isolate medians improved from 0.169 -> 0.081 ms at 1K rows and 1.998 -> 0.967 ms at 10K rows in the clean paired pass", - "Point-query throughput stayed neutral/noisy with overlapping confidence intervals, so the per-query schema construction cost did not show a hot single-row select regression" - ], - "nextSignals": [ - "keep result-shape experiments on full consumer benchmarks such as select_maps, not transfer/setup-only measurements", - "do not infer from exp 158 that larger ResultSet API changes are attractive; this win is limited to the private schema index", - "do not raise the 32-column identity-scan threshold without a fresh width sweep and a full-consumption benchmark covering wider schemas", - "watch release select_maps metrics during soak because the local 10K runs showed visible machine noise even though the clean paired pass favored the candidate" - ] - }, "151": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "Changing writer response futures from `Completer()` to `Completer.sync()` is a coherent request-resolution implementation attempt, but it does not clear the Tracelite stream-dispatch primary gate", @@ -598,35 +865,29 @@ "use integrated Tracelite A/B primary gates before keeping stream scheduling changes, even when a profile smoke looks mixed or partially favorable" ] }, - "164": { - "directions": ["stream-rerun-dispatch", "measurement-system"], - "outcomeClass": "rejected_no_stable_signal", - "changedBeliefs": [ - "SQLite EXPLAIN QUERY PLAN can avoid a custom SQL parser for rowid lookup detection, but it did not make private row-level stream invalidation cheap enough to keep", - "The focused Tracelite stream-initial-drain rowid lane measured -2.33% with neutral/inconclusive evidence while text and indexed-int setup guardrails did not provide a clean offsetting signal", - "The broader stream-dispatch guard run stayed mostly neutral and did not turn the keyed-PK rowid idea into a stable end-to-end win" + "158": { + "directions": [ + "result-transfer-shape" ], - "nextSignals": [ - "do not keep native dirty-row harvesting plus plan inspection without a real workload or Tracelite primary lane that clearly pays for the extra dependency layer", - "use stream-initial-drain and warmup_elapsed_ns guardrails for future setup-heavy stream experiments", - "if row-level precision reopens, prefer a stronger dependency model over broadening resqlite-owned SQL recognition" - ] - }, - "146": { - "directions": ["parameter-encoding-and-binding"], - "outcomeClass": "rejected_no_signal", + "outcomeClass": "in_review_accepted", "changedBeliefs": [ - "Lowering the ASCII batch-packing guard from 8 params / 8192 total params to 2 params / 64 total params did not improve the Tracelite narrow-batch-insert primary lane", - "The clean A/B run measured resqlite at +1.45% with neutral verdict and 13.2% max CV, while the sqlite_async guardrail was too noisy to add confidence", - "Small and narrow batch writes should stay on the generic path unless a future workload shows parameter encoding is a material part of wall time" + "A narrow private RowSchema lookup change can still improve full row consumption without changing the ResultSet/Row public API", + "The schema-name identity fast path plus HashMap fallback cut row_map_facade hot lookup from 10.750 -> 5.136 ms and keys+lookup from 20.602 -> 10.650 ms", + "A local width sweep showed the identity scan still ahead at 32 columns but behind by 48 columns, so the fast path is capped at 32 columns and unusually wide selects fall straight through to the HashMap", + "Focused select_maps main-isolate medians improved from 0.169 -> 0.081 ms at 1K rows and 1.998 -> 0.967 ms at 10K rows in the clean paired pass", + "Point-query throughput stayed neutral/noisy with overlapping confidence intervals, so the per-query schema construction cost did not show a hot single-row select regression" ], "nextSignals": [ - "do not broaden the exp 125 ASCII fast path to small/narrow batches without a new workload and a Tracelite A/B decision that clears the primary gate", - "when experimenting in this direction, prefer the integrated Tracelite A/B wrapper so baseline/candidate histories, policy, decision, insights, and graph data stay together" + "keep result-shape experiments on full consumer benchmarks such as select_maps, not transfer/setup-only measurements", + "do not infer from exp 158 that larger ResultSet API changes are attractive; this win is limited to the private schema index", + "do not raise the 32-column identity-scan threshold without a fresh width sweep and a full-consumption benchmark covering wider schemas", + "watch release select_maps metrics during soak because the local 10K runs showed visible machine noise even though the clean paired pass favored the candidate" ] }, "159": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The writer request path carried removable fixed cost on every round-trip: a per-request RawReceivePort, a guaranteed microtask hop awaiting the already-resolved worker-port future, and async reply completers — replacing them with a persistent reply port, cached SendPort, and sync FIFO completers drops exp 147 residual_us on all four audit workloads", @@ -639,60 +900,11 @@ "when a tracelite gate flags a regression with elevated within-run CVs relative to the other phase, re-run with collection order flipped before treating the flag as real" ] }, - "147": { - "directions": ["stream-rerun-dispatch", "measurement-system"], - "outcomeClass": "in_review_measurement", - "changedBeliefs": [ - "SQLite-facing write calls are a minority of writer-side burst wall on active stream workloads: A11c overlap is 9.4% SQLite and keyed-PK is 18.1% SQLite", - "After subtracting SQLite-facing calls and stream invalidation, residual writer/request wall remains the largest bucket on A11c overlap and keyed-PK", - "Future stream-dispatch work should target completion-side scheduling, reply/request coordination, or dirty-set harvest rather than SQLite-step tuning, and should carry any narrow measurement inside the implementation branch" - ], - "nextSignals": [ - "reader-reply batching candidate should reduce exp 136 completion_us / total_us on A11c overlap", - "if residual writer/request detail is needed, gather only the split required by a concrete reduction candidate and remove temporary scaffolding before merge unless it is reusable" - ] - }, - "125": { - "directions": ["parameter-encoding-and-binding"], - "outcomeClass": "in_review_accepted", - "changedBeliefs": [ - "Large wide ASCII-heavy batches still had removable per-string UTF-8 allocation after exp 113's matrix encoder", - "A guarded ASCII fast path can improve wide batch parameter packing without changing the public API or weakening Unicode fallback behavior" - ], - "nextSignals": [ - "watch release-suite Wide Batch Insert and narrow Batch Insert together", - "benchmark non-ASCII-heavy wide batches before broadening the fallback path" - ] - }, - "126": { - "directions": ["parameter-encoding-and-binding"], - "outcomeClass": "in_review_accepted", - "changedBeliefs": [ - "Large wide non-ASCII batches have the same removable per-string allocation pattern exp 125 found for ASCII: direct UTF-8 payload writing improves focused Unicode 10k x20 by 13.5% and emoji 10k x20 by 27.8%", - "Dart-compatible surrogate-pair and replacement-character encoding can stay private to the guarded batch encoder while preserving embedded-NUL byte lengths through sqlite3_bind_text" - ], - "nextSignals": [ - "watch release-suite Wide Batch Insert and narrow Batch Insert together because the public suite is still ASCII-heavy", - "only pursue blob-heavy or broader embedded-NUL work with a workload that crosses the same large/wide guard" - ] - }, - "136": { - "directions": ["stream-rerun-dispatch", "measurement-system"], - "outcomeClass": "in_review_measurement", - "changedBeliefs": [ - "On A11c overlap the reader-worker port handler accounts for 28.57% of total wall (burst + drain) at ~18 µs per call across 4,228 calls per burst — the largest remaining reachable main-isolate slice", - "Subscriber-fanout emit is < 1% of the completion chain on every measured workload; batching `controller.add` is not the optimization target", - "Most reader replies on A11c overlap short-circuit via `selectIfChanged` hash comparison (4,228 completions → 29 emits in the fresh pass); the per-call cost is handler bootstrap + Future resolution + hash check + `_flushQueue` admit, not real query result work", - "Most reader-completion wall fires AFTER the writer-burst stopwatch stops; the shared `audit_workloads.dart` now snapshots counters BOTH at burst-end and after the drain so future main-isolate audits can pick the right denominator. A11c drain switched from a fixed 50 ms wait to the same quiet-window pattern keyed-PK already uses" - ], - "nextSignals": [ - "evaluate reader-reply batching as the bounded implementation candidate that follows: must drop `completion_us / total_us` on A11c overlap AND stay neutral on A11c disjoint and keyed-PK (otherwise overall release-suite delta is workload-specific)", - "treat `stream_emit_us` as evidence-of-absence for subscriber-fanout optimization candidates until a workload with very many listeners per stream surfaces", - "future main-isolate counters should follow the exp 136 pattern (counter + post-drain snapshot in `audit_workloads.dart`) rather than redoing the harness wiring" - ] - }, "161": { - "directions": ["stream-rerun-dispatch", "measurement-system"], + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], "outcomeClass": "in_review_measurement", "changedBeliefs": [ "The exp 159 send-gated writer-lock pipelining shape is now visible on the release write suite as a paired Single Inserts (100 sequential) / Concurrent Single Inserts (100 concurrent) row pair using the same schema and parameter values", @@ -705,8 +917,27 @@ "if a future change moves the sequential row but not the concurrent row (or vice versa), record that asymmetry rather than treating it as calibration drift" ] }, + "164": { + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "rejected_no_stable_signal", + "changedBeliefs": [ + "SQLite EXPLAIN QUERY PLAN can avoid a custom SQL parser for rowid lookup detection, but it did not make private row-level stream invalidation cheap enough to keep", + "The focused Tracelite stream-initial-drain rowid lane measured -2.33% with neutral/inconclusive evidence while text and indexed-int setup guardrails did not provide a clean offsetting signal", + "The broader stream-dispatch guard run stayed mostly neutral and did not turn the keyed-PK rowid idea into a stable end-to-end win" + ], + "nextSignals": [ + "do not keep native dirty-row harvesting plus plan inspection without a real workload or Tracelite primary lane that clearly pays for the extra dependency layer", + "use stream-initial-drain and warmup_elapsed_ns guardrails for future setup-heavy stream experiments", + "if row-level precision reopens, prefer a stronger dependency model over broadening resqlite-owned SQL recognition" + ] + }, "167": { - "directions": ["result-transfer-shape"], + "directions": [ + "result-transfer-shape" + ], "outcomeClass": "rejected_no_stable_signal", "changedBeliefs": [ "Closed exp 141's direct ResultSet.forEach override does not show a stable current win after exp 158's RowSchema lookup change when measured on real rows returned by Database.select()", @@ -720,7 +951,9 @@ ] }, "169": { - "directions": ["measurement-system"], + "directions": [ + "measurement-system" + ], "outcomeClass": "in_review_measurement", "changedBeliefs": [ "The Tracelite workload-summary explanation layer now emits the interpretation exp 143 had to recover manually: dispatch floors, dispatch-bound point queries, work-bound merge rounds, tail spread, RSS movement, allocation counters, and WAL side effects", @@ -734,7 +967,9 @@ ] }, "170": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "A synchronous `Mutex.tryLock()` plus a non-`async` `Writer.execute` / `executeBatch` is a coherent request-side scheduling change — exactly the request-side counterpart to exp 151's response-side `Completer.sync()` attempt — but it does not move the primary sequential-write lane: Single Inserts (100 sequential) shifted +1.7 % (paired medians 3.099 ms -> 3.153 ms, 3.082 -> 2.982, 3.169 -> 3.501) and writer_pipelining `sequential-awaited (2000 writes)` shifted +2.0 % (34.047 -> 34.723), both within the run-to-run noise band and in the wrong direction", @@ -748,7 +983,9 @@ ] }, "171": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_below_current_signal", "changedBeliefs": [ "The `await _runtime` microtask hop at the `Database` layer above exp 159's `Writer._request` cache is real (~1-2 us per call, theoretical upper bound ~6-12% on a 2000-call sequential burst) but sits at or below the focused-harness noise floor: two order-flipped passes on writer_pipelining.dart produced alternating-sign deltas inside per-round variance (sequential-awaited -2.3%/+2.5%, transaction-guardrail -7.5%/+6.1%, concurrent-burst +4-5% both passes)", @@ -761,7 +998,10 @@ ] }, "172": { - "directions": ["long-text-stream-hashing", "measurement-system"], + "directions": [ + "long-text-stream-hashing", + "measurement-system" + ], "outcomeClass": "in_review_measurement", "changedBeliefs": [ "The streaming suite now covers mixed long payloads beyond exp 110's 4KB TEXT shape: 8 unchanged streams over 64 rows with 32KB TEXT plus 32KB BLOB cells", @@ -775,7 +1015,9 @@ ] }, "173": { - "directions": ["long-text-stream-hashing"], + "directions": [ + "long-text-stream-hashing" + ], "outcomeClass": "rejected_no_signal", "changedBeliefs": [ "At 32 KB cells on a pool-of-4 reader fleet the byte-stream fold is no longer the dominant wall component: even an unrolled 16-byte FNV body is +4.5 % to +12.1 % vs the exp 110 8-byte body across two order-flipped passes, all inside a 2.6 → 6.0 ms per-pass spread", @@ -789,7 +1031,9 @@ ] }, "174": { - "directions": ["result-transfer-shape"], + "directions": [ + "result-transfer-shape" + ], "outcomeClass": "in_review_win", "changedBeliefs": [ "The reader sacrifice path (Isolate.exit + reader respawn) is a win for the rows path (zero-copy transfer of already-built Dart objects) but counterproductive for selectBytes: native JSON must be Uint8List.fromList-copied before Isolate.exit can transfer it, so sacrifice saves zero copies on bytes and only adds a respawn; the non-sacrifice bytes path copied twice (fromList + SendPort)", @@ -804,7 +1048,10 @@ ] }, "175": { - "directions": ["result-transfer-shape", "measurement-system"], + "directions": [ + "result-transfer-shape", + "measurement-system" + ], "outcomeClass": "in_review_measurement", "changedBeliefs": [ "The release JSON-bytes suite now has an explicit large native-byte transfer row: `Large payload (~650KB) / resqlite selectBytes()` stays above the old 256KB sacrifice threshold and measured 0.323 ms wall / 0.000 ms main in the focused suite run", @@ -819,7 +1066,9 @@ ] }, "176": { - "directions": ["result-transfer-shape"], + "directions": [ + "result-transfer-shape" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "Exp 158 added the schema-name identity fast path to RowSchema.indexOf (serving Row.operator[]) but not to Row.containsKey, which still went straight to the private HashMap and hashed the key on every call. Exp 158's 'containsKey neutral' row compared Row vs LinkedHashMap with containsKey unchanged on both sides of that diff, so it was never evidence that routing containsKey through the identity path is neutral", @@ -833,7 +1082,9 @@ ] }, "177": { - "directions": ["measurement-system"], + "directions": [ + "measurement-system" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The JOURNAL's most-reapplied lesson — phase-ordered A/B gates confound code deltas with time-correlated drift — was enforced only by runner discipline. Every recent A/B writeup (exp 159, 167, 171, 173) re-derived the same CV-asymmetry + order-flip reasoning by hand. Encoding it as cvPct + classifyDriftFlag in benchmark/shared/stats.dart plus a benchmark/ab_drift_check.dart CLI makes the call deterministic and citable", @@ -846,7 +1097,9 @@ ] }, "178": { - "directions": ["measurement-system"], + "directions": [ + "measurement-system" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "The experiment->chart linker in generate_history.dart guarded the wrong-file case (_assertAcceptedExperimentsLinkToCandidates, the exp-109 mixup) but silently tolerated the more common missing-file case: a chartable experiment with no linked benchmark run AND no **Benchmark Run:** opt-out declaration drops off the chart with no CI error, which is exactly what a forgotten result file or a Date/timestamp mismatch produces", @@ -859,7 +1112,9 @@ ] }, "179": { - "directions": ["parameter-encoding-and-binding"], + "directions": [ + "parameter-encoding-and-binding" + ], "outcomeClass": "rejected_no_signal", "changedBeliefs": [ "Extending exp 125/149's direct-code-unit batch write to the single-row allocateParams path (the exp 142 / PR #130 idea) makes the encoder itself materially faster, not slower: a DB-free micro (single_row_param_packing.dart, 200k cycles x 15 samples) measures -45% / -58% / -37% on 1-short / 5-mixed / 1-KB ASCII shapes with a flat blob+int control. Exp 142's +6.86% / +16.4% 'slower' was workload/Tracelite-overhead confound, not the encoder", @@ -872,7 +1127,9 @@ ] }, "180": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "accepted", "changedBeliefs": [ "Cross-call write request batching (the 'group commit' lever exp 159 named) captures the per-round-trip residual that pipelining left: standalone execute() calls piling up while a send is in flight coalesce into one MultiExecuteRequest, and the release Concurrent Single Inserts lane improves -26% (baseline-first) / -32% (candidate-first), reproduced across order-flipped runs. exp 147's ~72% writer/request residual was attackable by collapsing N messages to ~2, not just by overlapping N messages (exp 159)", @@ -887,7 +1144,9 @@ ] }, "181": { - "directions": ["long-text-stream-hashing"], + "directions": [ + "long-text-stream-hashing" + ], "outcomeClass": "rejected_no_signal", "changedBeliefs": [ "The single-stream open candidate from exp 173 has been consumed: benchmark/experiments/single_stream_long_payload_hash.dart uses an internal one-reader runtime so one unchanged stream hashes 64 rows x 64 KB TEXT + 64 KB BLOB (~8 MB) before a queued COUNT(*) barrier can emit", @@ -901,7 +1160,9 @@ ] }, "182": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "rejected_no_stable_signal", "changedBeliefs": [ "Skipping the writer's preupdate_hook accumulation + reply harvest when _streamEngine.length == 0 at send time is a coherent residual-bucket attack — it removes per-row strcmp dedup + per-write FFI/toDartString/object work that no consumer reads, gated by a track_dirty flag on resqlite_db flipped from the writer isolate, with a DrainRequest no-op barrier from StreamEngine._createStream to fence in-flight non-tracking writes before a new stream's initial query", @@ -917,7 +1178,9 @@ ] }, "183": { - "directions": ["result-transfer-shape"], + "directions": [ + "result-transfer-shape" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "Exp 174's bounded RSS high-water (~+15 MB after 200 x 651 KB selectBytes) becomes pathological under a one-off concurrent-burst shape: 8 concurrent x 8 MB selectBytes pin 32 MB of native json_buf across the 4-reader pool for the rest of the connection's life, and the recurring-large pattern (1 large per 50 small over 300 iterations) settles at 16 MB pinned. The exp 174 future-note about high-threshold reclaim is a real candidate, not a hypothetical", @@ -932,7 +1195,9 @@ ] }, "184": { - "directions": ["stream-rerun-dispatch"], + "directions": [ + "stream-rerun-dispatch" + ], "outcomeClass": "accepted_measurement", "changedBeliefs": [ "Re-running exp 147's writer_sqlite_wall_audit on main after exp 159 + exp 180 shows the writer-burst breakdown essentially unchanged (A11c overlap SQLite 13.6% / invalidation 16.0% / residual 70.3% vs exp 147's 9.4 / 18.8 / 71.8). Expected: the audit issues writes sequentially (audit_workloads.dart:167) and exp 180 coalesces only concurrent bursts, so the sequential writer path it measures is unmoved", @@ -945,7 +1210,10 @@ ] }, "185": { - "directions": ["result-transfer-shape", "measurement-system"], + "directions": [ + "result-transfer-shape", + "measurement-system" + ], "outcomeClass": "in_review_measurement", "changedBeliefs": [ "Exp 183's json_buf reclaim now has release-suite visibility rather than only focused-audit evidence: SQLite Diagnostics emits a JSON buf (KiB) column and a dedicated `JSON buffer reclaim (8 large selectBytes + 64 small settles)` row.", @@ -959,7 +1227,9 @@ ] }, "186": { - "directions": ["parameter-encoding-and-binding"], + "directions": [ + "parameter-encoding-and-binding" + ], "outcomeClass": "accepted", "changedBeliefs": [ "The exp 179 single-row direct-ASCII allocateParams rewrite is materially beneficial once the bound text crosses the mid-tens-of-KB range — exactly the revisit condition exp 179 named. The new focused single_row_large_text_bind.dart workload (1 KB to 1 MB sequential INSERTs) measures -15.4% / -11.1% at 16 KB, -17.3% / -18.7% at 64 KB, -32.3% / -32.1% at 256 KB, and -26.7% / -28.5% at 1 MB across two order-flipped passes", @@ -974,7 +1244,9 @@ ] }, "187": { - "directions": ["parameter-encoding-and-binding"], + "directions": [ + "parameter-encoding-and-binding" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "Exp 186's UTF-8-heavy follow-up now has a representative workload: single_row_large_text_bind.dart emits byte-matched CJK rows beside the ASCII rows, so future single-row bind changes can test both direct-ASCII and direct-UTF-8 payloads in one focused harness.", @@ -988,7 +1260,9 @@ ] }, "188": { - "directions": ["measurement-system"], + "directions": [ + "measurement-system" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "Exp 178's pilot cutoff (`_benchmarkRunDeclarationCutoff = 178`) left ~17 pre-cutoff chartable experiments outside the missing-run-without-declaration guard's scope, still indistinguishable from forgotten result files. The follow-up exp 178 named (`backfill **Benchmark Run:** headers and lower the cutoff`) walks the cutoff back to 1, so every accepted/in-review experiment is now under the same forced-declaration discipline", @@ -1001,37 +1275,59 @@ "the measurement-system openCandidate `memory profiling harness with per-benchmark RSS acceptance criteria` is independent of this run; exp 183/185 added the underlying RSS signal (readerJsonBufHighWaterBytes + the SQLite Diagnostics row), so a future RSS-criteria pass should build on those rather than on the guard scope" ] }, - "195": { - "directions": ["result-transfer-shape"], + "189": { + "directions": [ + "transaction-control-paths" + ], + "outcomeClass": "rejected_below_signal", + "changedBeliefs": [ + "Savepoint naming compression (same-name cached SQL for `SAVEPOINT s`, `RELEASE s`, and `ROLLBACK TO s`) can move best-case focused control rows: empty fanout improved about 6-7%, rollback fanout improved 13-17%, and repeated deep chains improved 7-8% across the order-flipped pair.", + "The representative nested-write fanout did not reproduce: baseline-first measured -1.3%, then the candidate-first pass measured +21.6% slower. That is the load-bearing row because it matches the real nested transaction shape better than empty control-only fanout.", + "Per-savepoint string formatting, native UTF-8 allocation, caching, and naming are below the merge bar after exp 102, exp 111, and exp 189. Nested transaction headroom remains round-trip-shaped, not string-shaped." + ], + "nextSignals": [ + "Do not retry savepoint string caching, savepoint naming compression, or native helpers that still send one request per savepoint boundary without new evidence that string work is dominant.", + "Use benchmark/experiments/savepoint_name_compression.dart for quick savepoint-control probes, but treat the write-fanout row as the representative gate.", + "The remaining open implementation shape is multi-savepoint round-trip batching with preserved callback semantics and rollback legality." + ] + }, + "190": { + "directions": [ + "result-transfer-shape" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ - "Exp 190's per-query `tokens_buf` + first-row pre-encode walk is not noise on small-rowset selectBytes — it is invisible to the `wide_cols.dart` 1-row / 100-row regression guards because their reporting is millisecond-precision, not because the underlying ~0.5-1 µs per query is below the wall floor. Caching the encoded tokens on `resqlite_cached_stmt` (built lazily, freed in `stmt_cache_entry_dispose`) eliminates the per-query `buf_init(64)` + `free` pair and the first-row pre-encode walk while keeping per-row work and JSON output bit-identical to exp 190.", - "A new microsecond-precision focused harness `select_bytes_repeated_calls.dart` (1000 calls/sample × 11 samples after 16-call warmup) exposes the predicted shape: 1-row × 20-col improves −9.2 % / −7.2 % and 10-row × 20-col improves −5.1 % / −2.7 % across two order-flipped passes. Per exp 177's drift classifier this reproduces same-direction across the flip. 100/1000-row guards show sign reversal (+1.8 % / −2.7 % and +1.0 % / −1.7 %) consistent with drift-suspected, because per-query setup is < 0.2 % of wall at those sizes.", - "Exp 190's `wide_cols.dart` 10k-row shapes also trend candidate-faster on every lane across both order-flipped passes (1-7 % movement, magnitude varies but sign reproduces), consistent with eliminating the per-query malloc/free pair from the hot path of the C-side allocator.", - "Per-statement memory cost is `O(col_count * 8 + name_byte_count)` bytes — capped by `STMT_CACHE_MAX = 32` per reader/writer connection. The cache entry's existing lifecycle (eviction, connection close) handles cleanup with no new ownership rules." + "The C-side JSON encoder in `write_json_to_buf` (the only `selectBytes()` codepath) was re-running its per-column emission sequence (comma + `json_write_string` SWAR/escape scan + colon = four `buf_write*` calls) on every row even though the column name is invariant — the same per-row-amortizable pattern exp 034 caught at the Dart `RowSchema` layer and exp 037 caught at the json_buf layer was still firing at the byte-emission layer", + "Pre-building each column's `\"col\":` / `,\"col\":` token once at first-row time into a per-query scratch `resqlite_buf tokens_buf` and replacing the per-row inner loop with a single `buf_write(b, tokens_buf.data + token_offsets[i], token_lens[i])` removes that compounded work without behavior change (the pre-encode reuses `json_write_string`, so escape semantics for unusual column names are bit-identical)", + "Reproduced wins on the wide-many-row shapes the change targets: focused `select_bytes_wide_cols.dart` -4% to -11% across two order-flipped passes on 10k-row x 8 / 20-col shapes (largest shapes get the largest deltas, matching the compound hypothesis), and `large_bytes_transfer.dart` (exp 174's focused guard, ~650 KB per call) -8.7% / -8.2% on large/small lanes. Regression guards (1 row, 100 rows) stay inside the sub-microsecond noise floor — absolute deltas smaller than per-sample spread, with pass-to-pass signs not agreeing" ], "nextSignals": [ - "use `select_bytes_repeated_calls.dart` (not `wide_cols.dart`) as the durable gate for any future selectBytes amortization at small repeated rowsets — `wide_cols.dart` reports in milliseconds and cannot see µs-scale per-query work", - "further attach-onto-cache-entry encoder amortization (per-column type hints, constant value prefixes, schema-stable type dispatch) is only justified by a workload that shows specific structural per-query work dominating after exp 195", - "do not retry promoting the C-side allocator buffer reuse beyond what stmt-cache caching already provides; the json_buf is already persistent per reader (exp 037), and exp 195 amortized the only other per-query allocator pair on the selectBytes hot path" + "use `benchmark/experiments/select_bytes_wide_cols.dart` (and the existing exp 174 `large_bytes_transfer.dart` lane) before changing the selectBytes column-name emission path again", + "if a future selectBytes change wants to amortize more across rows (per-column value-prefix templates for constant-type integer columns, e.g.), the `tokens_buf` + `token_offsets` / `token_lens` arrays are the natural insertion point — extend them rather than adding a parallel scratch path", + "promoting the token scratch into `resqlite_cached_stmt` to skip the first-row pre-encode pass on cache hits is a strictly-larger change (cache lifetime, FFI surface) and is only worth a pass if a workload shows repeated small `selectBytes()` calls dominating wall time; exp 190's 1-row regression guard already shows the per-query overhead is at noise on the smallest shape", + "exp 175's `selectBytes() large bytes` release lane and exp 185's `JSON buffer reclaim` SQLite Diagnostics row remain the public guards for selectBytes transfer policy / RSS — exp 190 does not change either signal" ] }, - "193": { - "directions": ["result-transfer-shape"], - "outcomeClass": "rejected_regression", + "191": { + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_correctness_guard", "changedBeliefs": [ - "A JIT row_map_facade sample made Row.values look like the remaining Row facade gap (baseline values iteration 8.607 ms vs LinkedHashMap 4.779 ms), but repeated JIT samples were contradictory after VM warmup: the baseline itself alternated between ~3 ms, ~9 ms, and ~11 ms on the values lane.", - "Replacing Row.values with a fixed-length ListBase slice view produced stable-looking JIT samples around 3.4-3.7 ms, but the AOT check reversed the decision. Compiled baseline _RowValueIterator values iteration stayed at 2.663 / 2.675 / 2.687 ms, while the ListBase candidate regressed to 6.828 / 6.847 / 7.101 ms.", - "The controls stayed neutral under AOT (hot lookup, containsKey, entries iteration, Map.from clone), so the regression is specific to Row.values. The original custom iterator is the better compiled shape." + "The broader embedded-NUL public API audit candidate is consumed as correctness support, not a performance result. Current main already covers wide executeBatch with multibyte embedded-NUL TEXT, and exp 187 covers single-row execute in flight; exp 191 adds the missing reader surfaces.", + "selectBytes preserves embedded-NUL TEXT seeded by SQLite char(0): select() returns the expected Dart string, hex(CAST(body AS BLOB)) matches _hexUtf8(body), and native JSON bytes decode back to the same value.", + "stream initial and changed emissions preserve embedded-NUL TEXT bytes, proving the one-pass initial decode, result hashing, and re-query emission path do not treat TEXT as NUL-terminated." ], "nextSignals": [ - "do not replace Row.values with ListBase, getRange, or fixed-slice views based on JIT row_map_facade output alone", - "future Row.values work must compile the focused harness to AOT before acceptance; the JIT values lane is not a reliable decision gate", - "no runtime code kept; keep _RowValueIterator unless a future Dart runtime changes compiled iterator behavior" + "do not reopen broad embedded-NUL work without a new public API surface or a concrete regression report; coverage now spans wide executeBatch, in-flight single-row execute, selectBytes, and stream emissions", + "if _utf8Length, _writeUtf8, sqlite3_bind_text lengths, native JSON string emission, or stream row decoding changes, keep these tests in the focused validation set", + "future performance work in this direction still needs a workload where parameter encoding is material; exp 191 does not change the blob-heavy or small/narrow shape guidance" ] }, "192": { - "directions": ["result-transfer-shape"], + "directions": [ + "result-transfer-shape" + ], "outcomeClass": "in_review_accepted", "changedBeliefs": [ "Exp 023 left bounded headroom inside `write_json_to_buf`'s `SQLITE_INTEGER` arm — the single-digit `fast_i64_to_str` body runs one `% 10` / `/ 10` per output digit, and 10k × 20 INTEGER selectBytes performs 200k of these calls. Replacing the body with a two-digit `[00..99]` lookup table (one `% 100` / `/ 100` + one 2-byte memcpy per digit pair) cuts focused integer-heavy `selectBytes` by −8 to −26 % across two order-flipped passes on `select_bytes_int_heavy.dart`, with the biggest win on the deepest-digit shape (10k × 20 ~18-digit big ints, −24 to −26 %) — exactly the shape the algorithm predicts gains most from halving the division count", @@ -1044,47 +1340,37 @@ "the remaining bounded slice of `write_json_to_buf` is the `SQLITE_FLOAT` arm's `snprintf(\"%.17g\")`; exp 041 already rejected a vendored Grisu/Ryu replacement on size grounds, so any future float-encode candidate needs either a much smaller fast path or production evidence that FLOAT cells dominate selectBytes wall" ] }, - "190": { - "directions": ["result-transfer-shape"], - "outcomeClass": "in_review_accepted", - "changedBeliefs": [ - "The C-side JSON encoder in `write_json_to_buf` (the only `selectBytes()` codepath) was re-running its per-column emission sequence (comma + `json_write_string` SWAR/escape scan + colon = four `buf_write*` calls) on every row even though the column name is invariant — the same per-row-amortizable pattern exp 034 caught at the Dart `RowSchema` layer and exp 037 caught at the json_buf layer was still firing at the byte-emission layer", - "Pre-building each column's `\"col\":` / `,\"col\":` token once at first-row time into a per-query scratch `resqlite_buf tokens_buf` and replacing the per-row inner loop with a single `buf_write(b, tokens_buf.data + token_offsets[i], token_lens[i])` removes that compounded work without behavior change (the pre-encode reuses `json_write_string`, so escape semantics for unusual column names are bit-identical)", - "Reproduced wins on the wide-many-row shapes the change targets: focused `select_bytes_wide_cols.dart` -4% to -11% across two order-flipped passes on 10k-row x 8 / 20-col shapes (largest shapes get the largest deltas, matching the compound hypothesis), and `large_bytes_transfer.dart` (exp 174's focused guard, ~650 KB per call) -8.7% / -8.2% on large/small lanes. Regression guards (1 row, 100 rows) stay inside the sub-microsecond noise floor — absolute deltas smaller than per-sample spread, with pass-to-pass signs not agreeing" + "193": { + "directions": [ + "result-transfer-shape" ], - "nextSignals": [ - "use `benchmark/experiments/select_bytes_wide_cols.dart` (and the existing exp 174 `large_bytes_transfer.dart` lane) before changing the selectBytes column-name emission path again", - "if a future selectBytes change wants to amortize more across rows (per-column value-prefix templates for constant-type integer columns, e.g.), the `tokens_buf` + `token_offsets` / `token_lens` arrays are the natural insertion point — extend them rather than adding a parallel scratch path", - "promoting the token scratch into `resqlite_cached_stmt` to skip the first-row pre-encode pass on cache hits is a strictly-larger change (cache lifetime, FFI surface) and is only worth a pass if a workload shows repeated small `selectBytes()` calls dominating wall time; exp 190's 1-row regression guard already shows the per-query overhead is at noise on the smallest shape", - "exp 175's `selectBytes() large bytes` release lane and exp 185's `JSON buffer reclaim` SQLite Diagnostics row remain the public guards for selectBytes transfer policy / RSS — exp 190 does not change either signal" - ] - }, - "189": { - "directions": ["transaction-control-paths"], - "outcomeClass": "rejected_below_signal", + "outcomeClass": "rejected_regression", "changedBeliefs": [ - "Savepoint naming compression (same-name cached SQL for `SAVEPOINT s`, `RELEASE s`, and `ROLLBACK TO s`) can move best-case focused control rows: empty fanout improved about 6-7%, rollback fanout improved 13-17%, and repeated deep chains improved 7-8% across the order-flipped pair.", - "The representative nested-write fanout did not reproduce: baseline-first measured -1.3%, then the candidate-first pass measured +21.6% slower. That is the load-bearing row because it matches the real nested transaction shape better than empty control-only fanout.", - "Per-savepoint string formatting, native UTF-8 allocation, caching, and naming are below the merge bar after exp 102, exp 111, and exp 189. Nested transaction headroom remains round-trip-shaped, not string-shaped." + "A JIT row_map_facade sample made Row.values look like the remaining Row facade gap (baseline values iteration 8.607 ms vs LinkedHashMap 4.779 ms), but repeated JIT samples were contradictory after VM warmup: the baseline itself alternated between ~3 ms, ~9 ms, and ~11 ms on the values lane.", + "Replacing Row.values with a fixed-length ListBase slice view produced stable-looking JIT samples around 3.4-3.7 ms, but the AOT check reversed the decision. Compiled baseline _RowValueIterator values iteration stayed at 2.663 / 2.675 / 2.687 ms, while the ListBase candidate regressed to 6.828 / 6.847 / 7.101 ms.", + "The controls stayed neutral under AOT (hot lookup, containsKey, entries iteration, Map.from clone), so the regression is specific to Row.values. The original custom iterator is the better compiled shape." ], "nextSignals": [ - "Do not retry savepoint string caching, savepoint naming compression, or native helpers that still send one request per savepoint boundary without new evidence that string work is dominant.", - "Use benchmark/experiments/savepoint_name_compression.dart for quick savepoint-control probes, but treat the write-fanout row as the representative gate.", - "The remaining open implementation shape is multi-savepoint round-trip batching with preserved callback semantics and rollback legality." + "do not replace Row.values with ListBase, getRange, or fixed-slice views based on JIT row_map_facade output alone", + "future Row.values work must compile the focused harness to AOT before acceptance; the JIT values lane is not a reliable decision gate", + "no runtime code kept; keep _RowValueIterator unless a future Dart runtime changes compiled iterator behavior" ] }, - "191": { - "directions": ["parameter-encoding-and-binding"], - "outcomeClass": "in_review_correctness_guard", + "195": { + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "in_review_accepted", "changedBeliefs": [ - "The broader embedded-NUL public API audit candidate is consumed as correctness support, not a performance result. Current main already covers wide executeBatch with multibyte embedded-NUL TEXT, and exp 187 covers single-row execute in flight; exp 191 adds the missing reader surfaces.", - "selectBytes preserves embedded-NUL TEXT seeded by SQLite char(0): select() returns the expected Dart string, hex(CAST(body AS BLOB)) matches _hexUtf8(body), and native JSON bytes decode back to the same value.", - "stream initial and changed emissions preserve embedded-NUL TEXT bytes, proving the one-pass initial decode, result hashing, and re-query emission path do not treat TEXT as NUL-terminated." + "Exp 190's per-query `tokens_buf` + first-row pre-encode walk is not noise on small-rowset selectBytes — it is invisible to the `wide_cols.dart` 1-row / 100-row regression guards because their reporting is millisecond-precision, not because the underlying ~0.5-1 µs per query is below the wall floor. Caching the encoded tokens on `resqlite_cached_stmt` (built lazily, freed in `stmt_cache_entry_dispose`) eliminates the per-query `buf_init(64)` + `free` pair and the first-row pre-encode walk while keeping per-row work and JSON output bit-identical to exp 190.", + "A new microsecond-precision focused harness `select_bytes_repeated_calls.dart` (1000 calls/sample × 11 samples after 16-call warmup) exposes the predicted shape: 1-row × 20-col improves −9.2 % / −7.2 % and 10-row × 20-col improves −5.1 % / −2.7 % across two order-flipped passes. Per exp 177's drift classifier this reproduces same-direction across the flip. 100/1000-row guards show sign reversal (+1.8 % / −2.7 % and +1.0 % / −1.7 %) consistent with drift-suspected, because per-query setup is < 0.2 % of wall at those sizes.", + "Exp 190's `wide_cols.dart` 10k-row shapes also trend candidate-faster on every lane across both order-flipped passes (1-7 % movement, magnitude varies but sign reproduces), consistent with eliminating the per-query malloc/free pair from the hot path of the C-side allocator.", + "Per-statement memory cost is `O(col_count * 8 + name_byte_count)` bytes — capped by `STMT_CACHE_MAX = 32` per reader/writer connection. The cache entry's existing lifecycle (eviction, connection close) handles cleanup with no new ownership rules." ], "nextSignals": [ - "do not reopen broad embedded-NUL work without a new public API surface or a concrete regression report; coverage now spans wide executeBatch, in-flight single-row execute, selectBytes, and stream emissions", - "if _utf8Length, _writeUtf8, sqlite3_bind_text lengths, native JSON string emission, or stream row decoding changes, keep these tests in the focused validation set", - "future performance work in this direction still needs a workload where parameter encoding is material; exp 191 does not change the blob-heavy or small/narrow shape guidance" + "use `select_bytes_repeated_calls.dart` (not `wide_cols.dart`) as the durable gate for any future selectBytes amortization at small repeated rowsets — `wide_cols.dart` reports in milliseconds and cannot see µs-scale per-query work", + "further attach-onto-cache-entry encoder amortization (per-column type hints, constant value prefixes, schema-stable type dispatch) is only justified by a workload that shows specific structural per-query work dominating after exp 195", + "do not retry promoting the C-side allocator buffer reuse beyond what stmt-cache caching already provides; the json_buf is already persistent per reader (exp 037), and exp 195 amortized the only other per-query allocator pair on the selectBytes hot path" ] } } diff --git a/experiments/signals/base.json b/experiments/signals/base.json new file mode 100644 index 00000000..fae00da9 --- /dev/null +++ b/experiments/signals/base.json @@ -0,0 +1,341 @@ +{ + "schemaVersion": 2, + "purpose": "Machine-readable research map for scheduled experimenters. This is context, not an allowed list.", + "schemaNotes": { + "keyPriors": "Up to ~6 experiments a future runner must read to evaluate work in this direction. Curate when a new accepted experiment supersedes an older one.", + "archive": "Older or superseded evidence kept for searchable history. Not required reading.", + "openCandidates": "Dated candidate ideas waiting for the right workload, signal, or runner. Each entry has {idea, addedDate, addedAfter?, blockedOn?}. Prune entries older than ~3 months that nobody picked up.", + "blockedOnMeasurement": "Measurements (counters, benchmarks, profile harnesses) that must land before the next implementation experiment in this direction is worth attempting. Empty if no measurement is gating new work." + }, + "coverage": { + "experimentEntriesRequiredFrom": 110, + "directionFieldRequiredFrom": 110 + }, + "statusDefinitions": { + "active": "Good current candidate space. A runner may still choose another direction, but this area has enough current evidence to justify more work.", + "watch": "Worth revisiting when external/runtime/library/workload context changes. This is a watch note, not a prohibition.", + "measurement-needed": "The next useful step is likely a benchmark, trace, profiler run, or other measurement that exposes the suspected cost.", + "low-current-signal": "Recent attempts produced weak signal under current evidence. A novel implementation, workload, or measurement can still make the area interesting.", + "speculative": "A lightly evidenced or new direction that may be worth a bounded pass.", + "settled": "Mostly background context. Reopen when there is a clear new fact or workload." + }, + "directions": [ + { + "id": "stream-rerun-dispatch", + "status": "active", + "subsystems": [ + "streaming", + "dispatch", + "reader-pool", + "invalidation" + ], + "currentRead": "Stream fan-out performance is shaped by rerun scheduling, reader-pool admission, completion-side churn, writer/request residual, and dependency precision. Queue changes have produced both strong wins and sharp regressions. Exp 120 closed the upstream over-dispatch in StreamEngine._flushQueue (parked_total drops 3,590 -> 0 on A11c overlap and 1,198 -> 0 on keyed-PK; max_parked 46 -> 0), and exp 122 removed the remaining stream-admission async boundary by constructing StreamEngine with a concrete ReaderPool. Exp 121 ruled out invalidation traversal as the active implementation target: overlap invalidation is 10-15% of wall, column intersection is 2.5-5.7%, and the structural ceiling is at the per-benchmark decision-threshold edge. Exp 134 proved row-level dirty precision can halve keyed-PK writer-burst wall for a narrow `WHERE id = ?` proof, but its internal SQL recognizer is rejected; revive that area only through explicit API/design or real workload evidence. Exp 136 ships the completion-side reader-handler counter: on A11c overlap the reader worker port handler is 28.57% of total wall (burst + drain) at ~18 us per call across 4,228 calls/burst, and subscriber-fanout emit is only 0.35% of the chain. Exp 147 split writer-side burst wall from SQLite-facing writer calls: on A11c overlap, SQLite is 15.7 ms / 166.8 ms (9.4%), invalidation is 18.8%, and residual writer/request wall is 71.8%; keyed-PK shows the same shape (18.1% SQLite, 18.7% invalidation, 63.3% residual). Exp 148 tested the natural reader-reply batching follow-up and rejected it: the profile smoke cut A11c overlap completion callbacks 4,527 -> 1,425 and completion wall 109.6 ms -> 55.6 ms, but Tracelite measured elapsed stayed neutral/slower (+5.18% high-cardinality, +3.28% many-streams, +13.5% keyed-PK). Exp 151 tested synchronous writer response resolution against the residual writer/request bucket and rejected it: high-cardinality fanout stayed neutral (+2.92%), keyed-PK trended slower with too-noisy evidence (+18.5%), and many-streams writer throughput trended slower (+14.0%). Exp 170 tested the matching request-side variant — `Mutex.tryLock` plus a non-`async` `Writer.execute` / `executeBatch` to drop the uncontended `await _mutex.lock()` microtask hop — and rejected it: the primary Single Inserts (100 sequential) / sequential-awaited (2000 writes) lanes stayed within ±2 % (wrong direction) across paired runs, while the only positive signal (-7.8 % on Concurrent Single Inserts) is a row exp 159 already drives at -58 % to -61 %. SQLite-step tuning, stream admission, invalidation traversal, plain worker-side reader-reply batching, synchronous writer response resolution, synchronous writer request acquisition, SQL-recognizer-based keyed-PK precision, allocation-only `_flushQueue` cleanup, and standalone residual-split profiling are not active targets on currently-measured workloads. Exp 159 attacked the residual structurally: persistent writer reply port + cached SendPort + sync FIFO completion remove fixed per-round-trip scheduling cost, and releasing the write lock at send time pipelines concurrent standalone writes through the worker port FIFO; the focused concurrent-burst benchmark improved 36-45%, exp 147 residual_us dropped on all four audit workloads, and stream-dispatch Tracelite guardrails were neutral on the clean order-flipped pass. Exp 161 closes the release-suite gap by promoting the concurrent-burst shape into `benchmark/suites/writes.dart` as a paired Single Inserts (100 sequential) / Concurrent Single Inserts (100 concurrent) row pair, so exp 159's pipelining win and future writer-scheduling experiments are evaluable on a public release lane (resqlite concurrent median ~1.1 ms vs ~2.9 ms sequential). Exp 171 then tried to apply exp 159's `_sendPort` cache pattern one layer up — a sync-readable `_resolvedRuntime` field on `Database` so post-open hot paths skip the `await _runtime` microtask hop — and rejected it: two order-flipped passes on writer_pipelining.dart produced alternating-sign deltas inside per-round variance (sequential-awaited -2.3%/+2.5%, transaction-guardrail -7.5%/+6.1%), so a ~1-2 us per-call hop sits at or below the harness floor. Database-layer microtask hop trimming is now off the candidate list; the next sequential-write reduction must reduce round-trip count (group commit) or change transport.", + "keyPriors": [ + "120", + "121", + "134", + "136", + "147", + "148" + ], + "archive": [ + "045", + "075", + "083", + "084", + "085", + "100", + "105", + "106", + "114", + "115", + "118", + "119", + "122", + "145", + "151", + "164", + "170", + "171", + "182" + ], + "interestingIf": [ + "a change reduces duplicate reruns before reader-pool admission", + "a workload shows row-level precision is worth explicit API or metadata design outside a SQL text recognizer", + "a trace shows completion-side churn dominating a many-stream workload", + "a scheduler policy improves fan-out without harming unrelated reads" + ], + "openQuestions": [ + "Can duplicate stream work be coalesced earlier without reintroducing stale delivery or starvation?", + "Should explicit row-observer APIs cover aliases, joins, composite keys, or WITHOUT ROWID tables that exp 134 could not cover cleanly as an internal recognizer?", + "Which bounded stream scheduling or workload-shape implementation can be tried next with in-branch measurement, rather than another standalone residual split?" + ], + "openCandidates": [ + { + "idea": "long-running concurrent-reads workload that sustains parked dispatchers past pool size", + "addedDate": "2026-04-30", + "addedAfter": "115" + }, + { + "idea": "rerun coalescing implementation candidate with temporary per-entry dispatch lineage tracing", + "addedDate": "2026-05-02" + } + ], + "blockedOnMeasurement": [], + "notesForExperimenters": "Avoid assuming a larger reader pool helps; exp 105 found the opposite under A11c fan-out. After exp 118 + exp 120, `dispatcherParkedTotal` and `dispatcherWakeRetryTotal` stay at zero on every measured stream workload — the parked-dispatcher signal is not the active target. Exp 121 took invalidation traversal off the candidate list (10–15% of overlap wall, 2.5–5.7% intersection, 80–200 ns per probe). Exp 122 keeps admission simple by giving StreamEngine a concrete ReaderPool and moving stream registry checks to diagnostics. Exp 134 is proof that row-level precision can win, but its SQL-recognizer implementation is rejected; revive it only with explicit API/design or real workload evidence. Exp 136 showed completion-side reader handling was large enough to try batching, but exp 148 proved plain worker-side reader-reply batching is not mergeable: it reduced callback counters while failing measured-elapsed primary scenarios. Exp 147 still leaves residual writer/request wall as the biggest bucket, but standalone residual splitting has reached diminishing returns. Exp 151 tried the narrow response-side variant (`Completer.sync()` for writer responses) and rejected it under Tracelite. Exp 170 tried the matching request-side variant (`Mutex.tryLock` + non-`async` `Writer.execute` to drop the uncontended `await _mutex.lock()` microtask hop) and rejected it: Single Inserts (100 sequential) and writer_pipelining `sequential-awaited (2000 writes)` both moved <2 % in the wrong direction across paired runs, and the only positive lane (Concurrent Single Inserts, −7.8 %) is one exp 159 already drives at −58 % to −61 %. Do not retry either scheduling tweak without new runtime or workload evidence. Exp 171 tried the same shape one layer up (cached `_resolvedRuntime` on `Database` to skip `await _runtime` on hot paths) and rejected it on focused-harness noise — Database-layer microtask hop trimming above the writer no longer moves the sequential-write floor. Exp 182 took the residual bucket attack in a different direction — skip `preupdate_hook` accumulation + reply harvest when `_streamEngine.length == 0` via a `track_dirty` flag on `resqlite_db` + a `DrainRequest` no-op barrier on first stream registration — and rejected it: the no-stream wins are real (focused sequential −3.8 % / −5.3 %, wide-batch −2.4 % / −5.9 % across order-flipped passes) but the per-call gate adds reproduced overhead on the with-streams shape (focused +2.7 % / +3.7 %, classified `reproduced` by `ab_drift_check.dart`) and Tracelite stream-direction warmup elapsed regressed in the same direction across all three scenarios. Reactive streams are the library's primary use case, so the optimization helps a narrower workload mix than the one it slows — do not retry without a workload that shows write throughput without active streams is a hot path. A new stream experiment should try a concrete reduction candidate, add only the narrow measurement needed to explain the result in that same branch, remove temporary counters before merge unless they are reusable, and clear Tracelite measured-elapsed primary gates without harming keyed-PK." + }, + { + "id": "parameter-encoding-and-binding", + "status": "active", + "subsystems": [ + "ffi", + "params", + "writer", + "allocation" + ], + "currentRead": "Parameter work can still matter when it removes meaningful native allocation, copying, or repeated SQLite work. Smaller Dart-side allocation cleanups and two-parameter batch-list flattening changes usually measured flat, but exp 113 found a clear wide-row batch signal by avoiding the temporary flat Dart parameter list entirely. Exp 116 promotes the 10,000-row x 20-parameter mixed batch shape into release-suite coverage so width regressions are visible outside the focused script. Exp 125 then showed that large wide ASCII-heavy batches still had removable per-string UTF-8 list allocation inside the matrix encoder: direct ASCII payload packing improved focused 10k x20 from 17.199 ms to 12.760 ms and release Wide Batch Insert from 18.201 ms to 13.031 ms. Exp 126 extended that same allocation-removal shape to non-ASCII wide batches with direct UTF-8 payload writing: focused Unicode 10k x20 improves 21.945 ms to 18.988 ms and emoji 10k x20 improves 24.187 ms to 17.458 ms while release write-suite guardrails stay neutral. Exp 142 retested direct single-row text parameter encoding under Tracelite on chat-sim and narrow-batch-insert; it did not clear the primary gate and trended slower (+6.86% and +16.4%). Exp 146 tested lowering the ASCII batch-packing threshold to 2 params / 64 total params with a Tracelite A/B run over narrow-batch-insert; it produced no primary improvement (resqlite +1.45%, neutral) and a noisy sqlite_async guardrail, so small/narrow writes should stay on the generic path. Exp 149 found the middle ground with Tracelite profile merge rounds: repeated six-parameter ASCII merge batches improve executeBatch p50 88 -> 75 us and writer SQLite time 87,895 -> 75,947 us when admitted at 6 params / 600 total params. Exp 150 fixed the first-row-null blind spot inside that same guard: nullable ASCII 10k x8 improves 13.552 -> 11.152 ms and 10k x20 improves 25.738 -> 21.723 ms, while existing ASCII and Unicode wide guardrails stayed neutral in the focused pass. Exp 186 then closed exp 179's named revisit condition: added a focused single-row large-text-bind workload (single_row_large_text_bind.dart, 1 KB to 1 MB sequential INSERT shapes) and ran archive/exp-179's direct-ASCII allocateParams rewrite against it. The encoder savings are immaterial at 1 KB but become decisive once the bound text crosses the mid-tens-of-KB range — focused medians improve -15.4% / -11.1% at 16 KB, -17.3% / -18.7% at 64 KB, -32.3% / -32.1% at 256 KB, and -26.7% / -28.5% at 1 MB across two order-flipped passes, with encoder-isolation deltas reproducing exp 179's -45% / -58% / -37% on the synthetic micro. Small-payload release-suite lanes (Parameterized Queries, Single Inserts, Concurrent Single Inserts) stay neutral, so exp 179's small-bind finding stands — the encoder is now the right default for the single-row path because we have a representative workload where it matters, not because the small case stopped being flat. Exp 187 consumes exp 186's UTF-8-heavy follow-up by adding byte-matched CJK rows to the same harness and reusing the batch direct-UTF-8 writer for non-ASCII single-row strings; CJK improves roughly 31-39% from 16 KB through 1 MB across the order-flipped pair while the exp 186 all-ASCII fast path stays intact.", + "keyPriors": [ + "109", + "125", + "126", + "149", + "186", + "187" + ], + "archive": [ + "028", + "076", + "077", + "095", + "096", + "112", + "116", + "142", + "146", + "150" + ], + "interestingIf": [ + "the idea changes ownership/layout enough to remove native allocation or copying", + "a benchmark shows parameter encoding as a material part of write wall time", + "the approach preserves the lean API and targets parameter width as well as row count" + ], + "openQuestions": [ + "Are there remaining blob-heavy parameter shapes where encoding, not SQLite stepping, dominates?", + "Does any production workload need text/binary edge-case guarantees outside the public API surfaces covered by exp 191?" + ], + "openCandidates": [], + "blockedOnMeasurement": [], + "notesForExperimenters": "Do not treat tiny allocation removal as the whole category. Exp 109 shows this area can win when the implementation removes a larger allocation/copy pattern; exp 112 shows simple two-parameter batch-list construction tweaks are not enough by themselves; exp 113 shows wide batch rows can justify a direct matrix encoder. After exp 116, release-suite write runs include one 10k x 20-param mixed batch row; use the focused benchmark only when sweeping additional widths, row counts, or nullable text shapes. After exp 125 and exp 126, direct text payload packing is justified only for large wide batches: ASCII takes the first fast path, non-ASCII takes the direct UTF-8 path. Exp 142 rejected carrying the same direct text encoding pattern into the single-row `allocateParams` path on small workloads, and exp 179 reaffirmed that small-bind path is below the round-trip floor; exp 186 then revived the same encoder against a representative large-text bind workload (single_row_large_text_bind.dart 16 KB - 1 MB) where the encoder savings clear the floor (-15% to -32% reproduced across order-flipped passes). Exp 187 extends that single-row encoder to non-ASCII strings by replacing `_allocateParamsPreEncoded` with direct `_utf8Length` / `_writeUtf8` packing; the CJK rows in the same harness now carry the UTF-8-heavy acceptance gate. Exp 146 specifically rejected lowering the ASCII fast-path guard to small/narrow batches; exp 149 only admits repeated 6+ parameter batches after Tracelite showed merge-round writer cost. Exp 150 keeps that guard but treats first-row NULL values as possible text so nullable generated-statement batches can still use the packed encoder. Use single_row_large_text_bind.dart as the durable workload for any future single-row bind change; run both ASCII and CJK rows, with the 1 MB shapes as the load-bearing acceptance gates. Keep 2-3 parameter writes generic unless a future workload proves parameter encoding dominates wall time and clears a Tracelite A/B primary gate." + }, + { + "id": "long-text-stream-hashing", + "status": "settled", + "subsystems": [ + "streaming", + "hashing", + "text" + ], + "currentRead": "Native hashing is valuable, and long TEXT/BLOB cells now have four representative unchanged-fanout signals: exp 110's 4 KB TEXT shape, exp 172's mixed 32 KB TEXT + 32 KB BLOB release row, exp 173's 32 KB long-text focused harness, and exp 181's one-reader single-stream 64 KB TEXT + 64 KB BLOB harness. Exp 173 showed the unrolled 16-byte FNV body was +4.5% and +12.1% versus exp 110's 8-byte fold on the pool-of-4 32 KB fanout; exp 181 removed reader-pool parallelism and still measured flat across an order-flipped pair (baseline 2.771/2.777 ms, candidate 2.763/2.792 ms). The 8-byte fold remains the correct implementation. Further FNV loop unrolling is settled off unless a production profile or direct resqlite_query_hash microbenchmark proves the byte fold itself, not SQLite value access, reader dispatch, or reply delivery, is again dominant.", + "keyPriors": [ + "075", + "099", + "110", + "172", + "173", + "181" + ], + "archive": [ + "033" + ], + "interestingIf": [ + "a production profile shows long TEXT/BLOB stream hashing remains hot after chunked folding", + "a workload isolates the hash loop from reader-pool parallelism (single-stream long-payload, direct FFI microbenchmark)", + "a mixed BLOB/TEXT long-payload shape produces a different ratio of hash vs SQLite text retrieval" + ], + "openQuestions": [ + "What text and blob sizes appear in realistic stream workloads?", + "If long-payload unchanged fanout becomes hot again, does the remaining cost sit in hashing, SQLite value access, Dart decode, or result delivery?", + "Would a direct resqlite_query_hash microbenchmark show loop-only headroom, and would that translate back to a public stream workload?" + ], + "openCandidates": [], + "blockedOnMeasurement": [], + "notesForExperimenters": "Use the long-payload streaming rows before trying another hash-loop change: exp 110's 4 KB TEXT row, exp 172's mixed 32 KB TEXT + 32 KB BLOB row, exp 173's 32 KB long-text row (`Long-Text 32KB Unchanged Fanout` in streaming.dart, or `benchmark/experiments/long_text_32kb_hash.dart`), and exp 181's one-reader `benchmark/experiments/single_stream_long_payload_hash.dart` harness. The 8-byte fold (exp 110) and 16-byte fold (exp 173/181) have both been tried; further unrolling variants are not worth a pass on current public stream workloads. Reopen only with a production profile or direct hash microbenchmark that proves the byte fold is again the dominant cost." + }, + { + "id": "sqlite-version-and-build-config", + "status": "watch", + "subsystems": [ + "sqlite", + "sqlite3mc", + "build", + "planner" + ], + "currentRead": "SQLite compile/config changes have produced real wins, but version bumps need audit discipline because planner and text-format behavior can shift under the library. Exp 144 bumped sqlite3mc 2.3.2 → 2.3.5 (SQLite 3.51.3 → 3.53.2) once SQLite shipped its `.2` point release and sqlite3mc cut a tracking release, satisfying exp 090's revisit trigger exactly. Tests stayed green including the embedded-NUL and Unicode bind regression suite. The single-pass release-suite A/B swung between 19 wins / 18 regressions / 124 neutral on the canonical run and 30 wins / 2 regressions / 129 neutral on the rerun — a wide spread characteristic of single-pass noise at sub-ms granularity. The only metric flagged consistently across reruns is Concurrent Reads 8× wall median (+~20% on a sub-ms metric); the 4× concurrency case wins on the same baseline, so it is not a generic read-pool slowdown. Soak window is the right place to confirm whether this is a real 3.53.x reader-pool interaction or single-run tail noise. The 3.53.0 FP-rounding default change (15→17 digits) was confirmed irrelevant because resqlite reads REAL via `sqlite3_column_double` and serialises with its own `snprintf(\"%.17g\", ...)` rather than `sqlite3_column_text`, so the proposed `SQLITE_DBCONFIG_FP_DIGITS=15` shim was skipped.", + "keyPriors": [ + "016", + "044", + "090", + "144" + ], + "archive": [ + "020", + "021" + ], + "interestingIf": [ + "sqlite3mc tracks a stable SQLite point release with relevant planner or C API changes", + "a platform-specific storage feature maps cleanly to resqlite defaults", + "peer libraries report repeatable wins from a SQLite configuration resqlite does not use" + ], + "openQuestions": [ + "Which SQLite release changes affect resqlite's custom JSON/worker architecture rather than only generic SQL workloads?", + "Can any build option improve one target platform without harming others?", + "Does the exp 144 concurrent-reads-8× single-run slowdown survive a multi-pass release-suite rerun under the 3.53.x line, or does it collapse like the streaming overlap re-emit flag did?" + ], + "openCandidates": [ + { + "idea": "sqlite3mc bump audit when 3.54.x .2+ ships", + "addedDate": "2026-06-08", + "addedAfter": "144", + "blockedOn": "no stable SQLite 3.54.x point release tracked by sqlite3mc" + } + ], + "blockedOnMeasurement": [], + "notesForExperimenters": "Treat this as audit-first. A version bump is interesting when the changelog and current workload both suggest a plausible resqlite-specific win. Exp 144 is the canonical pattern: confirm the changelog touches a path resqlite actually exercises (not `sqlite3_column_text` for REAL, which we bypass), confirm the policy trigger (`.2`+ SQLite + sqlite3mc tracking it) is satisfied, then take a single-pass release-suite A/B with explicit follow-up confirmation runs on any flagged metric." + }, + { + "id": "transaction-control-paths", + "status": "low-current-signal", + "subsystems": [ + "writer", + "transactions", + "sqlite" + ], + "currentRead": "Cached top-level transaction control statements won. Nested-transaction string/native helpers stayed flat even after exp 111 added a worst-case shallow-fan-out savepoint workload (50× SAVEPOINT/RELEASE per iteration): exp 102's cache pattern measured -9 % (within the ±17 % decision threshold). Exp 189 then tried the smaller savepoint naming-compression variant: reusing same-name cached SQL (`SAVEPOINT s`, `RELEASE s`, `ROLLBACK TO s`) produced best-case focused wins on empty fanout (~-6%), rollback fanout (-17% / -13%), and repeated deep chains (-7% / -8%), but the representative nested-write fanout failed to reproduce (-1.3%, then +21.6% slower). Per-isolate-round-trip cost dominates per-savepoint allocation/naming savings, so string/naming work is closed.", + "keyPriors": [ + "101", + "102", + "111", + "189" + ], + "archive": [ + "027", + "103" + ], + "interestingIf": [ + "a change collapses multiple savepoint open/close operations into a single isolate round-trip (analogous to exp 009's read-side batching)", + "a profile mode shows savepoint boundary round-trips, not string allocation, as a measurable spike under a realistic workload", + "the change reduces prepare/string work without adding much native API surface" + ], + "openQuestions": [ + "How often do realistic resqlite users nest transactions deeply?", + "Can nested control be optimized without specialized C helpers for every operation?", + "Is there a multi-savepoint batching shape that collapses N round-trips into 1 without weakening rollback semantics?" + ], + "openCandidates": [ + { + "idea": "multi-savepoint round-trip batching (analogous to exp 009 read-side batching)", + "addedDate": "2026-04-28", + "addedAfter": "111" + } + ], + "blockedOnMeasurement": [ + "realistic deeply-nested transaction workload (exp 111 covers the shallow worst-case; deep cases still need a workload)" + ], + "notesForExperimenters": "Compare against the exp 111 nested-tx benchmark and exp 189's `savepoint_name_compression.dart` harness before claiming a win on any savepoint-related change. Per-call savepoint allocation, caching, and naming changes are below the merge bar on realistic nested-write fanout. The open headroom is round-trip-shaped, not allocation-shaped; do not retry savepoint string/naming tweaks without new evidence that the string path, rather than the isolate request boundary, has become dominant." + }, + { + "id": "result-transfer-shape", + "status": "watch", + "subsystems": [ + "results", + "isolate-transfer", + "api-shape" + ], + "currentRead": "The current ResultSet/Row shape is close to optimal for the shipped select() contract. Alternatives often move work rather than remove it, especially once main-isolate consumption is measured. Exp 158 found a narrow exception inside the existing shape: adding a schema-name identity fast path for schemas up to 32 columns plus private HashMap fallback for RowSchema.indexOf roughly halved focused row facade lookup and select_maps main-isolate full-consumption medians without changing transfer or public API, while point-query schema construction stayed neutral/noisy. Exp 167 rechecked closed exp 141's direct ResultSet.forEach override on a real SQLite-backed consumer lane after exp 158 and rejected it: a small first-pair win reversed on the longer confirmation pair, so no runtime code was kept. Exp 174 found a transport asymmetry: the reader 'sacrifice' path (Isolate.exit + reader respawn) is a real win for the rows path because it transfers already-built Dart objects with no re-copy, but it was applied by result size to selectBytes too, where the native JSON must be Uint8List.fromList-copied before Isolate.exit can transfer it — so sacrifice saved zero copies on bytes and only added a respawn, while the non-sacrifice bytes path copied twice. selectBytes now sends a Uint8List view over the connection's persistent json_buf and never sacrifices: -44% (~1.8x) on large (>256KB) byte reads by eliminating the respawn, -4% on small, at a bounded ~+15MB RSS high-water (readers no longer respawned). Exp 175 adds a named release-suite guard for that large-bytes path: `Large payload (~650KB) / resqlite selectBytes()` measured 0.323 ms wall / 0.000 ms main and is curated as `selectBytes() large bytes`, so the history no longer relies on the sub-256KB 1K-row metric to watch exp 174. Rows select() keeps sacrifice — there the zero-copy object transfer is real. Exp 176 closed a gap exp 158 left inside the same RowSchema index: Row.containsKey still hashed the key in the private HashMap on every call, bypassing the identity fast path its sibling operator[] already used, so it ran ~+3.6 ms slower than a LinkedHashMap on the focused containsKey lane. Routing it through a shared RowSchema.containsName (= indexOf(name) >= 0) improved that lane ~13.0 -> ~10.0 ms (-23%) with a flat hot-lookup control, flipping Row to at-parity, behavior-identical and no API change. The win is interned-key-specific (decoded schema names are not identical to user literals, so production probes generally fall through to the HashMap, same cost as before). Exp 193 rejected replacing Row.values' custom iterator with a fixed ListBase slice view: JIT row_map_facade values samples were unstable, and the AOT check showed the original _RowValueIterator faster (2.663-2.687 ms) than the list view (6.828-7.101 ms), so Row.values should keep the custom iterator unless a future Dart runtime changes compiled behavior. Exp 183 closed the remaining piece of exp 174's bounded RSS trade-off by quantifying it and reclaiming it: a new Diagnostics.readerJsonBufHighWaterBytes field exposes per-reader json_buf.cap, the focused json_buf_retention.dart audit confirms pathological retention is real (8 concurrent x 8 MB selectBytes pin 32 MB across the 4-reader pool for the rest of the connection), and a C-side reader-worker shrink fired after SendPort.send returns (gated by cap > 1 MB AND last_used_len < 256 KB) reclaims back to the 16 KB initial cap on subsequent small reads — post-burst settle 32 MB -> 64 KB, recurring-large 16 MB -> 64 KB, with neutral large_bytes_transfer.dart numbers. Exp 185 promotes that diagnostic into the release diagnostics suite: `SQLite Diagnostics / JSON buffer reclaim (8 large selectBytes + 64 small settles)` now records `jsonBufKiB` and fails if the post-settle high-water exceeds 512 KiB; the first focused suite run settled at 64.0 KiB with idle readers, so the exp 183 reclaim has public regression visibility. Exp 190 takes the encoder-side win inside the same direction: `write_json_to_buf` in `native/resqlite.c` now pre-builds each column's `\"col\":` / `,\"col\":` token once at first-row time into a per-query scratch buffer, so subsequent rows emit each column with one `buf_write` instead of comma + `json_write_string` (SWAR scan + escape walk) + colon. Focused `select_bytes_wide_cols.dart` measures -4% to -11% across two order-flipped passes on 10k-row x 8 / 20-col shapes; `large_bytes_transfer.dart` (exp 174's focused guard) also moves -8.7% / -8.2% on large/small lanes. Regression guards (1 row, 100 rows) stay in the sub-microsecond noise floor. Exp 192 closes the bounded headroom exp 023 left inside `write_json_to_buf`'s `SQLITE_INTEGER` arm: replacing the single-digit `fast_i64_to_str` body with a two-digit `[00..99]` lookup table (one `% 100` / `/ 100` and one 2-byte memcpy per digit pair) cuts focused integer-heavy selectBytes by −8 to −26 % across two order-flipped passes on `select_bytes_int_heavy.dart` — biggest win on 10k × 20 ~18-digit big ints (−24 to −26 %), where the digit-loop length is greatest. Mixed-cell and small-payload regression guards stay inside ±1 %. The release suite is not the right denominator (no lane is integer-heavy enough), so `select_bytes_int_heavy.dart` is the durable gate for future selectBytes integer-encode work. Exp 195 promotes exp 190's per-query `tokens_buf` scratch into the `resqlite_cached_stmt` entry, amortizing the per-query `buf_init(64)` + `free` pair and the first-row pre-encode walk across every re-execution of the same prepared SQL. Per-row inner loop and JSON output are byte-identical to exp 190. The exp 190 1-row regression guard sits at the millisecond-reporting harness floor and looked neutral; a new microsecond-precision focused harness `select_bytes_repeated_calls.dart` (1000 calls per sample) measures the predicted shape directly — 1-row × 20-col improves −9.2 % / −7.2 % and 10-row × 20-col improves −5.1 % / −2.7 % across two order-flipped passes, while 100/1000-row guards show sign reversal (drift-suspected per exp 177's classifier; per-query setup is < 0.2 % of wall there). Exp 190's `wide_cols.dart` 10k-row shapes also trend candidate-faster on every lane across both passes. Memory cost per cached statement is `O(col_count * 8 + name_byte_count)` capped by `STMT_CACHE_MAX = 32` per connection.", + "keyPriors": [ + "158", + "174", + "176", + "183", + "192", + "195" + ], + "archive": [ + "008", + "063", + "066", + "081", + "082", + "089", + "190", + "193" + ], + "interestingIf": [ + "Dart adds new deeply immutable or transfer primitives that support typed data and lists", + "a change preserves the lean API while removing end-to-end work", + "a benchmark measures full consumer cost, not just transfer or decode setup" + ], + "openQuestions": [ + "Can a transparent internal shape change preserve Map-like ergonomics without moving decode cost to the consumer?", + "Will future Dart transfer semantics reopen zero-copy result transport?" + ], + "openCandidates": [ + { + "idea": "re-evaluate deeply-immutable ResultSet when Dart SDK #50068 (DI typed-data factory) ships", + "addedDate": "2026-04-24", + "addedAfter": "089", + "blockedOn": "Dart SDK #50068 (deeply-immutable typed-data factory) not yet shipped" + } + ], + "blockedOnMeasurement": [], + "notesForExperimenters": "Be explicit about API impact. Some measured wins depend on a new public API, which is outside the lean-API goal. Use `select_maps`, `resultset_foreach_consumer.dart`, or a similarly full-consumer benchmark before accepting result-shape changes; setup-only transfer wins are not enough. Exp 158 is a narrow private data-structure win inside RowSchema, not evidence to revive larger ResultSet API changes. Exp 176 extended exp 158's identity fast path to Row.containsKey via a shared RowSchema.containsName helper; the two lookup methods (operator[]/indexOf and containsKey/containsName) now share one identity-then-HashMap path, so any future change to the 32-column cap or the identity scan applies to both. Do not retry the exp 141 ResultSet.forEach override unless a future Dart runtime or workload produces a stable target win with neutral controls. Do not retry Row.values ListBase/getRange/fixed-slice views from JIT row_map_facade output alone; exp 193 showed the JIT values lane can flip between ~3 ms and ~9 ms while AOT cleanly rejects the list view. Compile the focused harness before changing Row.values again. For selectBytes transfer-policy work, include the exp 175 `selectBytes() large bytes` metric so large native-byte payloads stay visible. Exp 183 wired a reclaim path for the bounded RSS high-water exp 174 left as future work: the 1 MB cap trigger + 256 KB last-len guard is what makes the policy safe under back-to-back large reads. Any future tuning should raise the trigger (e.g., 4 MB) before touching the guard. The new Diagnostics.readerJsonBufHighWaterBytes is the reusable signal for any future RSS-sensitive selectBytes work — including the broader measurement-system 'memory profiling harness with per-benchmark RSS acceptance criteria' candidate, which can build on top of it rather than from scratch. Exp 185 adds the public release guard for that signal in `benchmark/suites/sqlite_diagnostics.dart`; keep the `JSON buffer reclaim` row green before changing the selectBytes shrink thresholds or last-len guard. Exp 195 promoted exp 190's per-query `tokens_buf` + `token_offsets` / `token_lens` arrays into the `resqlite_cached_stmt` entry; the exp 190 1-row regression guard sits at the millisecond-reporting floor of `wide_cols.dart` (not noise — wrong harness resolution), and the new `select_bytes_repeated_calls.dart` (1000 calls/sample at µs precision) exposes the win on 1-row × 20-col (−9.2 % / −7.2 %) and 10-row × 20-col (−5.1 % / −2.7 %) across two order-flipped passes. Further selectBytes encoder amortization beyond exp 195 should attach onto the cache entry (per-column type hints, value prefixes) only if a workload shows repeated small selectBytes() with a specific structural pattern dominating wall time; the harness for that work is `select_bytes_repeated_calls.dart`, not `wide_cols.dart`." + }, + { + "id": "measurement-system", + "status": "active", + "subsystems": [ + "benchmarking", + "profiling", + "methodology" + ], + "currentRead": "Several plausible optimizations failed because the benchmark did not stress the target path or because run noise hid small effects. Measurement work can be the highest-signal experiment when it unlocks a named implementation or rejection decision, but scheduled runners should first look for an instrument-and-implement path. Exp 119/121/136/147 are the recent stream-dispatch examples: exp 119 located surviving dispatch pressure in stream admission, exp 121 ruled out invalidation traversal as a wall-time target (10–15% of overlap wall, intersection ~3–6%), exp 136 added the completion-side reader-handler counter plus drain-aware audit snapshots and found 28.57% of A11c overlap total wall in the reader worker port handler chain at ~18 us per call, and exp 147 added the writer SQLite wall split and found SQLite-facing writer calls are not the active stream-fanout bottleneck on A11c overlap or keyed-PK. Exp 143 confirmed the pinned Tracelite profile path is useful because it captures dispatch floors, floor-subtracted work, memory diagnostics, allocation counters, source provenance, and graph data in one run; exp 169 consumes its interpretation gap by validating that `tracelite explain` emits workload-summary insight IDs for dispatch floors, work-bound operations, tail spread, RSS, allocation, and WAL signal before the profile wrapper completes. Exp 161 closes the matching release-suite gap for the writer side by promoting exp 159's concurrent-burst shape into `benchmark/suites/writes.dart` as a paired Single Inserts (sequential) / Concurrent Single Inserts (concurrent) row pair; the resqlite concurrent median (~1.1 ms) is now ~60% below the sequential median (~2.9 ms) on a public lane, and future writer-scheduling experiments can claim release-suite wins without depending on the focused `writer_pipelining.dart` script. Exp 177 mechanizes the JOURNAL's order-flipped drift check: `cvPct` + `classifyDriftFlag` in `benchmark/shared/stats.dart` and a `benchmark/ab_drift_check.dart` CLI classify a phase-ordered A/B regression flag as reproduced / drift-suspected / inconclusive from two order-flipped passes of per-run values, reproducing the manual verdicts on the recorded exp 159 (CV asymmetry) and exp 167 (sign reversal) flags. It is methodology tooling (exp 161 / 169 class), not a wall-time change, so future A/B runners can cite a deterministic verdict instead of re-deriving the CV-asymmetry rule each time. Exp 178 closes a silent pipeline gap on the experiment->chart linker itself: `generate_history.dart` already failed the build when an Accepted experiment linked a baseline-shaped run while a candidate existed (`_assertAcceptedExperimentsLinkToCandidates`), but it tolerated the more common case — a chartable experiment with NO linked run and NO `**Benchmark Run:**` opt-out, which silently drops off the chart when a runner forgets the result file or mismatches its date. A structural tally on main showed only ~5 of 23 null-run accepted/in-review experiments declared the opt-out. Exp 178 adds `_assertNewExperimentsLinkOrDeclareRun` (over a pure `findUndeclaredMissingRunExperiments` detector) that fails the build for accepted/in-review experiments numbered >= 178 with a null run and no opt-out declaration; pre-178 experiments are grandfathered via a cutoff constant (same pattern as `experimentEntriesRequiredFrom`). No runtime code, history.json unchanged.", + "keyPriors": [ + "136", + "143", + "147", + "169", + "177", + "178" + ], + "archive": [ + "055", + "088", + "099", + "102", + "108", + "113", + "115", + "116", + "119", + "121" + ], + "interestingIf": [ + "a profiler or focused benchmark exposes a cost hidden by the release suite", + "a control metric can identify noisy runs before results are trusted", + "the measurement makes an entire class of ideas easier to evaluate" + ], + "openQuestions": [ + "Which allocation-oriented ideas need memory/RSS acceptance criteria?", + "Which stream workloads are missing from the current suite?" + ], + "openCandidates": [ + { + "idea": "memory profiling harness with per-benchmark RSS acceptance criteria", + "addedDate": "2026-05-02" + } + ], + "blockedOnMeasurement": [], + "notesForExperimenters": "It is valid for a scheduled run to improve measurement rather than implementation only when the missing signal is the bottleneck and the same branch cannot reasonably run the candidate it enables. Use the Tracelite profile wrapper before adding a legacy profile harness: exp 143 shows the structured artifacts already expose dispatch-vs-work and memory/allocation shape, and exp 169 makes the generated insight layer a required profile artifact instead of a manual JSON inspection step. If profile data still answers a question only after manual inspection, prefer improving Tracelite explanation rules or the Resqlite wrapper guard over duplicating the measurement path. For scoped resqlite counters, start them as temporary instrumentation inside the performance branch; keep them only when future experiments will reuse the counter or it replaces a weaker legacy path. Exp 136 is the main-isolate pattern: add the counter, snapshot inside the shared `audit_workloads.dart` scenario runners with both at-burst-end and post-drain snapshots, and have the harness pick the right one for its question. Exp 147 is the writer-isolate pattern: time the writer-side slice in the worker and aggregate it back through responses. Exp 178 added a build-time guard against silently-unmapped chartable experiments; if you legitimately have no release run (Tracelite A/B or focused harness), declare it with a `**Benchmark Run:**` header (`none` / `n/a` / `tracelite ...`) or the generator will fail. To pull older unmapped experiments under the guard, backfill their headers and lower `_benchmarkRunDeclarationCutoff` in the same change." + } + ] +} diff --git a/experiments/signals/entries/088.json b/experiments/signals/entries/088.json new file mode 100644 index 00000000..879f8a32 --- /dev/null +++ b/experiments/signals/entries/088.json @@ -0,0 +1,13 @@ +{ + "directions": [ + "measurement-system" + ], + "outcomeClass": "rejected_after_noise_check", + "changedBeliefs": [ + "Single-run tail wins need confirmation against control metrics before acceptance" + ], + "nextSignals": [ + "concurrent-reader lock-contention harness", + "shorter lock timeout sweep only with a workload that shows lock wait" + ] +} diff --git a/experiments/signals/entries/090.json b/experiments/signals/entries/090.json new file mode 100644 index 00000000..3691b28c --- /dev/null +++ b/experiments/signals/entries/090.json @@ -0,0 +1,13 @@ +{ + "directions": [ + "sqlite-version-and-build-config" + ], + "outcomeClass": "watch", + "changedBeliefs": [ + "sqlite3mc version bumps should wait for safer point releases when the newest SQLite release is a fresh .0" + ], + "nextSignals": [ + "sqlite3mc release tracking a stable SQLite point release", + "planner/API changes that map to resqlite hot paths" + ] +} diff --git a/experiments/signals/entries/099.json b/experiments/signals/entries/099.json new file mode 100644 index 00000000..8dcc015a --- /dev/null +++ b/experiments/signals/entries/099.json @@ -0,0 +1,13 @@ +{ + "directions": [ + "long-text-stream-hashing", + "measurement-system" + ], + "outcomeClass": "benchmark_gap", + "changedBeliefs": [ + "Byte-stream hash-loop changes need long-text stream workloads before they are meaningfully testable" + ], + "nextSignals": [ + "representative long-text streaming benchmark" + ] +} diff --git a/experiments/signals/entries/100.json b/experiments/signals/entries/100.json new file mode 100644 index 00000000..58a7de51 --- /dev/null +++ b/experiments/signals/entries/100.json @@ -0,0 +1,12 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_regression", + "changedBeliefs": [ + "Bounded rerun scheduling can harm high-cardinality stream fan-out even if it looks attractive for unrelated reads" + ], + "nextSignals": [ + "better separation of unrelated read latency from fan-out throughput" + ] +} diff --git a/experiments/signals/entries/101.json b/experiments/signals/entries/101.json new file mode 100644 index 00000000..99e6c578 --- /dev/null +++ b/experiments/signals/entries/101.json @@ -0,0 +1,12 @@ +{ + "directions": [ + "transaction-control-paths" + ], + "outcomeClass": "accepted", + "changedBeliefs": [ + "Top-level transaction control still had removable SQLite prepare/finalize work" + ], + "nextSignals": [ + "nested transaction benchmark before revisiting savepoint control" + ] +} diff --git a/experiments/signals/entries/103.json b/experiments/signals/entries/103.json new file mode 100644 index 00000000..1cce0a51 --- /dev/null +++ b/experiments/signals/entries/103.json @@ -0,0 +1,12 @@ +{ + "directions": [ + "transaction-control-paths" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "Native nested transaction helpers add complexity without a stable realistic-workload win" + ], + "nextSignals": [ + "realistic deeply nested transaction workload" + ] +} diff --git a/experiments/signals/entries/105.json b/experiments/signals/entries/105.json new file mode 100644 index 00000000..43cea786 --- /dev/null +++ b/experiments/signals/entries/105.json @@ -0,0 +1,12 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_regression", + "changedBeliefs": [ + "Increasing reader worker count can worsen writer throughput by increasing completion-side microtask churn" + ], + "nextSignals": [ + "queueing trace around completion and writer throughput" + ] +} diff --git a/experiments/signals/entries/108.json b/experiments/signals/entries/108.json new file mode 100644 index 00000000..7fde3a75 --- /dev/null +++ b/experiments/signals/entries/108.json @@ -0,0 +1,12 @@ +{ + "directions": [ + "measurement-system" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "Persistent selectBytes out-parameter slots are below the current benchmark and RSS signal" + ], + "nextSignals": [ + "profile evidence before retrying tiny scratch allocation changes" + ] +} diff --git a/experiments/signals/entries/109.json b/experiments/signals/entries/109.json new file mode 100644 index 00000000..6985d689 --- /dev/null +++ b/experiments/signals/entries/109.json @@ -0,0 +1,12 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "accepted", + "changedBeliefs": [ + "Parameter encoding still has worthwhile headroom when the change removes per-text/blob native allocations and SQLite strlen work" + ], + "nextSignals": [ + "look for larger ownership/layout changes rather than tiny scratch-object reuse" + ] +} diff --git a/experiments/signals/entries/110.json b/experiments/signals/entries/110.json new file mode 100644 index 00000000..f77fcac9 --- /dev/null +++ b/experiments/signals/entries/110.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "long-text-stream-hashing", + "measurement-system" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The missing long-text unchanged-fanout benchmark was the blocker for evaluating byte-stream hash-loop work", + "The archived 8-byte FNV byte fold becomes a clear win once the workload carries 4KB TEXT cells" + ], + "nextSignals": [ + "watch normal short-cell streaming metrics for platform-specific regressions", + "use the long-text unchanged-fanout benchmark before trying future hash-loop variants" + ] +} diff --git a/experiments/signals/entries/111.json b/experiments/signals/entries/111.json new file mode 100644 index 00000000..938fd402 --- /dev/null +++ b/experiments/signals/entries/111.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "transaction-control-paths", + "measurement-system" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "Even on a worst-case shallow-fan-out workload (50 SAVEPOINTs per iteration), the savepoint string allocation cost is below the per-benchmark decision threshold — per-isolate-round-trip cost dominates", + "The missing nested-transaction benchmark from exp 102 / exp 103 is now in the suite, so future savepoint-related experiments can be evaluated without forcing the same gap" + ], + "nextSignals": [ + "look for round-trip-batching opportunities for savepoint open/close pairs (analogous to exp 009 read-side batching)", + "compare any future savepoint-path change against the exp 111 nested-tx benchmark before claiming a win" + ] +} diff --git a/experiments/signals/entries/112.json b/experiments/signals/entries/112.json new file mode 100644 index 00000000..15a6c80b --- /dev/null +++ b/experiments/signals/entries/112.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "parameter-encoding-and-binding", + "measurement-system" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "Pre-sizing the temporary executeBatch flat parameter list is below the current benchmark signal", + "The shared inline parameter encoder should stay centralized unless a profiler proves batch flattening is material" + ], + "nextSignals": [ + "writer-isolate profile showing flat-list construction as a material batch-write cost", + "batch workload with much wider parameter rows than the current two-param INSERT shape" + ] +} diff --git a/experiments/signals/entries/113.json b/experiments/signals/entries/113.json new file mode 100644 index 00000000..35a9dc8a --- /dev/null +++ b/experiments/signals/entries/113.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "parameter-encoding-and-binding", + "measurement-system" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The two-parameter batch insert shape is not a sufficient proxy for batch parameter overhead", + "Avoiding the temporary flat Dart parameter list becomes measurable once batch rows are wide enough" + ], + "nextSignals": [ + "watch release-suite two-parameter batch metrics for neutrality", + "include parameter width when evaluating future executeBatch encoder changes" + ] +} diff --git a/experiments/signals/entries/114.json b/experiments/signals/entries/114.json new file mode 100644 index 00000000..c16c8fda --- /dev/null +++ b/experiments/signals/entries/114.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "Reader-pool wake amplification (single shared completer waking every parked dispatcher) was a real but workload-dependent cost: -10 % to -32 % on streaming fan-out paths against pre-exp-106 main, then collapsed into noise once exp 106 polish landed and elided most stream re-queries on the writer side", + "Reader-pool-internal dispatch optimizations need a workload that exp 106's column-level elision cannot short-circuit (reads-only contention, or streams whose projections all intersect the modified columns) — without one, the parked-dispatcher path is rarely hit and the wins are unmeasurable" + ], + "nextSignals": [ + "a long-running concurrent-reads workload (or any non-streaming workload) that sustains parked dispatchers past the worker count for measurable durations (now verifiable via exp 115's `dispatcherMaxParkedConcurrent` counter)", + "a streaming workload whose stream projections all intersect every write's modified columns, defeating exp 106's column-tracking elision", + "exp 115 shipped the profile-mode counters; future re-evaluation can require `dispatcherWakeRetryTotal > 0` on the workload before judging a wall-time delta" + ] +} diff --git a/experiments/signals/entries/115.json b/experiments/signals/entries/115.json new file mode 100644 index 00000000..9f4affa0 --- /dev/null +++ b/experiments/signals/entries/115.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "measurement-system", + "stream-rerun-dispatch" + ], + "outcomeClass": "accepted_measurement", + "changedBeliefs": [ + "The parked-dispatcher path inside `ReaderPool._dispatch` is now directly observable via `ProfileCounters.dispatcherParkedTotal`, `dispatcherWakeRetryTotal`, and `dispatcherMaxParkedConcurrent` — gated behind `kProfileMode` so release builds stay zero-cost", + "Future dispatch-area experiments (exp 114 archive, exp 083, slot-handoff variants) can gate evaluation on direct parking evidence rather than wall-time delta alone, closing the exp 099 / exp 110 evaluation-gap pattern earlier in the loop" + ], + "nextSignals": [ + "use the new counters when re-evaluating exp 114's archived FIFO swap or other reader-pool-internal dispatch changes", + "profile real workloads (concurrent reads under writer, A11c stream fan-out post-exp-106) through the counters before choosing the next dispatch experiment target" + ] +} diff --git a/experiments/signals/entries/116.json b/experiments/signals/entries/116.json new file mode 100644 index 00000000..c320543d --- /dev/null +++ b/experiments/signals/entries/116.json @@ -0,0 +1,13 @@ +{ + "directions": [ + "parameter-encoding-and-binding", + "measurement-system" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The release write suite now covers parameter width directly instead of relying on the two-parameter batch insert as the only public batch proxy" + ], + "nextSignals": [ + "compare future batch-parameter changes against both narrow Batch Insert and Wide Batch Insert before claiming neutrality" + ] +} diff --git a/experiments/signals/entries/117.json b/experiments/signals/entries/117.json new file mode 100644 index 00000000..3c804542 --- /dev/null +++ b/experiments/signals/entries/117.json @@ -0,0 +1,14 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "deferred", + "changedBeliefs": [ + "Named-parameter support is functionally viable but too invasive for the v0.x launch path unless real user demand justifies widening the public parameter API", + "The focused wide-batch rerun suggested the apparent release-suite regression was mostly noise, but the write-path metrics still trended slower enough that a pre-launch performance package should defer the ergonomics win" + ], + "nextSignals": [ + "reopen named parameters only after community feedback shows they are an adoption blocker, or after a specialized positional-vs-named bind loop benchmarks neutral across a 5-run release suite", + "if reopened, cache name-to-bind-index lookups per statement cache entry before claiming named-bind performance is settled" + ] +} diff --git a/experiments/signals/entries/118.json b/experiments/signals/entries/118.json new file mode 100644 index 00000000..020c244a --- /dev/null +++ b/experiments/signals/entries/118.json @@ -0,0 +1,14 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The exp 115 counters convert exp 114's previously ambiguous FIFO waiter idea into a direct measurement: under overloaded reads, wake retries drop from 6/66/378 to 0 at concurrency 8/16/32 while max parked depth remains unchanged", + "Reader-pool shared-completer wake amplification is now a solved target; future dispatch work needs a different counter signal before adding more queue policy" + ], + "nextSignals": [ + "profile real app-shaped workloads after FIFO waiters to see whether any nonzero dispatcherWakeRetryTotal remains", + "look for completion-side batching or admission-level work only if dispatcherMaxParkedConcurrent stays high with wall-time impact" + ] +} diff --git a/experiments/signals/entries/119.json b/experiments/signals/entries/119.json new file mode 100644 index 00000000..a8cba652 --- /dev/null +++ b/experiments/signals/entries/119.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "Post-FIFO app-shaped workloads keep `dispatcherWakeRetryTotal` at zero, so ReaderPool wake policy is no longer the active dispatch target", + "A11c overlap and keyed-PK workloads still produce high parked-dispatcher counts despite coalesced or hash-suppressed visible emissions, pointing the next dispatch experiment toward stream re-query admission/completion" + ], + "nextSignals": [ + "exp 120/122 satisfied the stricter StreamEngine `_flushQueue` admission path; only revisit if A11c overlap or keyed-PK counters become nonzero again", + "branch away from dispatch if admission changes reduce counters without moving wall time" + ] +} diff --git a/experiments/signals/entries/120.json b/experiments/signals/entries/120.json new file mode 100644 index 00000000..4e7ab221 --- /dev/null +++ b/experiments/signals/entries/120.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The 3,590-park / 46-max signal exp 119 measured on A11c overlap was upstream over-dispatch from `StreamEngine._flushQueue`, not reader-pool admission proper. Snapshotting `availableWorkerCount` once and decrementing per pop drops parking to zero on overlap and keyed-PK, with disjoint unchanged and high-cardinality fan-out (the exp-100 killer) within ±10% noise", + "After this, every measured stream workload reports `dispatcherParkedTotal == 0` on top of `dispatcherWakeRetryTotal == 0`. Future reader-pool dispatch optimizations need a different counter (completion-side scheduling, writer-side dispatch wall, invalidation traversal) before they are worth implementing" + ], + "nextSignals": [ + "profile completion-side microtask scheduling cost on A11c overlap", + "exp 147 split writer-isolate wall vs SQLite wall; follow up on residual writer/request scheduling rather than SQLite-step tuning", + "audit invalidation traversal cost (`invalidateUs`/`intersectionUs`) as a fraction of overlap wall" + ] +} diff --git a/experiments/signals/entries/121.json b/experiments/signals/entries/121.json new file mode 100644 index 00000000..2664a625 --- /dev/null +++ b/experiments/signals/entries/121.json @@ -0,0 +1,18 @@ +{ + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "Under a corrected wall convention (stopwatch stops on the last write, not after a fixed drain), invalidation traversal is at the *edge* of the wall-time noise floor on A11c overlap (10–15% of wall, intersection 2.5–5.7%) and keyed-PK (13.5–13.9%, intersection ~4%). The earlier ~7%/~1.6% figures were biased low by an arbitrary 50 ms drain sleep included in the wall denominator", + "Per-write `onDependencyChanges` cost is workload-shape-stable at ~16–27 µs A11c, ~14 µs keyed-PK. Removing the entire path would save ~8–14 ms on a 100 ms A11c overlap burst — measurable in the focused harness, but at the per-benchmark decision threshold edge of the release suite", + "Column-set intersection is already O(1) bitset (80–200 ns per probe, ~3 ms total per overlap burst); the rest of the traversal is `_tableIndex` map lookup, dirtyEntries Set.add, and the synchronous portion of `_flushQueue` — none obviously reducible without a dependency-model redesign", + "Wall-clock measurement convention for stream-fanout audits should stop the stopwatch on the last write, not after a fixed drain or quiet-window wait, and emission counts should be read after the stopwatch stops. Both `audit_workloads.dart` runners enforce this; future audits inherit it" + ], + "nextSignals": [ + "build a completion-side microtask scheduling cost counter and re-audit A11c overlap", + "exp 147 built the writer-isolate wall vs SQLite split; use it to steer follow-up toward residual writer/request or completion-side scheduling", + "until a non-zero counter signal appears for one of those, dispatch-area implementation experiments stay on hold" + ] +} diff --git a/experiments/signals/entries/122.json b/experiments/signals/entries/122.json new file mode 100644 index 00000000..866c38b4 --- /dev/null +++ b/experiments/signals/entries/122.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Exp 120 fixed the per-flush over-dispatch signal; exp 122 removes the remaining async handoff by constructing `StreamEngine` with a concrete `ReaderPool`, so `_flushQueue` stays synchronous and admitted reads reach pool dispatch without awaiting a pool future", + "The post-rebase profile pass on top of exp 120 and exp 121 preserves zero `dispatcherParkedTotal`, zero `dispatcherWakeRetryTotal`, and zero max parked depth on A11c overlap and keyed-PK workloads, while tests now verify stream registry cleanup through `Database.diagnostics().streamLength` instead of a private stream-engine getter" + ], + "nextSignals": [ + "do not pursue more ReaderPool or stream-admission work unless exp 115 counters become nonzero again on an app-shaped workload or a new overlap shape breaks the concrete-pool admission invariant", + "look for keyed/row-level invalidation or observer APIs to reduce keyed-PK miss-path work beyond what admission accuracy can solve" + ] +} diff --git a/experiments/signals/entries/125.json b/experiments/signals/entries/125.json new file mode 100644 index 00000000..5de73d6f --- /dev/null +++ b/experiments/signals/entries/125.json @@ -0,0 +1,14 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Large wide ASCII-heavy batches still had removable per-string UTF-8 allocation after exp 113's matrix encoder", + "A guarded ASCII fast path can improve wide batch parameter packing without changing the public API or weakening Unicode fallback behavior" + ], + "nextSignals": [ + "watch release-suite Wide Batch Insert and narrow Batch Insert together", + "benchmark non-ASCII-heavy wide batches before broadening the fallback path" + ] +} diff --git a/experiments/signals/entries/126.json b/experiments/signals/entries/126.json new file mode 100644 index 00000000..ff79dedf --- /dev/null +++ b/experiments/signals/entries/126.json @@ -0,0 +1,14 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Large wide non-ASCII batches have the same removable per-string allocation pattern exp 125 found for ASCII: direct UTF-8 payload writing improves focused Unicode 10k x20 by 13.5% and emoji 10k x20 by 27.8%", + "Dart-compatible surrogate-pair and replacement-character encoding can stay private to the guarded batch encoder while preserving embedded-NUL byte lengths through sqlite3_bind_text" + ], + "nextSignals": [ + "watch release-suite Wide Batch Insert and narrow Batch Insert together because the public suite is still ASCII-heavy", + "only pursue blob-heavy or broader embedded-NUL work with a workload that crosses the same large/wide guard" + ] +} diff --git a/experiments/signals/entries/134.json b/experiments/signals/entries/134.json new file mode 100644 index 00000000..c79ce218 --- /dev/null +++ b/experiments/signals/entries/134.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_preserved_potential", + "changedBeliefs": [ + "Row-level dirty precision can remove the keyed-PK miss-path cost: verified simple `WHERE id = ?` INTEGER PRIMARY KEY streams skipped when dirty rowids did not overlap", + "The keyed-PK profile workload reached per-stream column intersection only for actual watched-row hits: `intersection_entries` dropped 10000 → 3 and writer-burst wall dropped 25.54 → 12.45 ms", + "The implementation is rejected because internal SQL text recognition is too fragile to grow into a production dependency model; preserve the result as evidence for a future explicit row observer API, trace metadata, or stronger dependency model" + ], + "nextSignals": [ + "do not retry this by broadening the SQL/schema recognizer; use `archive/exp-134` only as implementation evidence", + "revive row-level dependency precision only if a real workload shows keyed-PK miss writes dominate or the API/design can express watched row identity directly" + ] +} diff --git a/experiments/signals/entries/136.json b/experiments/signals/entries/136.json new file mode 100644 index 00000000..7f1930d0 --- /dev/null +++ b/experiments/signals/entries/136.json @@ -0,0 +1,18 @@ +{ + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "On A11c overlap the reader-worker port handler accounts for 28.57% of total wall (burst + drain) at ~18 µs per call across 4,228 calls per burst — the largest remaining reachable main-isolate slice", + "Subscriber-fanout emit is < 1% of the completion chain on every measured workload; batching `controller.add` is not the optimization target", + "Most reader replies on A11c overlap short-circuit via `selectIfChanged` hash comparison (4,228 completions → 29 emits in the fresh pass); the per-call cost is handler bootstrap + Future resolution + hash check + `_flushQueue` admit, not real query result work", + "Most reader-completion wall fires AFTER the writer-burst stopwatch stops; the shared `audit_workloads.dart` now snapshots counters BOTH at burst-end and after the drain so future main-isolate audits can pick the right denominator. A11c drain switched from a fixed 50 ms wait to the same quiet-window pattern keyed-PK already uses" + ], + "nextSignals": [ + "evaluate reader-reply batching as the bounded implementation candidate that follows: must drop `completion_us / total_us` on A11c overlap AND stay neutral on A11c disjoint and keyed-PK (otherwise overall release-suite delta is workload-specific)", + "treat `stream_emit_us` as evidence-of-absence for subscriber-fanout optimization candidates until a workload with very many listeners per stream surfaces", + "future main-isolate counters should follow the exp 136 pattern (counter + post-drain snapshot in `audit_workloads.dart`) rather than redoing the harness wiring" + ] +} diff --git a/experiments/signals/entries/142.json b/experiments/signals/entries/142.json new file mode 100644 index 00000000..9997493f --- /dev/null +++ b/experiments/signals/entries/142.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "rejected_no_signal", + "changedBeliefs": [ + "Direct single-row text parameter encoding did not produce current Tracelite production evidence despite the old focused-harness win from PR #130", + "The retest over chat-sim and narrow-batch-insert did not clear the primary gate: resqlite changed +6.86% and +16.4% with neutral/inconclusive verdicts and high CV", + "Small single-row string binding should stay on the generic `allocateParams` path unless a future workload makes that encoding cost material" + ], + "nextSignals": [ + "do not carry the PR #130 `allocateParams` direct UTF-8 path without a Tracelite A/B decision that clears a scenario where single-row string binding is material", + "if high-frequency string-parameter reads/writes become important, add a focused Tracelite scenario or profile lane before retrying the implementation", + "continue comparing parameter-encoding changes against both narrow batch and wider app-shaped lanes rather than accepting focused microbenchmark wins alone" + ] +} diff --git a/experiments/signals/entries/143.json b/experiments/signals/entries/143.json new file mode 100644 index 00000000..1c3447be --- /dev/null +++ b/experiments/signals/entries/143.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "Pinned Tracelite profile runs already produce decision-useful structured evidence: dispatch floors, floor-subtracted work, operation tails, memory deltas, allocation counters, source provenance, and validated graph data", + "The current canonical profile workload shows point queries are median dispatch-bound (0-1 us of floor-subtracted work), merge rounds are work-bound (77 us of floor-subtracted work), and point queries still carry allocation/RSS cost that wall time alone hides", + "`tracelite explain` is not yet doing enough interpretation for resqlite experimenters; it loaded the workload summary but did not identify dispatch-bound, work-bound, memory-heavy, or tail-noisy workloads" + ], + "nextSignals": [ + "add Tracelite workload-summary explain rules before adding another resqlite-local interpretation harness", + "keep raw trace regions local while committing aggregate markdown and graph data only when it feeds Pages", + "use repeated Tracelite profile runs before making p99 or max-tail claims" + ] +} diff --git a/experiments/signals/entries/144.json b/experiments/signals/entries/144.json new file mode 100644 index 00000000..0548ddb9 --- /dev/null +++ b/experiments/signals/entries/144.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "sqlite-version-and-build-config" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Once SQLite ships a `.2` point release and sqlite3mc cuts a tracking release, exp 090's vendoring-bump audit moves from `defer` to `do` without any further policy debate — the 3.53.x line is the canonical case", + "The 3.53.0 default-FP-rounding hazard does not reach resqlite output because we never use `sqlite3_column_text` for REAL values; the proposed `SQLITE_DBCONFIG_FP_DIGITS=15` shim is unnecessary", + "Single-pass release-suite A/B on a vendoring bump is direction signal only; the first pass on this worktree showed 19/18/124 and the rerun 30/2/129, so a regression flag should be reconfirmed across two independent passes before being treated as real", + "The Concurrent Reads 8× wall-median metric is the only regression that persisted across both candidate passes (+~20%); 4× concurrency wins on the same baseline, so the slowdown is not a generic read-pool effect and is the canonical soak-window question for this bump" + ], + "nextSignals": [ + "promote exp 144 from In Review to Accepted once the soak window closes with no release-suite regression", + "if a multi-pass rerun of Concurrent Reads 8× under 3.53.x stays slower than 2.3.2-baseline, open a follow-up against the read-pool path or revert the bump", + "future bump audits should run two independent single-pass A/Bs before claiming a behavioral change in SQLite or release-suite signal" + ] +} diff --git a/experiments/signals/entries/145.json b/experiments/signals/entries/145.json new file mode 100644 index 00000000..7ba471f9 --- /dev/null +++ b/experiments/signals/entries/145.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "`StreamEngine._flushQueue` helper allocation is not the active stream-dispatch target after exp 120 and exp 122; replacing `take(...).toList()` and `where(...).length` with inline loops kept dispatch counters at zero but produced mixed wall-time movement", + "A11c overlap can move in a favorable direction from tiny queue-shape changes, but keyed-PK noise/regression risk means collection-helper cleanup needs allocator evidence before it is worth carrying", + "The current stream-admission path is behaviorally settled; future stream work should try bounded completion-side scheduling or writer/request changes with in-branch measurement instead of polishing the dequeue loop" + ], + "nextSignals": [ + "do not retry `_flushQueue` inline dequeue unless an allocation profile identifies the temporary list or availability count as material", + "if a stream implementation needs a completion-side scheduling counter, add it in the same branch and remove it before merge unless it becomes reusable", + "use the existing A11c/keyed-PK profile counter gates to reject stream-admission cleanups whose counters are already zero" + ] +} diff --git a/experiments/signals/entries/146.json b/experiments/signals/entries/146.json new file mode 100644 index 00000000..854f0a13 --- /dev/null +++ b/experiments/signals/entries/146.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "rejected_no_signal", + "changedBeliefs": [ + "Lowering the ASCII batch-packing guard from 8 params / 8192 total params to 2 params / 64 total params did not improve the Tracelite narrow-batch-insert primary lane", + "The clean A/B run measured resqlite at +1.45% with neutral verdict and 13.2% max CV, while the sqlite_async guardrail was too noisy to add confidence", + "Small and narrow batch writes should stay on the generic path unless a future workload shows parameter encoding is a material part of wall time" + ], + "nextSignals": [ + "do not broaden the exp 125 ASCII fast path to small/narrow batches without a new workload and a Tracelite A/B decision that clears the primary gate", + "when experimenting in this direction, prefer the integrated Tracelite A/B wrapper so baseline/candidate histories, policy, decision, insights, and graph data stay together" + ] +} diff --git a/experiments/signals/entries/147.json b/experiments/signals/entries/147.json new file mode 100644 index 00000000..ebe0a5ee --- /dev/null +++ b/experiments/signals/entries/147.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "SQLite-facing write calls are a minority of writer-side burst wall on active stream workloads: A11c overlap is 9.4% SQLite and keyed-PK is 18.1% SQLite", + "After subtracting SQLite-facing calls and stream invalidation, residual writer/request wall remains the largest bucket on A11c overlap and keyed-PK", + "Future stream-dispatch work should target completion-side scheduling, reply/request coordination, or dirty-set harvest rather than SQLite-step tuning, and should carry any narrow measurement inside the implementation branch" + ], + "nextSignals": [ + "reader-reply batching candidate should reduce exp 136 completion_us / total_us on A11c overlap", + "if residual writer/request detail is needed, gather only the split required by a concrete reduction candidate and remove temporary scaffolding before merge unless it is reusable" + ] +} diff --git a/experiments/signals/entries/148.json b/experiments/signals/entries/148.json new file mode 100644 index 00000000..b0787cb3 --- /dev/null +++ b/experiments/signals/entries/148.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "Plain worker-side reader-reply batching reduces completion callback counters but does not produce a mergeable measured-elapsed win on the stream-dispatch Tracelite suite", + "The profile smoke cut A11c overlap completion callbacks from 4,527 to 1,425 and completion wall from 109.6 ms to 55.6 ms, but the formal A/B still measured high-cardinality fanout at +5.18%, many-streams writer throughput at +3.28%, and keyed-PK subscriptions at +13.5%", + "Completion callback count alone is not a sufficient acceptance signal; future stream work needs direct measured-elapsed evidence plus only the in-branch residual measurement needed to explain the result" + ], + "nextSignals": [ + "do not retry the same SelectIfChanged batch-response protocol unless a new workload shows callback count dominates measured elapsed", + "do not run another standalone residual split before choosing a target; pick a concrete dirty-set, writer reply, request-resolution, or drain coordination change and measure that candidate directly", + "keep using the integrated Tracelite A/B wrapper for implementation attempts because it prevented a counter-only optimization from being merged" + ] +} diff --git a/experiments/signals/entries/149.json b/experiments/signals/entries/149.json new file mode 100644 index 00000000..64280149 --- /dev/null +++ b/experiments/signals/entries/149.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Tracelite profile merge rounds expose a material six-parameter batch shape between exp 146's rejected small/narrow guard and exp 125/126's large wide guard", + "Lowering the ASCII batch-packing guard to 6 params / 600 total params improved merge-round executeBatch p50 from 88 us to 75 us and writer SQLite time from 87,895 us to 75,947 us", + "The 2-3 parameter paths remain generic; this is a bounded middle-ground guard, not a retry of exp 146's broad small-batch threshold" + ], + "nextSignals": [ + "watch for release-suite or downstream evidence that six-parameter merge batches are common enough to deserve a strict Tracelite policy scenario", + "do not lower the guard below six parameters without a new workload and a Tracelite decision artifact that clears the primary gate", + "if six-parameter merge rounds become a recurring product workload, add an explicit Tracelite suite scenario instead of treating narrow-batch-insert as a proxy" + ] +} diff --git a/experiments/signals/entries/150.json b/experiments/signals/entries/150.json new file mode 100644 index 00000000..3bf7fe40 --- /dev/null +++ b/experiments/signals/entries/150.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The first-row string probe missed nullable generated-statement batches where row 0 carries NULL text columns and later rows carry strings", + "Treating first-row NULL values as possible text admits nullable batches only after the existing 6-param / 600-total guard, preserving exp 146's small/narrow rejection", + "A single batch payload classifier can choose ASCII versus UTF-8 packing without a separate ASCII-probe pass" + ], + "nextSignals": [ + "watch nullable generated-statement workloads for evidence that this shape deserves a strict Tracelite scenario", + "keep 2-3 parameter writes on the generic path unless a future workload clears a Tracelite A/B primary gate", + "if blob-heavy parameter shapes become interesting, benchmark them separately because this run targeted text-nullability rather than blob-only packing" + ] +} diff --git a/experiments/signals/entries/151.json b/experiments/signals/entries/151.json new file mode 100644 index 00000000..24d443c9 --- /dev/null +++ b/experiments/signals/entries/151.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "Changing writer response futures from `Completer()` to `Completer.sync()` is a coherent request-resolution implementation attempt, but it does not clear the Tracelite stream-dispatch primary gate", + "The formal A/B measured high-cardinality fanout at +2.92%, keyed-PK subscriptions at +18.5% with too-noisy evidence, and many-streams writer throughput at +14.0%", + "Exp 147's residual writer/request bucket is not solved by making the existing writer response future synchronous; future work needs a more structural dirty-set, reply coordination, or workload-shape candidate" + ], + "nextSignals": [ + "do not retry synchronous writer response completion unless a Dart runtime change or new workload changes the request-resolution cost model", + "continue to avoid standalone residual-split profiling; carry only the narrow measurement needed by a concrete implementation candidate", + "use integrated Tracelite A/B primary gates before keeping stream scheduling changes, even when a profile smoke looks mixed or partially favorable" + ] +} diff --git a/experiments/signals/entries/158.json b/experiments/signals/entries/158.json new file mode 100644 index 00000000..e6d36dff --- /dev/null +++ b/experiments/signals/entries/158.json @@ -0,0 +1,19 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "A narrow private RowSchema lookup change can still improve full row consumption without changing the ResultSet/Row public API", + "The schema-name identity fast path plus HashMap fallback cut row_map_facade hot lookup from 10.750 -> 5.136 ms and keys+lookup from 20.602 -> 10.650 ms", + "A local width sweep showed the identity scan still ahead at 32 columns but behind by 48 columns, so the fast path is capped at 32 columns and unusually wide selects fall straight through to the HashMap", + "Focused select_maps main-isolate medians improved from 0.169 -> 0.081 ms at 1K rows and 1.998 -> 0.967 ms at 10K rows in the clean paired pass", + "Point-query throughput stayed neutral/noisy with overlapping confidence intervals, so the per-query schema construction cost did not show a hot single-row select regression" + ], + "nextSignals": [ + "keep result-shape experiments on full consumer benchmarks such as select_maps, not transfer/setup-only measurements", + "do not infer from exp 158 that larger ResultSet API changes are attractive; this win is limited to the private schema index", + "do not raise the 32-column identity-scan threshold without a fresh width sweep and a full-consumption benchmark covering wider schemas", + "watch release select_maps metrics during soak because the local 10K runs showed visible machine noise even though the clean paired pass favored the candidate" + ] +} diff --git a/experiments/signals/entries/159.json b/experiments/signals/entries/159.json new file mode 100644 index 00000000..c83935ed --- /dev/null +++ b/experiments/signals/entries/159.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The writer request path carried removable fixed cost on every round-trip: a per-request RawReceivePort, a guaranteed microtask hop awaiting the already-resolved worker-port future, and async reply completers — replacing them with a persistent reply port, cached SendPort, and sync FIFO completers drops exp 147 residual_us on all four audit workloads", + "Holding the write lock across the full round-trip for standalone writes added no exclusion a later BEGIN needs (the worker's port FIFO already orders them); releasing at send time pipelines concurrent standalone writes and improves the focused concurrent-burst benchmark 36-45%", + "Phase-ordered A/B collection (all baseline runs, then all candidate runs) is vulnerable to time-correlated machine drift: the first gate pass flagged +12-19% on stream scenarios that an order-flipped second pass showed were contamination (clean-pass deltas +1.02%, -0.26%, +4.00%, all CIs straddling zero)" + ], + "nextSignals": [ + "promote a concurrent standalone-write scenario into release or tracelite coverage (exp 116 pattern) so the pipelined path has public regression visibility", + "remaining sequential-write residual is the round-trip floor itself; the next structural candidates are cross-call request batching (group commit) or a different transport when the Dart SDK allows shared-memory result passing", + "when a tracelite gate flags a regression with elevated within-run CVs relative to the other phase, re-run with collection order flipped before treating the flag as real" + ] +} diff --git a/experiments/signals/entries/161.json b/experiments/signals/entries/161.json new file mode 100644 index 00000000..8ee07536 --- /dev/null +++ b/experiments/signals/entries/161.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "The exp 159 send-gated writer-lock pipelining shape is now visible on the release write suite as a paired Single Inserts (100 sequential) / Concurrent Single Inserts (100 concurrent) row pair using the same schema and parameter values", + "Resqlite concurrent median is ~1.1 ms vs sequential ~2.9 ms in two paired smoke runs (-58% to -61%); the pipelining effect lands in the same band as exp 159's focused script (-36% to -45%)", + "The sqlite3 (sync) concurrent row drops from sequential as well, because Future.wait skips the per-call await microtask cost — that row is the no-isolate-boundary floor and should not be read as evidence of pipelining" + ], + "nextSignals": [ + "any future writer-scheduling experiment in stream-rerun-dispatch can compare concurrent vs sequential rows on the public lane without depending on `benchmark/experiments/writer_pipelining.dart`", + "do not sweep burst sizes in release; focused sweeps stay in `benchmark/experiments/writer_pipelining.dart`", + "if a future change moves the sequential row but not the concurrent row (or vice versa), record that asymmetry rather than treating it as calibration drift" + ] +} diff --git a/experiments/signals/entries/164.json b/experiments/signals/entries/164.json new file mode 100644 index 00000000..5cbd3def --- /dev/null +++ b/experiments/signals/entries/164.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "stream-rerun-dispatch", + "measurement-system" + ], + "outcomeClass": "rejected_no_stable_signal", + "changedBeliefs": [ + "SQLite EXPLAIN QUERY PLAN can avoid a custom SQL parser for rowid lookup detection, but it did not make private row-level stream invalidation cheap enough to keep", + "The focused Tracelite stream-initial-drain rowid lane measured -2.33% with neutral/inconclusive evidence while text and indexed-int setup guardrails did not provide a clean offsetting signal", + "The broader stream-dispatch guard run stayed mostly neutral and did not turn the keyed-PK rowid idea into a stable end-to-end win" + ], + "nextSignals": [ + "do not keep native dirty-row harvesting plus plan inspection without a real workload or Tracelite primary lane that clearly pays for the extra dependency layer", + "use stream-initial-drain and warmup_elapsed_ns guardrails for future setup-heavy stream experiments", + "if row-level precision reopens, prefer a stronger dependency model over broadening resqlite-owned SQL recognition" + ] +} diff --git a/experiments/signals/entries/167.json b/experiments/signals/entries/167.json new file mode 100644 index 00000000..bebad75c --- /dev/null +++ b/experiments/signals/entries/167.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "rejected_no_stable_signal", + "changedBeliefs": [ + "Closed exp 141's direct ResultSet.forEach override does not show a stable current win after exp 158's RowSchema lookup change when measured on real rows returned by Database.select()", + "The focused SQLite-backed consumer lane showed Pair A forEach lookup at 30.525 -> 28.383 ms, but the longer confirmation reversed to 28.563 -> 30.942 ms; forEach length also reversed from 9.439 -> 8.010 ms to 8.652 -> 9.490 ms", + "The right durable artifact is the resultset_foreach_consumer benchmark lane, not the runtime override" + ], + "nextSignals": [ + "keep result-shape rechecks on SQLite-backed full-consumer lanes rather than synthetic ResultSet-only microbenchmarks", + "do not revive ResultSet.forEach override without a future Dart runtime or workload change and a stable target win with neutral for-in/indexed controls", + "AOT comparison would be useful if native asset resolution for standalone benchmark executables becomes straightforward" + ] +} diff --git a/experiments/signals/entries/169.json b/experiments/signals/entries/169.json new file mode 100644 index 00000000..9dc29535 --- /dev/null +++ b/experiments/signals/entries/169.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "The Tracelite workload-summary explanation layer now emits the interpretation exp 143 had to recover manually: dispatch floors, dispatch-bound point queries, work-bound merge rounds, tail spread, RSS movement, allocation counters, and WAL side effects", + "`benchmark/profile/run_tracelite_profile.dart` now validates the stable workload insight IDs after `tracelite explain`; a thin `workload_loaded`-only artifact exits 65 instead of silently looking complete", + "`workload_dispatch_bound` is useful but intentionally not a hard guard because it is threshold-based and can disappear when point-query floor-subtracted work lands slightly above the near-zero band on a noisy machine" + ], + "nextSignals": [ + "use the profile wrapper's generated `insights.md` before opening raw workload-summary JSON for dispatch-vs-work and memory/allocation shape", + "keep raw profile regions and workload JSON local under `build/`; commit compact aggregates unless graph data is feeding Pages", + "if future profile workload shape changes remove a required stable insight ID, update the guard and signal map in the same experiment rather than weakening the contract silently" + ] +} diff --git a/experiments/signals/entries/170.json b/experiments/signals/entries/170.json new file mode 100644 index 00000000..68f6e788 --- /dev/null +++ b/experiments/signals/entries/170.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "A synchronous `Mutex.tryLock()` plus a non-`async` `Writer.execute` / `executeBatch` is a coherent request-side scheduling change — exactly the request-side counterpart to exp 151's response-side `Completer.sync()` attempt — but it does not move the primary sequential-write lane: Single Inserts (100 sequential) shifted +1.7 % (paired medians 3.099 ms -> 3.153 ms, 3.082 -> 2.982, 3.169 -> 3.501) and writer_pipelining `sequential-awaited (2000 writes)` shifted +2.0 % (34.047 -> 34.723), both within the run-to-run noise band and in the wrong direction", + "Only Concurrent Single Inserts moved consistently (1.231 -> 1.135 ms, -7.8 %) — but that lane is owned by exp 159's send-gated writer-lock pipelining at -58 % to -61 %, so a second pass at the same workload is sub-decision-threshold gain at the cost of an extra slow-path branch plus the behavioral change of sync-before-close writes no longer being rejected", + "Closes the request-side scheduling tweak as a candidate for the exp 147 residual writer/request bucket; the next bounded implementation candidates are still the structural ones exp 159 named (cross-call request batching / group commit, or a shared-memory transport when the Dart SDK allows it)" + ], + "nextSignals": [ + "do not retry `Mutex.tryLock` + non-`async` Writer.execute without a Dart runtime change that makes `await` over a resolved Future provably more expensive than today", + "if a future profile shows sequential writes spending materially more wall in main-isolate spans than the writer-handle span, the sub-decision-threshold signal here might invert — re-check with a Tracelite A/B over both Single Inserts and Concurrent Single Inserts before reopening", + "the rewritten `close()` contention test (long-running transaction holding the lock, parked writer wakes and sees `_closed`) generalizes the previous microtask-hop-specific test and should be retained even after this experiment reverts" + ] +} diff --git a/experiments/signals/entries/171.json b/experiments/signals/entries/171.json new file mode 100644 index 00000000..4f0a0cce --- /dev/null +++ b/experiments/signals/entries/171.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_below_current_signal", + "changedBeliefs": [ + "The `await _runtime` microtask hop at the `Database` layer above exp 159's `Writer._request` cache is real (~1-2 us per call, theoretical upper bound ~6-12% on a 2000-call sequential burst) but sits at or below the focused-harness noise floor: two order-flipped passes on writer_pipelining.dart produced alternating-sign deltas inside per-round variance (sequential-awaited -2.3%/+2.5%, transaction-guardrail -7.5%/+6.1%, concurrent-burst +4-5% both passes)", + "Trimming microtask hops at the Database layer above the writer does not move exp 159's stated sequential-write residual floor (port wake + event-loop scheduling); the next reduction candidate in this area needs to be group-commit-shaped or transport-shaped, not hop-shaped" + ], + "nextSignals": [ + "do not retry the `Database` runtime cache against the same writer_pipelining harness without a measurement system that can resolve sub-1us per-call deltas on real workloads", + "keep exp 159's framing: the next stream-rerun-dispatch implementation candidate against the sequential-write floor should reduce round-trip count (group commit) or change transport, not chase per-call microtask hops", + "same shape as the recent overhead-removal rejection cluster (exp 145, exp 148, exp 151): theoretical hop savings or counter reductions may move but measured-elapsed does not once per-call cost drops below ~2 us" + ] +} diff --git a/experiments/signals/entries/172.json b/experiments/signals/entries/172.json new file mode 100644 index 00000000..5fb8e942 --- /dev/null +++ b/experiments/signals/entries/172.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "long-text-stream-hashing", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "The streaming suite now covers mixed long payloads beyond exp 110's 4KB TEXT shape: 8 unchanged streams over 64 rows with 32KB TEXT plus 32KB BLOB cells", + "The current 8-byte FNV fold scales acceptably on the broader row in a focused run: 3.446 ms median for mixed long payloads beside 2.356 ms for the existing 4KB TEXT row", + "Long-BLOB hash correctness is covered directly: same-length no-op BLOB updates suppress emission, while changes after the first 8-byte chunk emit" + ], + "nextSignals": [ + "do not reopen byte-stream hash-loop implementation work without checking both long TEXT and mixed TEXT/BLOB unchanged-fanout rows", + "future hash variants need a concrete mechanism or production profile showing this path remains dominant; broader measurement alone is no longer the blocker", + "the direct streaming-suite entry point can be used for focused stream measurements before compiling the full release runner" + ] +} diff --git a/experiments/signals/entries/173.json b/experiments/signals/entries/173.json new file mode 100644 index 00000000..214e6723 --- /dev/null +++ b/experiments/signals/entries/173.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "long-text-stream-hashing" + ], + "outcomeClass": "rejected_no_signal", + "changedBeliefs": [ + "At 32 KB cells on a pool-of-4 reader fleet the byte-stream fold is no longer the dominant wall component: even an unrolled 16-byte FNV body is +4.5 % to +12.1 % vs the exp 110 8-byte body across two order-flipped passes, all inside a 2.6 → 6.0 ms per-pass spread", + "The named ≥ 32 KB cell candidate is closed: the workload now exists (release-suite `Long-Text 32KB Unchanged Fanout` and focused `benchmark/experiments/long_text_32kb_hash.dart`) and shows that hash-loop unrolling beyond exp 110's 8-byte fold is below noise on this shape", + "The long-text direction is settled, not just `watch`: further FNV-shape variants need either a workload that isolates the hash loop from reader-pool parallelism, or a mixed BLOB/TEXT long-payload shape that may shift the hash-vs-retrieval ratio" + ], + "nextSignals": [ + "do not retry FNV unrolling variants on the existing parallel-fanout long-text shapes without a workload that isolates the hash loop", + "if a mixed BLOB + long-TEXT unchanged-fanout workload becomes interesting, exp 173's focused harness is a reusable template", + "a single-stream long-payload microbenchmark (one stream, no reader-pool parallelism) would expose hash-loop overhead that the current shape hides" + ] +} diff --git a/experiments/signals/entries/174.json b/experiments/signals/entries/174.json new file mode 100644 index 00000000..539dc37e --- /dev/null +++ b/experiments/signals/entries/174.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "in_review_win", + "changedBeliefs": [ + "The reader sacrifice path (Isolate.exit + reader respawn) is a win for the rows path (zero-copy transfer of already-built Dart objects) but counterproductive for selectBytes: native JSON must be Uint8List.fromList-copied before Isolate.exit can transfer it, so sacrifice saves zero copies on bytes and only adds a respawn; the non-sacrifice bytes path copied twice (fromList + SendPort)", + "Sending a Uint8List view over the reader connection's persistent json_buf is sound (SendPort snapshots the bytes at send time; the single-threaded handler keeps the buffer stable through the send) and is the single mandatory copy at every size, so selectBytes never needs to sacrifice", + "Large (>256KB) selectBytes is -44% (~1.8x) by eliminating the per-query reader respawn; small (<256KB) is -4% (two copies -> one); the cost is a bounded ~+15MB RSS high-water because readers are no longer respawned (json_buf reused, not reallocated)" + ], + "nextSignals": [ + "exp 175 promoted a large-byte (>256KB) selectBytes lane into the release suite and curated metric list; use `selectBytes() large bytes` as the guard for future bytes-transfer policy changes", + "if a memory-sensitive workload shows problematic json_buf retention, reopen with a high memory-reclaim sacrifice threshold for bytes (e.g. >8MB) or a C-side json_buf shrink, preserving the view-send win for the realistic large range", + "do not extend view-send to the rows path: there the payload is Dart objects and sacrifice's zero-copy object transfer is the real win (exp 008b/019)", + "do not pursue true zero-copy bytes transfer (fresh malloc per query + send address + NativeFinalizer): a size sweep 256KB->64MB (benchmark/experiments/transfer_mechanism_ab.dart) showed it 14-24% SLOWER at every size, no crossover. The decisive factor is buffer reuse, not the copy: the current path reuses a warm json_buf (no per-query allocation), while zero-copy must malloc a fresh buffer per query (mmap/munmap syscalls + cold-page faults that scale with size) and that exceeds the memcpy it saves. Reuse is incompatible with handing the buffer to main, and the buffer can't be safely recycled because callers can retain the returned Uint8List. The single copy is structurally the floor. Do not reopen without a fundamentally different mechanism (e.g. a recycled WARM native transfer-buffer pool with cross-isolate finalizer recycling — complex, GC-timing-dependent, unbounded under load)" + ] +} diff --git a/experiments/signals/entries/175.json b/experiments/signals/entries/175.json new file mode 100644 index 00000000..63a2a265 --- /dev/null +++ b/experiments/signals/entries/175.json @@ -0,0 +1,18 @@ +{ + "directions": [ + "result-transfer-shape", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "The release JSON-bytes suite now has an explicit large native-byte transfer row: `Large payload (~650KB) / resqlite selectBytes()` stays above the old 256KB sacrifice threshold and measured 0.323 ms wall / 0.000 ms main in the focused suite run", + "The curated history now tracks `selectBytes() large bytes`, so exp 174's native-view transfer path is visible without relying on the existing 1K-row JSON-bytes metric that mostly exercises small-result copy behavior", + "Keeping the new row resqlite-only avoids multiplying large JSON encode work across every peer while still guarding the runtime policy that only resqlite implements", + "The lane provably responds to the transfer path (not a dead guard): its 651KB selectBytes workload is the shape exp 174's focused A/B moved -44% to -47% (large_bytes_transfer.dart), and a parallel release-lane A/B independently showed +13% median / +55% p90 when the pre-174 fromList+sacrifice path was restored, while adjacent standard lanes stayed neutral. The 1K-row lane (<256KB) and 10K-row lane (SQLite-step dominated) both miss this band" + ], + "nextSignals": [ + "use `selectBytes() large bytes` before changing selectBytes transfer policy again", + "if json_buf high-water memory becomes a problem, compare any memory-reclaim threshold against this row so reclaim work does not reintroduce the reader-respawn cost exp 174 removed", + "do not apply bytes-transfer conclusions to rows select(); rows keep sacrifice because Isolate.exit is a real object-graph handoff there" + ] +} diff --git a/experiments/signals/entries/176.json b/experiments/signals/entries/176.json new file mode 100644 index 00000000..b0d9e4f8 --- /dev/null +++ b/experiments/signals/entries/176.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Exp 158 added the schema-name identity fast path to RowSchema.indexOf (serving Row.operator[]) but not to Row.containsKey, which still went straight to the private HashMap and hashed the key on every call. Exp 158's 'containsKey neutral' row compared Row vs LinkedHashMap with containsKey unchanged on both sides of that diff, so it was never evidence that routing containsKey through the identity path is neutral", + "Routing Row.containsKey through the identity scan (via a shared RowSchema.containsName(name) => indexOf(name) >= 0) improved the focused row_map_facade containsKey lane ~13.0 -> ~10.0 ms (-23%) across three order-stable passes, with the hot lookup control lane flat, flipping Row from ~+3.6 ms slower than LinkedHashMap to at-parity. The win is smaller than exp 158's -52% on operator[] because containsKey still pays MapMixin dispatch and the key-is-String check that operator[] does not; the identity scan removes only the hash", + "The win is interned-key-specific, like exp 158's operator[] path: in the real decode path schema names are freshly allocated via fastDecodeText, so a user-supplied literal is generally not identical to a decoded name and containsName falls through to the HashMap (same cost as before, never worse). The benchmark hits the identity path because both schema and probe key are string literals canonicalized to the same object" + ], + "nextSignals": [ + "containsName is coupled to indexOf by construction; it inherits whatever the identity scan does. Revisit only if exp 158's identity fast path is removed or its 32-column cap changes", + "do not infer a broader result-shape change from this; it is a consistency fix inside the same private schema index (exp 158 Future Notes still apply)", + "if a workload supplies containsKey with high-frequency equal-but-non-identical keys, the cost reverts to the HashMap probe (unchanged from before); no further work unless that workload is shown hot" + ] +} diff --git a/experiments/signals/entries/177.json b/experiments/signals/entries/177.json new file mode 100644 index 00000000..df780974 --- /dev/null +++ b/experiments/signals/entries/177.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "measurement-system" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The JOURNAL's most-reapplied lesson — phase-ordered A/B gates confound code deltas with time-correlated drift — was enforced only by runner discipline. Every recent A/B writeup (exp 159, 167, 171, 173) re-derived the same CV-asymmetry + order-flip reasoning by hand. Encoding it as cvPct + classifyDriftFlag in benchmark/shared/stats.dart plus a benchmark/ab_drift_check.dart CLI makes the call deterministic and citable", + "classifyDriftFlag reaches drift-suspected by two mechanisms: (1) CV asymmetry — the flagged phase's CV is >= 4x its clean phase's and above an 8% floor (the exp 159 signature, 0.20-0.46 vs 0.01-0.06), or (2) sign reversal across the order flip with both effects above a 3% floor (the exp 167 signature). Re-run against reconstructed exp 159 and exp 167 flags, the tool reproduces the verdicts those runners reached manually. reproduced requires both order-flipped passes to agree same-direction with comparable CVs", + "This is methodology tooling in the class of exp 161 / exp 169: it changes no runtime code and is not a measurement-that-unlocks-an-implementation, so the paired-run carry rule does not apply. A single pass can only reach drift-suspected (via CV asymmetry) or be deferred to the runner; the tool interprets the order-flipped pass, it does not replace running it" + ], + "nextSignals": [ + "wire decide_tracelite.dart / run_tracelite_experiment.dart to emit per-run sample arrays for flagged scenarios in the JSON shape ab_drift_check.dart reads, so the drift check is one command after a flagged pass rather than a hand-built fixture; blocked on the upstream tracelite decision JSON exposing per-run samples", + "if a future workload has a naturally higher noise floor, pass --cv-asymmetry-ratio / --clean-cv-pct / --effect-floor-pct rather than editing the defaults (which are derived from exp 144/159/167/171/173)" + ] +} diff --git a/experiments/signals/entries/178.json b/experiments/signals/entries/178.json new file mode 100644 index 00000000..43befc45 --- /dev/null +++ b/experiments/signals/entries/178.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "measurement-system" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The experiment->chart linker in generate_history.dart guarded the wrong-file case (_assertAcceptedExperimentsLinkToCandidates, the exp-109 mixup) but silently tolerated the more common missing-file case: a chartable experiment with no linked benchmark run AND no **Benchmark Run:** opt-out declaration drops off the chart with no CI error, which is exactly what a forgotten result file or a Date/timestamp mismatch produces", + "A structural tally on main confirmed the gap is live: of 23 accepted/in-review experiments whose linker produced a null run, only ~5 declared the **Benchmark Run:** opt-out header; the rest (several recent — 116, 118, 119, 125, 126, 136, 161, 172) are silently unmapped, indistinguishable from a forgotten file", + "Adding _assertNewExperimentsLinkOrDeclareRun (over a pure, unit-testable findUndeclaredMissingRunExperiments detector) makes the declaration mandatory for accepted/in-review experiments numbered >= a cutoff (178) with a null run; the repo's existing _skipsReleaseBenchmarkRunMapping header (none/n/a/tracelite) is the escape hatch. Pre-cutoff experiments are grandfathered, history.json is byte-for-byte unchanged, and no runtime code (lib/native/hook) is touched" + ], + "nextSignals": [ + "to pull the ~17 pre-178 silently-unmapped chartable experiments under the guard, backfill their **Benchmark Run:** headers (or commit their missing result files) and lower _benchmarkRunDeclarationCutoff in the same change — completed by exp 188 (cutoff walked to 1; 16 chartable experiments backfilled + exp 174 declaration tightened to start with `none — `; history.json byte-for-byte unchanged)", + "if the experiments page ever charts rejected experiments that ship a result file, widen the guard's status scope (currently accepted/in-review only) rather than touching the cutoff" + ] +} diff --git a/experiments/signals/entries/179.json b/experiments/signals/entries/179.json new file mode 100644 index 00000000..c315c220 --- /dev/null +++ b/experiments/signals/entries/179.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "rejected_no_signal", + "changedBeliefs": [ + "Extending exp 125/149's direct-code-unit batch write to the single-row allocateParams path (the exp 142 / PR #130 idea) makes the encoder itself materially faster, not slower: a DB-free micro (single_row_param_packing.dart, 200k cycles x 15 samples) measures -45% / -58% / -37% on 1-short / 5-mixed / 1-KB ASCII shapes with a flat blob+int control. Exp 142's +6.86% / +16.4% 'slower' was workload/Tracelite-overhead confound, not the encoder", + "The encoder win is immaterial on every representative workload: release-suite A/B (5x5) is neutral (Parameterized Queries +0.04%, Single Inserts -1.7%, Concurrent +0.8%; 1 timing win / 0 regressions / 166 neutral, the flags on untouched result-read paths). The single-row bind is too small a fraction of any lane to register", + "Reaffirms exp 142's conclusion with cleaner evidence: small single-row string binding stays on the generic allocateParams path. The operative reason is immateriality (not a regression) plus ~60 lines of duplicated packing logic for the fallback" + ], + "nextSignals": [ + "do not re-test single-row allocateParams direct encoding again without a representative large-single-row-ASCII-text-bind workload where the round-trip/result cost no longer hides the encoder; the encoder mechanism is now measured and settled (see single_row_param_packing.dart)", + "implementation preserved at archive/exp-179 for cherry-pick if such a workload appears" + ] +} diff --git a/experiments/signals/entries/180.json b/experiments/signals/entries/180.json new file mode 100644 index 00000000..9c79f995 --- /dev/null +++ b/experiments/signals/entries/180.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "accepted", + "changedBeliefs": [ + "Cross-call write request batching (the 'group commit' lever exp 159 named) captures the per-round-trip residual that pipelining left: standalone execute() calls piling up while a send is in flight coalesce into one MultiExecuteRequest, and the release Concurrent Single Inserts lane improves -26% (baseline-first) / -32% (candidate-first), reproduced across order-flipped runs. exp 147's ~72% writer/request residual was attackable by collapsing N messages to ~2, not just by overlapping N messages (exp 159)", + "Batching transport beats batching commits here: each coalesced statement runs as its own autocommit (per-call success/failure unchanged, no group BEGIN/COMMIT), because SQLite/commit is only ~9% of writer wall and a group transaction would change failure atomicity for independent writes", + "A backpressure trigger (send-when-idle, coalesce-only-while-a-send-is-in-flight) is required, not microtask coalescing: the microtask variant taxed every write ~+3% (sequential lane); backpressure leaves isolated/sequential writes at baseline cost (-7%/neutral both orderings) while still collapsing bursts", + "The single-pass Nested Transactions x50 +20% flag was drift, not a regression: it sign-flipped to -22% candidate-first, the timed region uses Transaction.execute (never the coalescing pump), and the lane's own per-repeat spread is ~45% (baseline 0.665-0.979 ms, candidate 0.789-0.990 ms)" + ], + "nextSignals": [ + "true fsync-merging group commit (wrap a coalesced group in one BEGIN/COMMIT) only buys the ~9% SQLite slice and changes failure atomicity; not worth it without a workload where commit/fsync dominates", + "the remaining transport floor is the shared-memory transport exp 159 named, still gated on Dart SDK support", + "behavior change shipped: a buffered write group racing db.close() is atomic (all flush or all reject), never the old lock-order-dependent partial outcome; still never hangs" + ] +} diff --git a/experiments/signals/entries/181.json b/experiments/signals/entries/181.json new file mode 100644 index 00000000..5af8886e --- /dev/null +++ b/experiments/signals/entries/181.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "long-text-stream-hashing" + ], + "outcomeClass": "rejected_no_signal", + "changedBeliefs": [ + "The single-stream open candidate from exp 173 has been consumed: benchmark/experiments/single_stream_long_payload_hash.dart uses an internal one-reader runtime so one unchanged stream hashes 64 rows x 64 KB TEXT + 64 KB BLOB (~8 MB) before a queued COUNT(*) barrier can emit", + "Removing reader-pool parallelism did not reveal a 16-byte FNV win. Across an order-flipped pair, baseline medians were 2.771 / 2.777 ms and candidate medians were 2.763 / 2.792 ms (-0.3% / +0.5%), with overlapping ranges", + "The exp 110 8-byte FNV body remains the correct implementation. The exp 173 16-byte unroll is now rejected under both pooled fanout and single-stream long-payload public stream shapes" + ], + "nextSignals": [ + "do not retry FNV loop unrolling on the existing long-text, long-payload, or single-stream public stream workloads", + "if long-payload unchanged hashing becomes hot again, use a direct resqlite_query_hash microbenchmark or production profile to split SQLite value access, hashing, reader dispatch, and reply delivery before implementing another hash-loop variant", + "the one-reader harness is a measurement tool only; it is not evidence for a public Database.open reader-count option" + ] +} diff --git a/experiments/signals/entries/182.json b/experiments/signals/entries/182.json new file mode 100644 index 00000000..690f73f1 --- /dev/null +++ b/experiments/signals/entries/182.json @@ -0,0 +1,18 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "rejected_no_stable_signal", + "changedBeliefs": [ + "Skipping the writer's preupdate_hook accumulation + reply harvest when _streamEngine.length == 0 at send time is a coherent residual-bucket attack — it removes per-row strcmp dedup + per-write FFI/toDartString/object work that no consumer reads, gated by a track_dirty flag on resqlite_db flipped from the writer isolate, with a DrainRequest no-op barrier from StreamEngine._createStream to fence in-flight non-tracking writes before a new stream's initial query", + "The no-stream win is real: two order-flipped focused passes show sequential-awaited (2000 writes) -3.8% / -5.3% and wide-batch (10k rows x 20 params) -2.4% / -5.9%, matched by the release-suite Single Inserts -5.2% / Concurrent Single Inserts -3.8% / Batch Insert 10k -6.8% deltas (sub-MDE flags on the public guard)", + "The with-streams cost is also real: the focused with-streams guardrail (1000 writes, 1 stream) regressed +2.7% / +3.7% across the same order-flipped passes — classified `reproduced` (not drift) by ab_drift_check.dart's direction-plus-CV-match rule — and Tracelite stream-rerun-dispatch came back inconclusive with the warmup elapsed guardrail regressing in the same direction across all three scenarios (+12.4% / +35.9% / +21.0%, gated too_noisy but matching the focused signal)", + "Reactive streams are the library's primary use case, so the workload mix in which the optimization helps (write-heavy without active streams) is narrower than the one it slows (any write while at least one stream is active). Net unfavorable on the realistic mix; the rejection class is rejected_no_stable_signal because the primary gate stayed inconclusive rather than reproducing a hard fail" + ], + "nextSignals": [ + "do not retry the same gate without a workload that shows write throughput without active streams is on a hot path — at that point the focused dep_tracking_skip.dart harness is the reusable lane", + "if a future revisit happens, consider amortizing the per-write _streamEngine.length check via a cached bool hasStreams on Writer (toggled on stream registration / cancellation) and/or restricting the gate to single-row standalone writes so the per-write overhead lands where the harvest savings are largest", + "the DrainRequest no-op barrier in archive/exp-182 is also useful as a standalone test-side fence (drain the writer FIFO before an assertion); factor it out separately if needed", + "implementation preserved at archive/exp-182 for cherry-pick" + ] +} diff --git a/experiments/signals/entries/183.json b/experiments/signals/entries/183.json new file mode 100644 index 00000000..8bff6052 --- /dev/null +++ b/experiments/signals/entries/183.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Exp 174's bounded RSS high-water (~+15 MB after 200 x 651 KB selectBytes) becomes pathological under a one-off concurrent-burst shape: 8 concurrent x 8 MB selectBytes pin 32 MB of native json_buf across the 4-reader pool for the rest of the connection's life, and the recurring-large pattern (1 large per 50 small over 300 iterations) settles at 16 MB pinned. The exp 174 future-note about high-threshold reclaim is a real candidate, not a hypothetical", + "Adding Diagnostics.readerJsonBufHighWaterBytes (sum of per-reader json_buf.cap) makes the high-water directly observable from any benchmark or downstream user — a reusable signal for future memory-sensitive selectBytes work alongside the existing exp 174 'large bytes' release lane", + "A high-threshold C-side realloc shrink, fired by the reader worker after SendPort.send returns on a selectBytes reply when json_buf.cap > 1 MB AND the just-sent result was < 256 KB, reclaims pathological retention without per-call cost on warm small buffers or back-to-back large reads. Post-burst settle: 32 MB -> 64 KB (32 MB freed); recurring: 16 MB -> 64 KB. Focused large_bytes_transfer.dart neutral within noise (large +0.7%, small +2%)", + "The 256 KB last-len guard is the load-bearing piece of the policy: it prevents the realloc churn anti-case the exp 174 future-note explicitly named (back-to-back large reads would shrink-then-regrow if shrinks were unconditional)" + ], + "nextSignals": [ + "promote a release lane that asserts json_buf_total stays bounded after a representative one-off concurrent-burst workload (exp 161 pattern) so this reclaim mechanism has public regression visibility", + "if a future workload shows the 1 MB trigger cap is too aggressive (hot 1-MB-class reads followed by a steady 500-KB-class read paying realloc churn), the natural tuning is to raise the trigger cap to 4 MB, not to drop the last_used_len guard", + "the new diagnostic field is the reusable signal for any future RSS-sensitive selectBytes work; the existing measurement-system openCandidate ('memory profiling harness with per-benchmark RSS acceptance criteria') can build on top of it rather than from scratch" + ] +} diff --git a/experiments/signals/entries/184.json b/experiments/signals/entries/184.json new file mode 100644 index 00000000..293926e5 --- /dev/null +++ b/experiments/signals/entries/184.json @@ -0,0 +1,15 @@ +{ + "directions": [ + "stream-rerun-dispatch" + ], + "outcomeClass": "accepted_measurement", + "changedBeliefs": [ + "Re-running exp 147's writer_sqlite_wall_audit on main after exp 159 + exp 180 shows the writer-burst breakdown essentially unchanged (A11c overlap SQLite 13.6% / invalidation 16.0% / residual 70.3% vs exp 147's 9.4 / 18.8 / 71.8). Expected: the audit issues writes sequentially (audit_workloads.dart:167) and exp 180 coalesces only concurrent bursts, so the sequential writer path it measures is unmoved", + "The sequential writer residual (55-70%) is the per-write isolate round-trip — the SDK-gated floor; the only lever left there is a shared-memory transport. Writer-residual micro-optimization is retired from the active candidate list", + "The cleanly-attackable buckets are smaller and spoken-for: invalidation/re-query precision (14-22%) is owned by in-flight exp 160 (incremental view maintenance, #155) and its traversal is already at floor (exp 121); the dirty-table harvest (~4-6%) is small and gating it regresses with-streams (exp 182); SQLite (14-38%) is sqlite3mc config, mined" + ], + "nextSignals": [ + "next reactive-engine win is exp 160; next sequential-write reduction needs the shared-memory transport when the Dart SDK allows it", + "do not re-open writer-residual micro-opts (harvest, scheduling hops) without a workload showing one of them is newly dominant" + ] +} diff --git a/experiments/signals/entries/185.json b/experiments/signals/entries/185.json new file mode 100644 index 00000000..bfbec88c --- /dev/null +++ b/experiments/signals/entries/185.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "result-transfer-shape", + "measurement-system" + ], + "outcomeClass": "in_review_measurement", + "changedBeliefs": [ + "Exp 183's json_buf reclaim now has release-suite visibility rather than only focused-audit evidence: SQLite Diagnostics emits a JSON buf (KiB) column and a dedicated `JSON buffer reclaim (8 large selectBytes + 64 small settles)` row.", + "The row is an active regression guard, not just a reported counter. After an 8-call concurrent large selectBytes burst and 64 small selectBytes settles, the diagnostics suite throws if readerJsonBufHighWaterBytes remains above 512 KiB; the focused suite run settled at 64.0 KiB with readersBusy=0.", + "The sqlite diagnostics parser and release-run sidecar/history JSON now carry optional `jsonBufKiB`, while legacy benchmark files without the column still parse with jsonBufKiB omitted." + ], + "nextSignals": [ + "Use the SQLite Diagnostics JSON buffer reclaim row before tuning exp 183 shrink thresholds, changing the last_used_len guard, or otherwise touching selectBytes memory-retention policy.", + "Keep latency policy changes on the existing exp 175 `selectBytes() large bytes` timing row; use exp 185 for bounded native-buffer retention.", + "If a future platform legitimately needs a higher retained buffer after small settles, raise the guard threshold in the same experiment that proves the new bound is safe." + ] +} diff --git a/experiments/signals/entries/186.json b/experiments/signals/entries/186.json new file mode 100644 index 00000000..f84c6aa2 --- /dev/null +++ b/experiments/signals/entries/186.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "accepted", + "changedBeliefs": [ + "The exp 179 single-row direct-ASCII allocateParams rewrite is materially beneficial once the bound text crosses the mid-tens-of-KB range — exactly the revisit condition exp 179 named. The new focused single_row_large_text_bind.dart workload (1 KB to 1 MB sequential INSERTs) measures -15.4% / -11.1% at 16 KB, -17.3% / -18.7% at 64 KB, -32.3% / -32.1% at 256 KB, and -26.7% / -28.5% at 1 MB across two order-flipped passes", + "Encoder isolation reproduces exp 179's deltas exactly (-45% / -58% / -37% on the single_row_param_packing.dart micro), so the mechanism question (encoder faster vs slower) is settled separately from the workload-materiality question", + "Small-payload single-row binds remain at the noise floor — Parameterized Queries, Single Inserts, and Concurrent Single Inserts all neutral on the release suite. Exp 179's small-bind finding stands: the encoder is the right default for the single-row path now because we have a representative workload where it matters, not because the small case stopped being flat", + "The one flagged release-suite regression (Batched Write Inside Transaction 100 rows, +44% on a 0.37 ms / 18.8% CV lane) is sub-decision-threshold noise on a code path the encoder cannot mechanistically reach (allocateBatchParams, not allocateParams); the 1000-row sibling moves the opposite direction (-10%) on the same change" + ], + "nextSignals": [ + "use benchmark/experiments/single_row_large_text_bind.dart as the durable workload for any future single-row bind change; the 1 MB shape is the load-bearing acceptance gate", + "do not extend the single-row encoder to UTF-8 long payloads (extending the Unicode fallback in the same shape exp 126 did for batches) without a representative non-ASCII large single-row workload; the encoder mechanism is settled but the materiality threshold is workload-specific", + "exp 142 / exp 179 small-bind rejections are sharpened (not overturned) by this run: do not retry direct encoding on small single-row text shapes without new evidence that the round-trip floor has moved" + ] +} diff --git a/experiments/signals/entries/187.json b/experiments/signals/entries/187.json new file mode 100644 index 00000000..5b14dce6 --- /dev/null +++ b/experiments/signals/entries/187.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Exp 186's UTF-8-heavy follow-up now has a representative workload: single_row_large_text_bind.dart emits byte-matched CJK rows beside the ASCII rows, so future single-row bind changes can test both direct-ASCII and direct-UTF-8 payloads in one focused harness.", + "Replacing the single-row non-ASCII _allocateParamsPreEncoded fallback with direct _utf8Length/_writeUtf8 packing removes one temporary Uint8List allocation plus one copy per non-ASCII string param. The CJK focused rows improve roughly 31-39% at 16 KB through 1 MB across the order-flipped pair.", + "The ASCII single-row path remains the exp 186 direct code-unit path; ASCII controls stayed comparable across the widened harness. Small unicode-1 encoder micro differences are tens of nanoseconds and not the acceptance gate." + ], + "nextSignals": [ + "use benchmark/experiments/single_row_large_text_bind.dart for both ASCII and CJK rows before accepting any future single-row bind encoder rewrite", + "the old single-row Unicode fallback is gone; future correctness changes to _utf8Length/_writeUtf8 must preserve multibyte, surrogate-pair, and embedded-NUL behavior on execute() as well as executeBatch()", + "if a release-suite large single-row text lane is added later, it should include a non-ASCII shape rather than only repeating exp 186's ASCII gate" + ] +} diff --git a/experiments/signals/entries/188.json b/experiments/signals/entries/188.json new file mode 100644 index 00000000..65d8c358 --- /dev/null +++ b/experiments/signals/entries/188.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "measurement-system" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Exp 178's pilot cutoff (`_benchmarkRunDeclarationCutoff = 178`) left ~17 pre-cutoff chartable experiments outside the missing-run-without-declaration guard's scope, still indistinguishable from forgotten result files. The follow-up exp 178 named (`backfill **Benchmark Run:** headers and lower the cutoff`) walks the cutoff back to 1, so every accepted/in-review experiment is now under the same forced-declaration discipline", + "All 16 silently-unmapped chartable experiments at the time of the backfill (003, 004, 007, 008, 009, 037, 038, 083, 116, 118, 119, 125, 126, 136, 161, 172) plus exp 174 (whose existing `**Benchmark Run:** Focused A/B (...)` header was descriptive but did not start with one of the opt-out keywords) gain a declared `**Benchmark Run:** none ...` line that explains why no release artifact exists. docs/experiments/history.json is byte-for-byte unchanged because the `_skipBenchmarkRunMapping` flag is consumed inside `_attachBenchmarkRunMappings` before serialization and none of these experiments were linked to a release run anyway", + "Test fixture `100-test.md` in benchmark_pipeline_test.dart predated the original cutoff at 178 too, and is updated to declare the opt-out so the test exercises a representative valid post-cutoff state. A new regression anchor (`exp 188 walked the cutoff to 1 — every chartable id is in scope`) lives next to exp 178's tests, asserting that `findUndeclaredMissingRunExperiments(..., cutoff: 1)` flags low-numbered chartable ids without declarations" + ], + "nextSignals": [ + "do not add a new chartable experiment without a `**Benchmark Run:**` declaration — the guard now fires for every id, not just 178+", + "if a pre-178 experiment later gains a real release artifact (e.g. someone re-runs exp 125 against current main to produce `2026-MM-DDTHH-MM-SS-exp125-*.md`), remove the opt-out header in the same change so the linker maps it correctly", + "the measurement-system openCandidate `memory profiling harness with per-benchmark RSS acceptance criteria` is independent of this run; exp 183/185 added the underlying RSS signal (readerJsonBufHighWaterBytes + the SQLite Diagnostics row), so a future RSS-criteria pass should build on those rather than on the guard scope" + ] +} diff --git a/experiments/signals/entries/189.json b/experiments/signals/entries/189.json new file mode 100644 index 00000000..67b676a1 --- /dev/null +++ b/experiments/signals/entries/189.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "transaction-control-paths" + ], + "outcomeClass": "rejected_below_signal", + "changedBeliefs": [ + "Savepoint naming compression (same-name cached SQL for `SAVEPOINT s`, `RELEASE s`, and `ROLLBACK TO s`) can move best-case focused control rows: empty fanout improved about 6-7%, rollback fanout improved 13-17%, and repeated deep chains improved 7-8% across the order-flipped pair.", + "The representative nested-write fanout did not reproduce: baseline-first measured -1.3%, then the candidate-first pass measured +21.6% slower. That is the load-bearing row because it matches the real nested transaction shape better than empty control-only fanout.", + "Per-savepoint string formatting, native UTF-8 allocation, caching, and naming are below the merge bar after exp 102, exp 111, and exp 189. Nested transaction headroom remains round-trip-shaped, not string-shaped." + ], + "nextSignals": [ + "Do not retry savepoint string caching, savepoint naming compression, or native helpers that still send one request per savepoint boundary without new evidence that string work is dominant.", + "Use benchmark/experiments/savepoint_name_compression.dart for quick savepoint-control probes, but treat the write-fanout row as the representative gate.", + "The remaining open implementation shape is multi-savepoint round-trip batching with preserved callback semantics and rollback legality." + ] +} diff --git a/experiments/signals/entries/190.json b/experiments/signals/entries/190.json new file mode 100644 index 00000000..fa4fd789 --- /dev/null +++ b/experiments/signals/entries/190.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "The C-side JSON encoder in `write_json_to_buf` (the only `selectBytes()` codepath) was re-running its per-column emission sequence (comma + `json_write_string` SWAR/escape scan + colon = four `buf_write*` calls) on every row even though the column name is invariant — the same per-row-amortizable pattern exp 034 caught at the Dart `RowSchema` layer and exp 037 caught at the json_buf layer was still firing at the byte-emission layer", + "Pre-building each column's `\"col\":` / `,\"col\":` token once at first-row time into a per-query scratch `resqlite_buf tokens_buf` and replacing the per-row inner loop with a single `buf_write(b, tokens_buf.data + token_offsets[i], token_lens[i])` removes that compounded work without behavior change (the pre-encode reuses `json_write_string`, so escape semantics for unusual column names are bit-identical)", + "Reproduced wins on the wide-many-row shapes the change targets: focused `select_bytes_wide_cols.dart` -4% to -11% across two order-flipped passes on 10k-row x 8 / 20-col shapes (largest shapes get the largest deltas, matching the compound hypothesis), and `large_bytes_transfer.dart` (exp 174's focused guard, ~650 KB per call) -8.7% / -8.2% on large/small lanes. Regression guards (1 row, 100 rows) stay inside the sub-microsecond noise floor — absolute deltas smaller than per-sample spread, with pass-to-pass signs not agreeing" + ], + "nextSignals": [ + "use `benchmark/experiments/select_bytes_wide_cols.dart` (and the existing exp 174 `large_bytes_transfer.dart` lane) before changing the selectBytes column-name emission path again", + "if a future selectBytes change wants to amortize more across rows (per-column value-prefix templates for constant-type integer columns, e.g.), the `tokens_buf` + `token_offsets` / `token_lens` arrays are the natural insertion point — extend them rather than adding a parallel scratch path", + "promoting the token scratch into `resqlite_cached_stmt` to skip the first-row pre-encode pass on cache hits is a strictly-larger change (cache lifetime, FFI surface) and is only worth a pass if a workload shows repeated small `selectBytes()` calls dominating wall time; exp 190's 1-row regression guard already shows the per-query overhead is at noise on the smallest shape", + "exp 175's `selectBytes() large bytes` release lane and exp 185's `JSON buffer reclaim` SQLite Diagnostics row remain the public guards for selectBytes transfer policy / RSS — exp 190 does not change either signal" + ] +} diff --git a/experiments/signals/entries/191.json b/experiments/signals/entries/191.json new file mode 100644 index 00000000..89c26bec --- /dev/null +++ b/experiments/signals/entries/191.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "parameter-encoding-and-binding" + ], + "outcomeClass": "in_review_correctness_guard", + "changedBeliefs": [ + "The broader embedded-NUL public API audit candidate is consumed as correctness support, not a performance result. Current main already covers wide executeBatch with multibyte embedded-NUL TEXT, and exp 187 covers single-row execute in flight; exp 191 adds the missing reader surfaces.", + "selectBytes preserves embedded-NUL TEXT seeded by SQLite char(0): select() returns the expected Dart string, hex(CAST(body AS BLOB)) matches _hexUtf8(body), and native JSON bytes decode back to the same value.", + "stream initial and changed emissions preserve embedded-NUL TEXT bytes, proving the one-pass initial decode, result hashing, and re-query emission path do not treat TEXT as NUL-terminated." + ], + "nextSignals": [ + "do not reopen broad embedded-NUL work without a new public API surface or a concrete regression report; coverage now spans wide executeBatch, in-flight single-row execute, selectBytes, and stream emissions", + "if _utf8Length, _writeUtf8, sqlite3_bind_text lengths, native JSON string emission, or stream row decoding changes, keep these tests in the focused validation set", + "future performance work in this direction still needs a workload where parameter encoding is material; exp 191 does not change the blob-heavy or small/narrow shape guidance" + ] +} diff --git a/experiments/signals/entries/192.json b/experiments/signals/entries/192.json new file mode 100644 index 00000000..97c8401a --- /dev/null +++ b/experiments/signals/entries/192.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Exp 023 left bounded headroom inside `write_json_to_buf`'s `SQLITE_INTEGER` arm — the single-digit `fast_i64_to_str` body runs one `% 10` / `/ 10` per output digit, and 10k × 20 INTEGER selectBytes performs 200k of these calls. Replacing the body with a two-digit `[00..99]` lookup table (one `% 100` / `/ 100` + one 2-byte memcpy per digit pair) cuts focused integer-heavy `selectBytes` by −8 to −26 % across two order-flipped passes on `select_bytes_int_heavy.dart`, with the biggest win on the deepest-digit shape (10k × 20 ~18-digit big ints, −24 to −26 %) — exactly the shape the algorithm predicts gains most from halving the division count", + "The release suite is not the right denominator for this change: no public selectBytes lane is integer-heavy enough for per-cell encoder savings to register. The same `Select 10k rows → JSON Bytes` lane reads 3.545 / 5.213 / 3.598 / 3.739 ms across baseline-first / candidate-second / candidate-first / baseline-second — *which side ran first* explains more variance than *which side is candidate*. Remaining release flags (Long-Text / Long-Payload Unchanged Fanout, Nested Transactions, column-granularity re-emit counters) reverse sign across the order flip and live on hash/savepoint/dependency paths the change cannot mechanically touch, so they read as exp 159 / exp 177 phase-correlated drift, not real regressions", + "Correctness for the int64 boundaries (LLONG_MIN, LLONG_MAX, zero, ±9 / ±99 / ±999 / ±10000 / ±1234567890) is preserved bit-for-bit by reusing exp 023's `(unsigned long long)(-(val + 1)) + 1` LLONG_MIN trick and the `val == 0` short circuit, and is covered end-to-end by a new `selectBytes encodes int64 extremes` test against `db.selectBytes()`" + ], + "nextSignals": [ + "use `select_bytes_int_heavy.dart` as the durable gate for future selectBytes int-encode work — release suite is not the right denominator at typical int-cell counts", + "do not retry deeper itoa variants (4- or 8-digit tables, branchless log10 length prediction) without a workload that goes further than the 10k × 20 big-ints shape, or a production profile showing INTEGER cells dominate `selectBytes` wall on a representative payload — exp 023 → exp 192 has now consumed the bounded itoa headroom for typical magnitudes", + "the remaining bounded slice of `write_json_to_buf` is the `SQLITE_FLOAT` arm's `snprintf(\"%.17g\")`; exp 041 already rejected a vendored Grisu/Ryu replacement on size grounds, so any future float-encode candidate needs either a much smaller fast path or production evidence that FLOAT cells dominate selectBytes wall" + ] +} diff --git a/experiments/signals/entries/193.json b/experiments/signals/entries/193.json new file mode 100644 index 00000000..30cca273 --- /dev/null +++ b/experiments/signals/entries/193.json @@ -0,0 +1,16 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "rejected_regression", + "changedBeliefs": [ + "A JIT row_map_facade sample made Row.values look like the remaining Row facade gap (baseline values iteration 8.607 ms vs LinkedHashMap 4.779 ms), but repeated JIT samples were contradictory after VM warmup: the baseline itself alternated between ~3 ms, ~9 ms, and ~11 ms on the values lane.", + "Replacing Row.values with a fixed-length ListBase slice view produced stable-looking JIT samples around 3.4-3.7 ms, but the AOT check reversed the decision. Compiled baseline _RowValueIterator values iteration stayed at 2.663 / 2.675 / 2.687 ms, while the ListBase candidate regressed to 6.828 / 6.847 / 7.101 ms.", + "The controls stayed neutral under AOT (hot lookup, containsKey, entries iteration, Map.from clone), so the regression is specific to Row.values. The original custom iterator is the better compiled shape." + ], + "nextSignals": [ + "do not replace Row.values with ListBase, getRange, or fixed-slice views based on JIT row_map_facade output alone", + "future Row.values work must compile the focused harness to AOT before acceptance; the JIT values lane is not a reliable decision gate", + "no runtime code kept; keep _RowValueIterator unless a future Dart runtime changes compiled iterator behavior" + ] +} diff --git a/experiments/signals/entries/195.json b/experiments/signals/entries/195.json new file mode 100644 index 00000000..680f528f --- /dev/null +++ b/experiments/signals/entries/195.json @@ -0,0 +1,17 @@ +{ + "directions": [ + "result-transfer-shape" + ], + "outcomeClass": "in_review_accepted", + "changedBeliefs": [ + "Exp 190's per-query `tokens_buf` + first-row pre-encode walk is not noise on small-rowset selectBytes — it is invisible to the `wide_cols.dart` 1-row / 100-row regression guards because their reporting is millisecond-precision, not because the underlying ~0.5-1 µs per query is below the wall floor. Caching the encoded tokens on `resqlite_cached_stmt` (built lazily, freed in `stmt_cache_entry_dispose`) eliminates the per-query `buf_init(64)` + `free` pair and the first-row pre-encode walk while keeping per-row work and JSON output bit-identical to exp 190.", + "A new microsecond-precision focused harness `select_bytes_repeated_calls.dart` (1000 calls/sample × 11 samples after 16-call warmup) exposes the predicted shape: 1-row × 20-col improves −9.2 % / −7.2 % and 10-row × 20-col improves −5.1 % / −2.7 % across two order-flipped passes. Per exp 177's drift classifier this reproduces same-direction across the flip. 100/1000-row guards show sign reversal (+1.8 % / −2.7 % and +1.0 % / −1.7 %) consistent with drift-suspected, because per-query setup is < 0.2 % of wall at those sizes.", + "Exp 190's `wide_cols.dart` 10k-row shapes also trend candidate-faster on every lane across both order-flipped passes (1-7 % movement, magnitude varies but sign reproduces), consistent with eliminating the per-query malloc/free pair from the hot path of the C-side allocator.", + "Per-statement memory cost is `O(col_count * 8 + name_byte_count)` bytes — capped by `STMT_CACHE_MAX = 32` per reader/writer connection. The cache entry's existing lifecycle (eviction, connection close) handles cleanup with no new ownership rules." + ], + "nextSignals": [ + "use `select_bytes_repeated_calls.dart` (not `wide_cols.dart`) as the durable gate for any future selectBytes amortization at small repeated rowsets — `wide_cols.dart` reports in milliseconds and cannot see µs-scale per-query work", + "further attach-onto-cache-entry encoder amortization (per-column type hints, constant value prefixes, schema-stable type dispatch) is only justified by a workload that shows specific structural per-query work dominating after exp 195", + "do not retry promoting the C-side allocator buffer reuse beyond what stmt-cache caching already provides; the json_buf is already persistent per reader (exp 037), and exp 195 amortized the only other per-query allocator pair on the selectBytes hot path" + ] +}