experiment pipeline: make derived + accumulated files bot-owned (phases 1+2)

[danReynolds]

Why

Experiment PRs kept hitting routine merge conflicts. The root cause: experiment branches hand-maintain shared aggregate files that the post-merge Update-Docs bot also rewrites on main. So both sides of every merge touch the same files, and every merge (plus every bot commit) re-stales every other open PR. Across one recent session that was ~90% mechanical toil (regenerate → commit → re-conflict).

The fix is one principle: an experiment branch should only write files that belong to that one experiment. Everything shared or derived is generated and owned post-merge, never hand-committed on a branch.

Phase 1 — generated docs become bot-owned

docs/experiments/history.json and docs/benchmarks/devices.json must exist on main (the Pages site fetches them at runtime), but they do not need to exist on branches. A merge only conflicts when both sides change a file — so if branches stop touching them, every bot commit becomes a one-sided auto-merge and the cascade dies.

• check_generated_data.dart now verifies the sources build cleanly (it runs the generators, preserving every validity assert — missing-run declarations, experiment↔run mapping, signal validity) instead of diffing the committed docs/*.json.
• finalize_experiment.dart no longer writes history.json; it only validates.
• New CI job guard-generated-docs fails any PR that modifies the two files (mirrors the existing raw-profile-JSON guard; PR-only, so the bot's main commits are unaffected).
• The bot stays the sole owner on main.

Phase 2 (signal entries) — split the accumulated map into per-experiment files

The 67-entry experiments{} map in signals.json was the other frequent conflict (every concurrent "add my entry" collided on the same closing brace). Now:

• experiments/signals/base.json — schema meta + the per-direction synthesis (directions[]).
• experiments/signals/entries/NNN.json — one file per experiment, so two experiments never touch the same one.
• New generate_signals.dart assembles signals.json from those (entries sorted numerically). signals.json becomes a generated, bot-owned aggregate.
• check_experiment_signals.dart validates the assembled-from-source map.

Round-trip proof: regenerating signals.json from the fragments yields the same 67 entries with byte-identical content and an identical base; the only change is entry order normalizing to ascending. (Validated in-tree.)

Deferred (with reason): generating the README tables

I'd originally scoped Phase 2 to also generate the README experiment tables. While implementing, I found generate_history.dart parses README.md rows as a source, so generating the README requires reworking the history parser first — a bigger, riskier change for a lower-frequency conflict (README rows mostly auto-merge). Tracked as a focused follow-up rather than risk the core history pipeline here.

Rollout note

Merging this conflicts with in-flight experiment PRs (#155, #200) on signals.json, and they'd trip the new guard. Cleanest order: land those experiment PRs (old model) first, then this; or migrate their signals.json edits into entry fragments. A one-line follow-up should add experiments/signals.json to guard-generated-docs once the generated baseline is on main.

Test plan

• dart run benchmark/check_generated_data.dart — all three sources build cleanly
• dart run benchmark/check_experiment_signals.dart — valid from fragments
• generate_signals.dart idempotent; round-trip semantically identical to prior signals.json
• dart analyze --fatal-infos on all changed/new Dart files — clean
• finalize_experiment.dart --dry-run — runs the two checks, writes nothing
• PR touches no docs/*.json (passes its own guard-generated-docs)