experiment pipeline: generate README tables from per-experiment fragments - #203
Merged
Conversation
…ents
Completes the conflict-reduction work (the deferred README half of phase 2).
README.md was the last shared file experiments hand-edited (and a *source*
for generate_history, which is why it was deferred). Now:
- experiments/index/NNN.json — one fragment per experiment holding its README
row {file, title, impact, status, link}. A "split" experiment (014: an
accepted finding + a rejected one) stores an array of rows.
- experiments/README.template.md — prose + table headers with {{ROWS:status}}
placeholders.
- generate_readme.dart assembles README.md from template + fragments, newest
first (id descending). README.md becomes a generated, bot-owned aggregate.
- generate_history.dart now sources experiments from the fragments (it builds
the equivalent table markdown in memory and feeds the UNCHANGED, golden-
tested parser), with a README.md fallback for test fixtures. Its sort is
made stable (insertion-order tiebreak) so split/suffixed ids (008/008b, the
two 014 rows) are deterministic regardless of source.
- check_generated_data builds README too; check_experiment_signals and
finalize_experiment read the index fragments instead of README rows.
- The bot regenerates README.md; instructions updated to "add index/NNN.json,
never edit the README table".
Verified: README round-trips content-exactly (same row set per section, prose
byte-identical) — only the row ORDER normalizes to id-descending (the prior
order was an irregular historical artifact). history.json is byte-identical
except the two duplicate-014 entries normalize to accepted-first via the new
stable sort. Golden-fixture + pipeline tests pass; analyze clean.
Two follow-ups, mirroring the signals.json rollout:
- add experiments/README.md to guard-generated-docs once this baseline lands
(left out so this bootstrap PR can commit the reordered README).
- README-table generation does not touch the chart pipeline beyond the stable
sort.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
danReynolds
added a commit
that referenced
this pull request
Jun 23, 2026
README.md is now a generated, bot-owned aggregate (assembled from experiments/index/NNN.json fragments), so block branches from committing it too — closing the bootstrap gap left when the generated README baseline landed in #203. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
This completes the conflict-reduction work — the deferred README half of phase 2 (#202 did history/devices/signals).
README.mdwas the last shared file experiments hand-edited, and the reason it was deferred:generate_history.dartparses README rows as a source, so generating it requires breaking that dependency carefully.How
experiments/index/NNN.json— one fragment per experiment with its README row ({file, title, impact, status, link}). Two experiments never touch the same file. A split experiment (014 has both an accepted and a rejected finding) stores an array of rows.experiments/README.template.md— the prose + section/table headers with{{ROWS:status}}placeholders.generate_readme.dartassemblesREADME.mdfrom template + fragments, newest-first. README.md becomes a generated, bot-owned aggregate.generate_history.dartnow sources experiments from the fragments — it builds the equivalent table markdown in memory and feeds the unchanged, golden-tested parser (with a README.md fallback for fixtures). Its sort is made stable (insertion-order tiebreak) so ids sharing a number (008/008b, the two 014 rows) are deterministic regardless of source.check_generated_databuilds README too;check_experiment_signalsandfinalize_experimentread the fragments instead of README rows; the bot regenerates README.md.Correctness (round-trip proofs)
014entries normalize to accepted-first (the old order was an unstable-sort artifact; consumers index by id, so it's cosmetic).dart analyze --fatal-infosclean on all changed files; this PR touches nodocs/*.jsonorsignals.json(passesguard-generated-docs).Rollout
Mirrors the signals.json bootstrap: this PR commits the reordered README.md as the new generated baseline, so
experiments/README.mdis not added toguard-generated-docshere — a one-line follow-up adds it once the baseline is onmain.Test plan
check_generated_data— devices/history/signals/README all build cleanlycheck_experiment_signals— valid from fragmentshistory.jsonround-trip: identical modulo the 014 stable-sort normalizationdart test benchmark_pipeline_test benchmark_generated_outputs_test— pass (golden fixtures)dart analyze --fatal-infos— clean