docs: reconcile CONTEXT.md to current science state - #55
Closed
jakobtfaber wants to merge 157 commits into
Closed
Conversation
Restore the four frozen census CSVs (byte-verified against the pinned sha256s in reproducibility-foreground-galaxies.md) into galaxies/foreground/data/frozen_census/, and the adjudication scripts removed by 9096a60 into galaxies/foreground/adjudication/ (byte-identical to the flits-rerun worktree copies; sha-pinned in the test since these frozen-era scripts execute at import and cannot be import-tested without running live TAP queries). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author analysis/fit_generations.yaml: two trust-revoked generations (mixed-legacy-2026-06, beta-campaign-2026-07) with artifact globs, the external runs-store note, manuscript consumers map, and the chromatica three-way gate conflict recorded for P5.3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both table emitters already exist on main (budget_table_emitter.py, foreground_table_emitter.py with its colocated tests); this closes the remaining P0.4 gap: a tests/-side parity assertion that format_budget_table_tex() matches the committed exports/budget_table.tex. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Land DM-locked joint-fit campaign and Oran qualification
P0 provenance freeze: frozen census + adjudication package, fit-generation inventory, budget parity test
* test(scint): qualify polarization cross-ACF recovery * docs(scintillation): record recovered notebook replay * test(scintillation): falsify recovered notebook peak * test(scintillation): bound Freya CHIME sensitivity * feat(scint): dynesty 1-vs-2 component ACF evidence engine with rail flags (A1 trigger, limb i) Nested-sampling dlnZ comparison: single Lorentzian vs the reference physical two-screen form lor1+lor2+lor1*lor2+c (width-ordered via gamma2=f*gamma1, f log-uniform [3,300]). HWHM convention, one-sided lag-0-excluded ACF, MVN likelihood over a supplied covariance. Edge-mass rail flags use the ADR-0008 criterion; a railed second component is model-family rejection. rwalk sampling + maxcall cap: the default uniform-ellipsoid proposal stalls on the large-gamma1 x large-f likelihood plateau for some noise seeds. Plan: Faber2026 docs/rse/specs/plan-a1-trigger-calibration.md Phase 1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scint): MC correlated-lag ACF covariance, wired to the evidence engine (A1 Phase 2) Null-conditioned single-screen realizations (exponential-PBF field synthesis, HWHM = 1/(2*pi*tau_s), C1=1) through the production calculate_acf path; Ledoit-Wolf shrinkage toward the diagonal + jitter for Cholesky stability. evidence_with_mc_covariance: production ACF -> gamma-hat (interpolated half-max, production convention) -> matched covariance -> dlnZ, so the second component cannot feed on finite-scintle sample variance (verified in the ~6-scintle regime test). Plan: Faber2026 docs/rse/specs/plan-a1-trigger-calibration.md Phase 2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sim): A1 trigger injections + calibration-campaign kernel and driver (Phases 3-4) Single/two-screen truth-known injections rendered through the production calculate_acfs_for_subbands path (sim_gate pattern), HWHM convention stamped. Campaign kernel: per-burst summed dlnZ over valid subbands with matched MC covariances; null cells, power cells, conservative max-over-cells threshold envelope with an explicit evidence_failed accounting (a >50%-failure cell refuses to set a threshold). Driver: checkpointed cells, --workers process parallelism, --smoke end-to-end path (verified: parallel == sequential by seed). Tests live under tests/ (repo convention; the plan's simulation/tests path does not exist). Plan: Faber2026 docs/rse/specs/plan-a1-trigger-calibration.md Phases 3-4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sim): scale calibration band with subband count (120 channels each) First h17 launch failed the whole num_subbands=8 arm instantly: a fixed 6 MHz band gives 15 channels per subband, below calculate_acf's 20-channel minimum, so every injection raised and the cell checkpointed 200/200 evidence failures. Band width is now num_subbands x 120 channels, keeping the per-subband channel budget constant across the grid. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(scint): escalation_trigger_verdict — dlnZ + PPC limbs, rail = model-family rejection (A1 Phase 5 wiring) Verdict semantics per the A1 design: railed second component rejects the model family regardless of dlnZ; PPC band [0.05, 0.95] on the registered lag1_acf statistic is the independent limb; no-escalation is censored, never a single-screen claim. Calibration dict must carry the injection-calibrated dlnz_threshold. 8/8 tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sim): calibration-report figures — null contact sheet, operating curve, power curves Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(scint): consolidate CHIME qualification artifacts * chore(entire): record CHIME artifact consolidation --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Squash-merge the reviewed diagnostic-only Freya two-band figure prototype. Full local and CI validation passed; generated figures remain untracked and manuscript replacement remains Phase-0 gated.
Replace the per-pair O(n_pairs) delegation with the exact identity
sum_{a<b admissible} (v_a v_b)_sym = 0.5 (T*T - sum_t W_t*W_t) over
block-demeaned, normalization-scaled folds; auto terms cancel between the
subtractions when same-time pairs are excluded. Freya-sized call (2 pols x
15 times, 23064 channels, 40 lags) drops from ~30 s to 0.026 s (~1100x),
making the 24-cell x 128-trial blinded calibration matrix feasible.
Behavior pinned by test_all_pairs_cross_acf_matches_pinned_reference
against a fixture generated from the per-pair implementation (rtol 1e-10;
fixture SHA256 in fixtures/PROVENANCE.md). Block statistics extracted to
_lag_block_stats and shared with blockwise_cross_acf_pairs unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…/aggregate/unblind) validate_freya_c1.py implements the c1-allpairs-crossgp route from the owner decision: all-pairs distinct-time cross products on the retained pol-resolved product, LOO-rotated off-pulse nuisance templates, a 24-cell x 128-trial real-background injection matrix (B4 recovery gates carried forward unchanged), held-out + pairing-scramble nulls at the 1% family-wise threshold, and a structurally enforced blinding boundary: only the unblind subcommand can fit the on-pulse window and it refuses without a hash-bound GO verdict. frozen_config.json predeclares grid, gates, scrambles, and input SHA256s (prerequisites all pass). Includes the prior session's diagnostic + handoff and the bounded-parallel matrix driver run_c1_calibration.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Complete 24-cell x 128-trial blinded matrix on the retained freya pol-resolved product. All m<=0.30 cells fail width recovery (gated m=0.15/0.17 cells by 4.6-8.3x the limit; low-m fits collapse onto the fit-window bounds); all m=1.00 cells pass. Null campaign fails the family-wise gate (max |z| 4.810 > 4.408, 0 fit detections). No unblinded on-pulse fit was produced. Adds RESULT.md, figures + manifest + review, inventory entry, and the plotting script. Per the decision-doc stop rule, the successor must change input-product information content (p1-window-upchan). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…caveat hygiene) hamilton (stays measurement_eligibility: candidate) and johndoeII carry provenance_caveat: single_block in the product registry. The caveat is an annotation, not a gate: finalize_measurement_status passes it through in both the non-CHIME early return and the final status (including the inconclusive fail-closed branch), and the campaign runner injects it from the registry into patched per-burst configs so campaign outputs carry it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d-20260722 Add controlled seeded joint-fit runner
…able-20260722 Preserve controlled virtualenv executable identity
…-window-20260722 Fix joint resolution selection against final windows
…fiers-parent-20260723 Reconcile canonical burst identifiers in budget source
…erdict-20260723 Align foreground table with census verdict
…t-support-20260722
…rt-20260722 Preserve exact model-grid noise support
* data: point filterbank provenance at h17 Replace the twelve dead filterbank paths with their h17 locations under /data/Faber2026/data/dsa-110/<burst>/, taken verbatim from the h17 migration ledger, and record h17 as the data-access authority in the fixture header. * docs: name h17 the raw-data authority in DATA_LOCATIONS The header declared Google Drive the data authority and demoted h17 to compute, which contradicts the raw-data authority. Splits the two: h17 owns raw input, Google Drive remains the bulk archive for derived products. * data: mark the codetections manifest superseded as an authority Its canonical_target names a drained staging tree. Banner only -- counts, byte totals and sentinel checksums are left untouched as a dated measurement record. * docs: carry the results-library path repair onto main Folds in the sole unique commit of the parent's pipeline pin (1d5633c), which repoints the results-library catalog and materializer at the sibling analysis repo. With this the pin holds nothing main will not, so the pin can move.
…233) The ~/Developer/overleaf/Faber2026 checkout is retired (Overleaf pulls from GitHub in the browser). sync_figures and the budget-figure script now target the canonical parent repo; the freya local-run configs point at the canonical ~/Data/Faber2026 data home and this repo's own scattering configs. All referenced paths verified to exist. Claude-Session: https://claude.ai/code/session_01AcBQdvMPgyQc1WNeVKcXBM Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The frozen census bursts.csv still carried FRB 20230325A / 20230913A / 20240203A, superseded by the load_tns_name authority and the pinned registry (#225): 20230325C, 20230913G, 20240203D. The Figure 3 input builder reads this roster's tns column directly, so the rendered figure printed the stale names. Regenerates sightline_halo_grid.csv and the expanded catalog (registry 96bfd323; only the seven transient-name cells change), and advances the roster hash pin in the reproducibility doc with a supersession note. Claude-Session: https://claude.ai/code/session_01AcBQdvMPgyQc1WNeVKcXBM Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Owner direction 2026-07-26: the rendered panels effectively show only budget-contributing systems (non-budget neighboring clusters all lie at impact >1000 kpc, outside the +/-600 kpc window), so the open-diamond overlay and its legend entry are redundant. Removes _contributor_mask and the diamond draws; the cluster-crossing ring is unchanged.
… config (salvage of c6cd1c3) (#236) Cherry-picked from codex/b4-figure-review-20260720 before local-branch consolidation. The cross-reference CSV conflict resolved to main's richer landed version; only the config registrations for the catalogs that produced it were missing from main. (cherry picked from commit c6cd1c3) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…+ provenance) (#237) * archive(h17): frozen snapshot of joint-tf-fits dirty worktree, 2026-07-27 (read-only capture; base d292f4b) * archive(h17): frozen snapshot of model-grid-diagnostic dirty worktree, 2026-07-27 (read-only capture; base 31f7744) * archive(h17): frozen snapshot of resolution-diagnostic dirty worktree, 2026-07-27 (read-only capture; base 0864939) * drop editor backup file from the snapshot integration
…persession, extended census, finalized scint campaign + first-pass demotion). Verified 2026-07-27 against docs/adr/0006+0008, intervening_census_registry.csv (52 rows: 29/16/7), chime-scintillation README, and the parent manuscript. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
Opened against the wrong base — canonical development is jakobtfaber/dsa110-FLITS; reopening there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Owner-requested science-state refresh (2026-07-27 audit follow-up; owner approved PR+merge). Dated supersession notes, vocabulary kept for reading historical artifacts:
tab:alpha/tab:betain the manuscript, no scattering value quoted pending refit.scratch/codetection/framing marked historical; manuscript states the extended census (38 rows → 31 systems, 11 confirmed = 9 halos + 2 clusters); live registry 52 rows (29 confirmed / 16 inconclusive / 7 refuted).chromatica_hi+ 23 diagnostic-only) and the 2026-07-26 owner demotion of the whole campaign to first-pass evidence (scint-redo-01).Every claim verified against docs/adr/0006+0008, the census registry, the chime-scintillation README, and the parent manuscript this session.
🤖 Generated with Claude Code