From 898fe12cf2d36bab5ac5562ac150e9e3cf35b0aa Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Thu, 21 May 2026 15:16:12 +0100 Subject: [PATCH] likelihood: split into likelihood_breakdown + likelihood_runtime packages MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Separates the two distinct goals that previously coexisted in every `likelihood//.py` script: - `likelihood_breakdown/` — per-step JIT decomposition, single config (CPU fp64 default; `--gpu` opt-in). Answers *"where does time go inside this likelihood?"* Pedagogical / optimisation-focused. - `likelihood_runtime/` — full-pipeline JIT only, driven by the existing CPU/GPU/A100 × fp64/mp sweep harness. Answers *"how long will this take on this hardware?"* Production-cost-focused. Side-effect: the cheaper per-script runtime mode runs in seconds-to-low-minutes; the heavier per-step mode only runs on demand. # Moves (git mv — history preserved) - `likelihood/_profile_cli.py` → `_profile_cli.py` (repo root) - `likelihood/adapt_image_util.py` → `_adapt_image_util.py` (repo root) - `likelihood/OPTIMIZATION_NOTES.md` → `likelihood_runtime/OPTIMIZATION_NOTES.md` - `scripts/sweep_likelihood.py` → `likelihood_runtime/sweep.py` - `scripts/aggregate_sweep.py` → `likelihood_runtime/aggregate.py` - 9 per-cell scripts: `likelihood//.py` → `likelihood_runtime//.py` # Per-cell split 5 cells get both variants (script previously did both jobs): - `imaging/{mge,pixelization,delaunay}.py` - `interferometer/delaunay.py` - `datacube/delaunay.py` For each, `likelihood_runtime//.py` has the per-step `PART B` JIT section + per-step JSON keys + per-step bar chart deleted; `likelihood_breakdown//.py` is the per-step half written fresh, sharing the setup/Part-A code with runtime by design (per CLAUDE.md "three similar lines is better than a premature abstraction"). 4 cells stay runtime-only (no breakdown variant) — their existing scripts were full-pipeline-by-design: - `interferometer/mge.py` (intentional, per docstring) - `interferometer/pixelization.py` (sparse-DFT path is already a single block) - `point_source/{image_plane,source_plane}.py` (single short JIT shots) # Import-path simplification The split lets the two-line sys.path-insert dance in every cell collapse to a single insert at `parents[2]` (autolens_profiling root), where both `_profile_cli` and `_adapt_image_util` now live. # Deep-research deliverable Per-package README.md docs the methodology, output schema, and "when-to-use" guidance for each package. The empirical findings (per-cell timings, mp verdicts, GPU NUFFT regression, upstream blockers) stay in `likelihood_runtime/OPTIMIZATION_NOTES.md`. # Removed - The old `likelihood/` tree (all content moved or replicated). - 5 per-class README.md files under `likelihood/{imaging,interferometer, datacube,point_source,}` — content folded into the two package READMEs. # Follow-ups - `scripts/build_readme.py` needs its glob paths updated for the new layout (it still scans `likelihood/results/*.json`). Out of scope for this PR; flagged in the top-level README. Co-Authored-By: Claude Opus 4.7 --- README.md | 23 +- ...dapt_image_util.py => _adapt_image_util.py | 0 likelihood/_profile_cli.py => _profile_cli.py | 0 likelihood/README.md | 63 -- likelihood/datacube/README.md | 57 -- likelihood/imaging/README.md | 59 -- likelihood/interferometer/README.md | 47 -- likelihood/point_source/README.md | 57 -- likelihood_breakdown/README.md | 113 +++ .../datacube/__init__.py | 0 .../datacube/delaunay.py | 191 +----- .../imaging/delaunay.py | 221 +----- .../imaging/mge.py | 237 ++----- .../imaging/pixelization.py | 238 +------ .../interferometer/delaunay.py | 340 +-------- .../OPTIMIZATION_NOTES.md | 0 likelihood_runtime/README.md | 148 ++++ .../aggregate.py | 18 +- likelihood_runtime/datacube/__init__.py | 0 likelihood_runtime/datacube/delaunay.py | 601 ++++++++++++++++ likelihood_runtime/imaging/delaunay.py | 578 ++++++++++++++++ likelihood_runtime/imaging/mge.py | 507 ++++++++++++++ likelihood_runtime/imaging/pixelization.py | 574 ++++++++++++++++ likelihood_runtime/interferometer/delaunay.py | 647 ++++++++++++++++++ .../interferometer/mge.py | 1 - .../interferometer/pixelization.py | 6 +- .../point_source/image_plane.py | 1 - .../point_source/source_plane.py | 1 - .../sweep.py | 16 +- 29 files changed, 3362 insertions(+), 1382 deletions(-) rename likelihood/adapt_image_util.py => _adapt_image_util.py (100%) rename likelihood/_profile_cli.py => _profile_cli.py (100%) delete mode 100644 likelihood/README.md delete mode 100644 likelihood/datacube/README.md delete mode 100644 likelihood/imaging/README.md delete mode 100644 likelihood/interferometer/README.md delete mode 100644 likelihood/point_source/README.md create mode 100644 likelihood_breakdown/README.md rename {likelihood => likelihood_breakdown}/datacube/__init__.py (100%) rename {likelihood => likelihood_breakdown}/datacube/delaunay.py (81%) rename {likelihood => likelihood_breakdown}/imaging/delaunay.py (81%) rename {likelihood => likelihood_breakdown}/imaging/mge.py (75%) rename {likelihood => likelihood_breakdown}/imaging/pixelization.py (80%) rename {likelihood => likelihood_breakdown}/interferometer/delaunay.py (70%) rename {likelihood => likelihood_runtime}/OPTIMIZATION_NOTES.md (100%) create mode 100644 likelihood_runtime/README.md rename scripts/aggregate_sweep.py => likelihood_runtime/aggregate.py (95%) create mode 100644 likelihood_runtime/datacube/__init__.py create mode 100644 likelihood_runtime/datacube/delaunay.py create mode 100644 likelihood_runtime/imaging/delaunay.py create mode 100644 likelihood_runtime/imaging/mge.py create mode 100644 likelihood_runtime/imaging/pixelization.py create mode 100644 likelihood_runtime/interferometer/delaunay.py rename {likelihood => likelihood_runtime}/interferometer/mge.py (99%) rename {likelihood => likelihood_runtime}/interferometer/pixelization.py (99%) rename {likelihood => likelihood_runtime}/point_source/image_plane.py (99%) rename {likelihood => likelihood_runtime}/point_source/source_plane.py (99%) rename scripts/sweep_likelihood.py => likelihood_runtime/sweep.py (94%) diff --git a/README.md b/README.md index 138d932..be3cc05 100644 --- a/README.md +++ b/README.md @@ -24,23 +24,9 @@ Results are framed by **astronomy instrument** (HST, Euclid, JWST, …) rather t ## Latest run-times -The table below is auto-generated from the latest versioned artifacts under `results/`. Each row is the latest steady-state per-call cost for a likelihood path at a given instrument; numbers refresh whenever the producing scripts are rerun and committed. Hardware tier is **CPU only** today — laptop GPU and HPC GPU columns will land once `results/**` artifacts are tagged with a hardware label. - - -| Section | Script | Instrument | Latest single-JIT per-call | PyAutoLens version | -|---------|--------|------------|----------------------------|--------------------| -| likelihood/datacube | `delaunay.py` | hannah | — | v2026.5.14.2 | -| likelihood/imaging | `delaunay.py` | hst | 833.4 ms | v2026.5.14.2 | -| likelihood/imaging | `mge.py` | hst | 41.6 ms | v2026.5.14.2 | -| likelihood/imaging | `pixelization.py` | hst | 782.3 ms | v2026.5.14.2 | -| likelihood/interferometer | `delaunay.py` | sma | 154.5 ms | v2026.5.14.2 | -| likelihood/interferometer | `mge.py` | sma | 33.6 ms | v2026.5.14.2 | -| likelihood/interferometer | `pixelization.py` | sma | 113.6 ms | v2026.5.14.2 | -| likelihood/point_source | `image_plane.py` | — | 22.5 ms | v2026.5.14.2 | -| likelihood/point_source | `source_plane.py` | — | 691 μs | v2026.5.14.2 | - - -(Generator: `scripts/build_readme.py`. Run `python scripts/build_readme.py` after producing new artifacts to refresh; `--check` exits non-zero in CI if it would change anything.) +Cell-level full-pipeline numbers live in [`likelihood_runtime/OPTIMIZATION_NOTES.md`](./likelihood_runtime/OPTIMIZATION_NOTES.md), which carries the latest CPU + local-GPU per-call costs together with per-cell "where to optimize next" recommendations and the mp-vs-fp64 verdicts. The detailed multi-config `comparison.json` artifacts are committed under [`autolens_workspace_developer/jax_profiling/results/jit///`](https://github.com/PyAutoLabs/autolens_workspace_developer/tree/main/jax_profiling/results/jit) and are re-aggregated by `likelihood_runtime/aggregate.py` whenever a new sweep finishes. + +(The previous auto-generated table in this README was retired when the likelihood profiling was split into `likelihood_breakdown/` + `likelihood_runtime/`; `scripts/build_readme.py` is queued for a path-update follow-up.) ## JAX gradients — currently out of scope @@ -67,7 +53,8 @@ Examples that already exist in the source-of-truth repo: | Folder | Contents | |--------|----------| -| [`likelihood/`](./likelihood/README.md) | Likelihood JIT profiling — imaging, interferometer, point-source, datacube. | +| [`likelihood_breakdown/`](./likelihood_breakdown/README.md) | Per-step JIT decomposition. Single config. *Where does time go inside the likelihood?* | +| [`likelihood_runtime/`](./likelihood_runtime/README.md) | Full-pipeline JIT only, driven by `sweep.py` across CPU/GPU/A100 × fp64/mp. *How long will this likelihood take on this hardware?* | | [`simulators/`](./simulators/README.md) | Run-time tracking for the PyAutoLens simulators. | | [`searches/`](./searches/README.md) | Sampler / search profiling, Nautilus first. | | [`results/`](./results/README.md) | Versioned JSON + PNG artifacts written by the above scripts. | diff --git a/likelihood/adapt_image_util.py b/_adapt_image_util.py similarity index 100% rename from likelihood/adapt_image_util.py rename to _adapt_image_util.py diff --git a/likelihood/_profile_cli.py b/_profile_cli.py similarity index 100% rename from likelihood/_profile_cli.py rename to _profile_cli.py diff --git a/likelihood/README.md b/likelihood/README.md deleted file mode 100644 index 0751281..0000000 --- a/likelihood/README.md +++ /dev/null @@ -1,63 +0,0 @@ -# likelihood - -JAX JIT profiling for the PyAutoLens likelihood function across imaging, interferometer, point-source, and datacube datasets, and across the MGE, pixelization, and Delaunay model compositions used in real science cases. - -## What "JIT likelihood profiling" means - -For each science case, the likelihood function turns a parameter vector into a single number (log-likelihood) via a chain of array operations: instantiate the model, build the `Tracer`, ray-trace grids through the lens, compute a mapping matrix, blur it with the PSF, solve a linear-algebra reconstruction problem, and finally compute a chi-squared. Under `xp=jnp`, every step is dispatched as a JAX op and can be compiled into a single XLA program with `jax.jit`. - -Profiling the **whole likelihood** as one JIT'd function gives the honest per-call cost a sampler will see in production. Profiling **each step individually** under its own JIT gives the breakdown that tells you where the time is going. Both numbers matter: the whole-function timing is the production cost, and the per-step breakdown is the optimisation target. Each script in this section reports both wherever the underlying pipeline supports per-step JIT-ing (the interferometer and datacube paths intentionally stay at full-pipeline JIT for now — see those subfolders' READMEs for why). - -Every script also reports a **batched (`jax.vmap`) per-likelihood cost** to make explicit how much the JIT amortises across a population of evaluations — the regime an actual sampler operates in. - -## How to read the per-script output - -Each script prints a structured narrative to stdout, ending in: - -- The eager (numpy) baseline log-likelihood for sanity. -- The single-JIT lower / compile / first-call / steady-state per-call timings. -- The vmap per-likelihood cost and speedup vs single-JIT. -- A correctness check: eager ≡ JIT ≡ vmap log-likelihoods at `rtol=1e-4`. -- A `results/likelihood//