Overview
Carve-out from PyAutoFit #1279 (Phase 4 of the JAX visualization roadmap). A picklability spike found that FitImaging cannot be pickled today because AbstractMeshGeometry.__init__ stores self._xp = xp — the literal numpy or jax.numpy module — and Python can't pickle modules. This blocks sending a FitImaging over an mp.Process+Queue or ProcessPoolExecutor boundary, which is the production design for Phase 4 subprocess visualization.
Fix: replace the module-attribute pattern with the _use_jax: bool + _xp property pattern already used in Analysis._xp (PyAutoFit) and AbstractMaker._xp (PyAutoArray decorators per CLAUDE.md). Eliminates the pickle barrier at the API level instead of working around it with __getstate__/__setstate__ hooks.
Spike evidence (PyAutoFit #1279):
- Before fix:
TypeError: cannot pickle 'module' object at inversion.linear_obj_list[0].interpolator.mesh_geometry._xp
- After fix (proven via monkey-patch):
FitImaging round-trips through pickle.dumps/loads with log_likelihood delta 0.00e+00 on both numpy and JAX backends. Pickle size ~4.6 MB.
Plan
- Change
AbstractMeshGeometry.__init__ to store self._use_jax = xp is not np instead of self._xp = xp
- Add
_xp as a @property deriving from self._use_jax (returns np or jnp on demand)
- All existing call sites read
self._xp (no writes) — they continue to work transparently via the property
- Add a unit test that pickles a
MeshGeometryRectangular instance and verifies _xp round-trips to the correct module
Detailed implementation plan
Affected Repositories
- PyAutoArray (primary, library, only repo touched)
Work Classification
Library
Branch Survey
| Repository |
Current Branch |
Dirty? |
| ./PyAutoArray |
main |
clean (canonical) — knn-barycentric task holds it via worktree, file-disjoint |
Suggested branch: feature/mesh-geometry-picklable
Worktree root: ~/Code/PyAutoLabs-wt/mesh-geometry-picklable/
Conflict resolution: File-level coexistence on PyAutoArray with knn-barycentric (#317). KNN edits inversion/mesh/interpolator/, inversion/mesh/mesh/, and plot/* — no overlap with our scope (inversion/mesh/mesh_geometry/abstract.py). Same precedent as ag-interferometer-kwargs.
Implementation Steps
autoarray/inversion/mesh/mesh_geometry/abstract.py:
- In
__init__: replace self._xp = xp with self._use_jax = xp is not np
- Add
@property method _xp that returns jax.numpy if _use_jax else numpy
- Import of
jax.numpy stays lazy (only when _use_jax=True and _xp is accessed) — matches Analysis._xp pattern in PyAutoFit
test_autoarray/inversion/pixelization/mesh_geometry/test_picklability.py (new):
test_picklable_numpy_backend — construct via __new__ + _use_jax=False, pickle round-trip, assert restored._xp is np
test_picklable_jax_backend — same with _use_jax=True, assert restored._xp is jnp (pytest.importorskip('jax'))
- Run the existing mesh_geometry tests + the new pickle tests + any pixelization tests that touch
_xp to verify no regression
Key Files
PyAutoArray/autoarray/inversion/mesh/mesh_geometry/abstract.py (line 23 + new property)
PyAutoArray/test_autoarray/inversion/pixelization/mesh_geometry/test_picklability.py (new)
Out of scope
__getstate__/__setstate__ workaround (Option A in the parent issue's spike) — rejected in favour of the cleaner property pattern (Option B)
- Subprocess visualization itself — that's the parent task #1279
FitInterferometer NUFFT picklability — separate spike under #1279
Parent
PyAutoFit #1279 — Phase 4 subprocess visualization feasibility (carve-out of the picklability prereq)
Overview
Carve-out from PyAutoFit #1279 (Phase 4 of the JAX visualization roadmap). A picklability spike found that
FitImagingcannot be pickled today becauseAbstractMeshGeometry.__init__storesself._xp = xp— the literalnumpyorjax.numpymodule — and Python can't pickle modules. This blocks sending aFitImagingover anmp.Process+QueueorProcessPoolExecutorboundary, which is the production design for Phase 4 subprocess visualization.Fix: replace the module-attribute pattern with the
_use_jax: bool+_xpproperty pattern already used inAnalysis._xp(PyAutoFit) andAbstractMaker._xp(PyAutoArray decorators perCLAUDE.md). Eliminates the pickle barrier at the API level instead of working around it with__getstate__/__setstate__hooks.Spike evidence (PyAutoFit #1279):
TypeError: cannot pickle 'module' objectatinversion.linear_obj_list[0].interpolator.mesh_geometry._xpFitImaginground-trips through pickle.dumps/loads withlog_likelihooddelta0.00e+00on both numpy and JAX backends. Pickle size ~4.6 MB.Plan
AbstractMeshGeometry.__init__to storeself._use_jax = xp is not npinstead ofself._xp = xp_xpas a@propertyderiving fromself._use_jax(returnsnporjnpon demand)self._xp(no writes) — they continue to work transparently via the propertyMeshGeometryRectangularinstance and verifies_xpround-trips to the correct moduleDetailed implementation plan
Affected Repositories
Work Classification
Library
Branch Survey
knn-barycentrictask holds it via worktree, file-disjointSuggested branch:
feature/mesh-geometry-picklableWorktree root:
~/Code/PyAutoLabs-wt/mesh-geometry-picklable/Conflict resolution: File-level coexistence on PyAutoArray with
knn-barycentric(#317). KNN editsinversion/mesh/interpolator/,inversion/mesh/mesh/, andplot/*— no overlap with our scope (inversion/mesh/mesh_geometry/abstract.py). Same precedent asag-interferometer-kwargs.Implementation Steps
autoarray/inversion/mesh/mesh_geometry/abstract.py:__init__: replaceself._xp = xpwithself._use_jax = xp is not np@propertymethod_xpthat returnsjax.numpyif_use_jaxelsenumpyjax.numpystays lazy (only when_use_jax=Trueand_xpis accessed) — matchesAnalysis._xppattern in PyAutoFittest_autoarray/inversion/pixelization/mesh_geometry/test_picklability.py(new):test_picklable_numpy_backend— construct via__new__+_use_jax=False, pickle round-trip, assertrestored._xp is nptest_picklable_jax_backend— same with_use_jax=True, assertrestored._xp is jnp(pytest.importorskip('jax'))_xpto verify no regressionKey Files
PyAutoArray/autoarray/inversion/mesh/mesh_geometry/abstract.py(line 23 + new property)PyAutoArray/test_autoarray/inversion/pixelization/mesh_geometry/test_picklability.py(new)Out of scope
__getstate__/__setstate__workaround (Option A in the parent issue's spike) — rejected in favour of the cleaner property pattern (Option B)FitInterferometerNUFFT picklability — separate spike under #1279Parent
PyAutoFit #1279 — Phase 4 subprocess visualization feasibility (carve-out of the picklability prereq)