feat(quick_update): IPython.display.update_display for live Jupyter cells

`BackgroundQuickUpdate` now pushes the freshly-written `subplot_fit.png`
into the active Jupyter / Colab cell via `IPython.display.update_display`
with a stable `display_id`, so the cell that ran `search.fit(...)` shows
a single self-updating image during the fit rather than just writing
PNGs to disk.

How it works:

- After `analysis.perform_quick_update(paths, instance)` returns, the
  daemon worker checks `_is_ipython_kernel()` (try-imports IPython,
  inspects `get_ipython().config` for `IPKernelApp`).
- If running in a kernel, locates `subplot_fit.png` (with `fit.png` /
  `subplot_tracer.png` fallbacks) under `paths.image_path` and calls
  `display(Image(filename=...), display_id="pyauto_fit_progress")` on
  the first iteration, `update_display(... same id)` on subsequent
  iterations.
- Reads the PNG from disk (not a matplotlib `Figure`) — the worker is
  a daemon thread, and matplotlib Figure handling is not thread-safe.

Outside a kernel (plain `python my_fit.py`), the display layer is
silently skipped — script behaviour is unchanged. Users inside a
kernel can opt out via `PYAUTO_DISABLE_IPYTHON_DISPLAY=1` (papermill,
nbconvert pipelines, etc.). Any IPython-side exception is logged and
swallowed so a display failure never takes the search down.

New optional kwarg `BackgroundQuickUpdate(display_id="pyauto_fit_progress")`
for users running two concurrent searches in the same kernel.

Tests added:
- `_is_ipython_kernel()` returns False in pytest (script-mode fallback).
- `_push_to_ipython()` no-ops when no PNG exists yet.
- With a mocked IPython.display module, first call uses `display(...)`
  with the configured display_id; subsequent calls use `update_display`
  with the same id.

Phase C of z_features/fast_visualization.md. Closes #1289
(library half; autofit_workspace cookbook docs follow after this lands
in a release).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: 2 people

autofit/non_linear/quick_update.py

@@ -1,6 +1,8 @@
 import copy
 import logging
+import os
 import threading
+from pathlib import Path
 
 import numpy as np
 
@@ -26,6 +28,14 @@ def _convert_jax_to_numpy(instance):
     return instance
 
 
+# Filenames the worker will look for under `paths.image_path`, in priority
+# order, when pushing a quick-update frame to a Jupyter / Colab cell. The
+# first existing file wins. Imaging analyses write `subplot_fit.png`;
+# interferometer / dataset-model variants may write `fit.png` or
+# `subplot_tracer.png`.
+_DISPLAY_CANDIDATES = ("subplot_fit.png", "fit.png", "subplot_tracer.png")
+
+
 class BackgroundQuickUpdate:
     """
     Runs ``analysis.perform_quick_update`` on a background daemon thread so
@@ -34,15 +44,38 @@ class BackgroundQuickUpdate:
     Uses a **latest-only** pattern: if a new best-fit arrives before the
     previous visualisation finishes, the stale request is silently replaced.
 
+    When the search is running inside a Jupyter / Colab kernel, the worker
+    additionally pushes the freshly-written subplot PNG to the active cell
+    via :func:`IPython.display.update_display` with a stable ``display_id``,
+    so the cell that ran ``search.fit(...)`` shows a single self-updating
+    image rather than just writing PNGs to disk. Outside a kernel (plain
+    ``python my_fit.py``) this layer is silently skipped — PNGs still land
+    on disk and no IPython side effects fire. Users can opt out inside a
+    kernel by setting ``PYAUTO_DISABLE_IPYTHON_DISPLAY=1`` (e.g. for
+    papermill / automated nbconvert pipelines).
+
     Parameters
     ----------
     convert_jax
         If ``True``, JAX arrays on the model instance are converted to
         NumPy before handing them to the worker thread.
+    display_id
+        Stable identifier passed to ``IPython.display.update_display`` so
+        every quick-update frame refreshes the same cell output element
+        rather than appending a new one. Default
+        ``"pyauto_fit_progress"`` is fine for almost all uses; override
+        only when running two concurrent searches in the same kernel and
+        wanting each in its own cell output.
     """
 
-    def __init__(self, convert_jax: bool = False):
+    def __init__(
+        self,
+        convert_jax: bool = False,
+        display_id: str = "pyauto_fit_progress",
+    ):
         self._convert_jax = convert_jax
+        self._display_id = display_id
+        self._display_initialised = False
 
         self._lock = threading.Lock()
         self._pending = None
@@ -76,6 +109,84 @@ def shutdown(self, timeout: float = 10.0):
         self._has_work.set()
         self._thread.join(timeout=timeout)
 
+    @staticmethod
+    def _is_ipython_kernel() -> bool:
+        """
+        Return ``True`` if running inside a Jupyter / Colab IPython kernel,
+        ``False`` otherwise (script mode, REPL, IPython terminal).
+
+        Detection works by importing ``IPython.get_ipython`` and checking
+        that the returned shell has ``IPKernelApp`` registered in its
+        config — only kernel-based shells (Jupyter, Colab, JupyterLab) do.
+        A plain IPython terminal returns a shell without the kernel app,
+        so it does not match. ``ImportError`` (IPython not installed) and
+        any other exception are swallowed: the worker stays decoupled from
+        the optional IPython dependency.
+        """
+        try:
+            from IPython import get_ipython
+        except ImportError:
+            return False
+        try:
+            ipy = get_ipython()
+            if ipy is None:
+                return False
+            return "IPKernelApp" in getattr(ipy, "config", {})
+        except Exception:
+            return False
+
+    def _resolve_display_image_path(self, paths):
+        """
+        Return the first existing PNG under ``paths.image_path`` that
+        matches one of the canonical quick-update filenames, or ``None``
+        if none of them exist yet (e.g. the analysis ran with
+        ``PYAUTO_FAST_PLOTS=1`` and wrote no figures this iteration).
+        """
+        image_path = getattr(paths, "image_path", None)
+        if image_path is None:
+            return None
+        base = Path(image_path)
+        for name in _DISPLAY_CANDIDATES:
+            candidate = base / name
+            if candidate.exists():
+                return candidate
+        return None
+
+    def _push_to_ipython(self, paths):
+        """
+        Display or refresh the latest quick-update subplot in the active
+        IPython cell using a stable ``display_id``.
+
+        The first call publishes the image via ``display(... display_id=...)``;
+        subsequent calls update it in place via
+        ``update_display(... display_id=...)``. The image is read from disk
+        as PNG bytes — we deliberately do **not** touch matplotlib Figure
+        objects here because this method runs on the daemon worker thread
+        and matplotlib Figures are not thread-safe.
+
+        Skipped entirely when ``PYAUTO_DISABLE_IPYTHON_DISPLAY=1`` is set
+        — useful for papermill / nbconvert pipelines that want PNGs on
+        disk but no inline display side effects.
+        """
+        if os.environ.get("PYAUTO_DISABLE_IPYTHON_DISPLAY") == "1":
+            return
+
+        png_path = self._resolve_display_image_path(paths)
+        if png_path is None:
+            return
+
+        try:
+            from IPython.display import Image, display, update_display
+        except ImportError:
+            return
+
+        img = Image(filename=str(png_path))
+        if self._display_initialised:
+            update_display(img, display_id=self._display_id)
+        else:
+            display(img, display_id=self._display_id)
+            self._display_initialised = True
+
     def _process_pending(self):
         with self._lock:
             work = self._pending
@@ -89,11 +200,24 @@ def _process_pending(self):
         try:
             analysis.perform_quick_update(paths, instance)
         except NotImplementedError:
-            pass
+            return
         except Exception:
             logger.exception(
                 "Background quick-update raised an exception (ignored)."
             )
+            return
+
+        # If running inside a Jupyter / Colab kernel, push the freshly-
+        # written subplot PNG to the active cell so the user sees the fit
+        # update in place. Any failure here is logged-and-swallowed so a
+        # display problem never takes the search down.
+        if self._is_ipython_kernel():
+            try:
+                self._push_to_ipython(paths)
+            except Exception:
+                logger.exception(
+                    "IPython display update raised an exception (ignored)."
+                )
 
     def _worker(self):
         while True:

test_autofit/non_linear/test_quick_update.py

@@ -161,3 +161,103 @@ def __init__(self):
         worker.shutdown()
 
         assert isinstance(analysis.calls[0].param, np.ndarray)
+
+
+class TestIPythonDisplayLayer:
+    """
+    Covers the Jupyter / Colab cell live-update wiring added on top of the
+    existing background-thread / latest-only / log-and-swallow behaviour.
+
+    The display layer is gated on `_is_ipython_kernel()` so script-mode
+    behaviour (PNG-on-disk, no IPython side effects) is preserved.
+    """
+
+    def test_is_ipython_kernel_false_in_pytest(self):
+        """
+        Plain pytest does not run inside a Jupyter / Colab kernel, so the
+        detection helper must return False. This is the script-mode
+        fallback path users rely on when running `python my_fit.py`.
+        """
+        worker = BackgroundQuickUpdate()
+        try:
+            assert worker._is_ipython_kernel() is False
+        finally:
+            worker.shutdown()
+
+    def test_push_to_ipython_no_op_when_png_missing(self, tmp_path):
+        """
+        If `perform_quick_update` produced no PNGs (e.g. `PYAUTO_FAST_PLOTS=1`
+        suppressed them, or an early-iteration scenario where the visualizer
+        wrote nothing yet), `_push_to_ipython` must silently no-op rather
+        than raising — the search must not be taken down by a missing file.
+        """
+        class Paths:
+            image_path = tmp_path  # exists but contains no PNGs
+
+        worker = BackgroundQuickUpdate()
+        try:
+            # No exception expected, no display side effects.
+            worker._push_to_ipython(Paths())
+            assert worker._display_initialised is False
+        finally:
+            worker.shutdown()
+
+    def test_push_to_ipython_display_then_update_sequence(
+        self, tmp_path, monkeypatch
+    ):
+        """
+        With a fake IPython.display module installed in `sys.modules`, the
+        first `_push_to_ipython` call must invoke `display(..., display_id=...)`
+        and the second must invoke `update_display(..., display_id=...)` with
+        the same id. This is the contract that lets the notebook cell
+        refresh in place rather than appending stacked frames.
+        """
+        import sys
+        import types
+
+        png_path = tmp_path / "subplot_fit.png"
+        png_path.write_bytes(b"\x89PNG\r\n\x1a\n" + b"\x00" * 16)  # fake PNG header
+
+        calls = []
+
+        def fake_display(obj, display_id=None):
+            calls.append(("display", display_id))
+
+        def fake_update_display(obj, display_id=None):
+            calls.append(("update_display", display_id))
+
+        class FakeImage:
+            def __init__(self, filename=None):
+                self.filename = filename
+
+        fake_display_module = types.ModuleType("IPython.display")
+        fake_display_module.display = fake_display
+        fake_display_module.update_display = fake_update_display
+        fake_display_module.Image = FakeImage
+        fake_ipython_module = types.ModuleType("IPython")
+        fake_ipython_module.display = fake_display_module
+
+        monkeypatch.setitem(sys.modules, "IPython", fake_ipython_module)
+        monkeypatch.setitem(sys.modules, "IPython.display", fake_display_module)
+        monkeypatch.delenv("PYAUTO_DISABLE_IPYTHON_DISPLAY", raising=False)
+
+        class Paths:
+            image_path = tmp_path
+
+        worker = BackgroundQuickUpdate(display_id="test-display-id")
+        try:
+            worker._push_to_ipython(Paths())
+            worker._push_to_ipython(Paths())
+            worker._push_to_ipython(Paths())
+        finally:
+            worker.shutdown()
+
+        # First call uses `display`, subsequent calls use `update_display`,
+        # and the display_id is stable across all of them so the cell
+        # output is replaced rather than appended.
+        assert calls == [
+            ("display", "test-display-id"),
+            ("update_display", "test-display-id"),
+            ("update_display", "test-display-id"),
+        ]
+        assert worker._display_initialised is True