feat: add circular/ring buffer and data wrapper, that can be used to facilitate a streaming viewer

[tlambert03]

#134 (now closed) was trying to add a simplified way to stream data into the viewer (as if coming off the scope). The primary challenge is mostly just to store that data somewhere, so the user can simply call append() as new frames come in, and then tell the viewer to update. The key bit of tech then is just a nice storage API for a ring-buffer.

This PR adds a ring buffer object (vendored and updated from https://github.com/eric-wieser/numpy_ringbuffer), and makes a DataWrapper for it so that you c

import ndv
from ndv.models import RingBuffer

MAX_FRAMES = 5
FRAME_SHAPE = (512, 512)
rb = RingBuffer(max_capacity=MAX_FRAMES, dtype=('uint16', FRAME_SHAPE))


viewer = ndv.ArrayViewer(rb)
viewer.show()

# and then later, to add frames:
for i in range(10):
    rb.append(np.random.randint(0, 60_000, FRAME_SHAPE).astype('uint16'))

    # and optionally update the viewer position
    viewer.display_model.current_index.update({0: max(MAX_FRAMES, i - 1)})

[codecov]

Codecov Report

Attention: Patch coverage is 88.82353% with 19 lines in your changes missing coverage. Please review.

Project coverage is 85.39%. Comparing base (87a10a9) to head (0f3ace6).

Files with missing lines	Patch %	Lines
src/ndv/models/_data_wrapper.py	48.14%	14 Missing ⚠️
src/ndv/models/_ring_buffer.py	97.79%	3 Missing ⚠️
src/ndv/controllers/_array_viewer.py	50.00%	1 Missing ⚠️
src/ndv/views/_app.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #186      +/-   ##
==========================================
+ Coverage   85.11%   85.39%   +0.28%     
==========================================
  Files          45       46       +1     
  Lines        4783     4951     +168     
==========================================
+ Hits         4071     4228     +157     
- Misses        712      723      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tlambert03]

@jacopoabramo ... let me know how you feel about this API

[jacopoabramo]

Looks nice! One question I have: supposing that the data incoming from a stream is processed on-the-fly, and the new data type of the array is changed (i.e. from np.uint16 to np.float32), I imagine the buffer has to be flushed/deleted and a new one created; is this possible to do with this PR?

EDIT: a second question, now that I've been thinking about it. Is a RingBuffer really necessary for this kind of operations? I initially imagined that a simple direct update similar to what the StreamViewer was attempting to do should suffice, but maybe I'm missing something.

[tlambert03]

and the new data type of the array is changed (i.e. from np.uint16 to np.float32)

then you should make a float32 ring buffer… not a uint16 one. Point is you need to create the buffer with whatever dtype you are going to append. You have control over that.

Is a RingBuffer really necessary for this kind of operations?

no, if one only ever wants to look at the last image ( and never see any previous images) then you don’t need this. You could use a ring buffer of max_capacity 1 for that too

[gselzer]

I love the functionality! Just wondering how much API actually has to be exposed...