Live Stitch

[alexlsa]

Hello,

Thanks for this nice and usefull stitching project

I am able to stitch my microscopic images, but I have a question:

Is it possible to stitch the images when scanning the tissue ? Because my images are high resolution (5Mp) and there are a lot of them(Sometimes 2000 - 3000 tiles). Stitching these tiles after scanning takes much time

Thanks again

[adamancer]

Hm maybe. Can you provide a little more detail about your process? For example, are you exporting the images to a folder as you go and just want to start stitching as images are being captured? Or are you hoping to be able to see the image as it's being stitched together?

[alexlsa]

Hello, first of all thank you for your interest

Normally, when scanning, I can both save the images to disk or send the images to a different process with a multiprocessing queue mechanism and process the images in this different process.

Simply I do this ;

import queue # In my code this is multiprocessing queue
import numpy as np

frames = queue.Queue()

def feed():
    while 1:
        frames.put(get_frame())

stitched = None
while 1:
    frame = frames.get()
    stitched = stitch2d.stitch(stitched, frame) 

While scanning, the focused image comes from the camera, I want to do the stitching process as these images come in and save the stitched image to the file when the scan is finished

[alexlsa]

I can help you with the coding, where should we start when we want to do this?

[adamancer]

Here's a potential approach you can try. For each iteration, the script builds a mosaic, then copies the tile position from the mosaic created during the previous iteration, which means that it should only place the new tiles. When no more tiles are found, it stops.

import shutil
import time
from pathlib import Path

from stitch2d import create_mosaic


prev_mosaic = None
prev_tiles = []

while True:

    # Get tiles from the working directory
    tiles = [str(t) for t in Path("working").glob("*.tif")]
    print(f"Found {len(tiles)} tiles")

    # Stop processing if no new tiles have been added
    if tiles == prev_tiles:
        break

    # Wait a few seconds to make sure the last tile has finished saving
    time.sleep(5)

    # Copy tile data from the previous iteration
    mosaic = create_mosaic(tiles, dim=11)
    if prev_mosaic is not None:
        prev_tiles = {t.source: t for t in prev_mosaic.tiles if isinstance(t.source, str)}
        for tile in mosaic.tiles:
            try:
                prev_tile = prev_tiles[tile.source]
            except (KeyError, TypeError):
                pass
            else:
                for attr in ("id", "y", "x"):
                    setattr(tile, attr, getattr(prev_tile, attr))
            
    # Align the mosaic. Tiles that were previously processed are skipped.
    mosaic.downsample(0.6)
    mosaic.align()
    mosaic.reset_tiles()
    mosaic.save("mosaic.jpg")

    # Update the previous iteration
    prev_mosaic = mosaic
    prev_tiles = tiles

Anecdotally, I've found that this approach generally works but seems to have a somewhat higher likelihood of a bad tile placement, especially early on. You might consider waiting until you've accumulated some tiles or even a whole row before aligning.

[alexlsa]

First of all thank you so much @adamancer

I am trying to stitch my images with the code you shared, I will experiment with new images to correct the results I get, I will also experiment by increasing the overlap amount and see how the result changes and then I will share it with you here

It's great that your project can be quickly adapted for live stitch

Thanks again

[alexlsa]

Hello,

I tried to stitch my images normally before trying live stitching, I was able to successfully stitch some images, but my other images was not stitched. I got RuntimeError: Tiles could not be aligned error Normally there is a 10% overlap between the images, I changed it to 15% but I got the same error. If you have time you can run the code I shared on the dataset

I also disabled downsampling, but it still give error

This is drive link to download images

Thank you very much