>8bit depth support

[trissim]

This library is super useful but it doesn't support stitching images that have a bit depth other than 8bit. Also, would it be possible to add an option to load image arrays directly rather than loading them from a file?

[adamancer]

It's not in the guide, but you can pass a list of arrays directly to create_mosaic() instead of a path. That function does expect the arrays to be image-like (so 1, 3, or 4 channels) but it should handle anything that the underlying libraries support (opencv by default). It sounds like reading anything other than 8-bit images is not working for you--do you have example tiles that I could take a look at?

I'll have to think about arrays with additional bands, but it should be possible to add support for that.

[trissim]

Here are some 12bit tif images (they appear as 16bit under FIJI but my microscope camera is 12bit): https://mcgill-my.sharepoint.com/:f:/g/personal/tristan_simas_mail_mcgill_ca/EovKa9rRji9FsnadWTx3BTEBXBIYvgYDIWm_mWXtrDI3Pw?e=bZNJu1

Thanks for the quick reply! This library is super useful, I'm looking forward to making use of it more

[adamancer]

Ah, interesting. I really haven't done much with this beyond the microscopes in my department and hadn't run into 12-bit images before.

It is possible to build a mosaic of the original images by aligning 8-bit images, then using the resulting parameter file to stitch the originals, like so:

import glob

import numpy as np
from stitch2d import OpenCVTile, create_mosaic

# Align mosaic using 8-bit arrays
tiles = [OpenCVTile(p) for p in glob.iglob("tiles/*.tif")]
for tile in tiles:
    tile.imdata = np.uint8(tile.imdata)
mosaic = create_mosaic(tiles)
mosaic.align()
mosaic.save_params()

# Build mosaic using the original images
mosaic = create_mosaic("tiles")
mosaic.load_params()
mosaic.save("mosaic.tif")

Note that the mosaic TIFF is 16 bit, not 12 bit, since opencv does the same thing as Fiji and reads the 12-bit images into 16-bit arrays. It doesn't seem to modify the original pixel values, so the images look dark to me.

Can you give this a try and see if it works for you? The mosaic looked okay to me but I could be missing something.

There is something a little odd about this. I was expecting to have to rescale the image when converting it to 8-bit, but doing that prevented the mosaic from aligning. I'm not too worried about it but it might be worth thinking about.

[trissim]

Hi Adam,

Your suggestions have been a great help! I have implemented your suggestions into my code. My code looks as follows:

def stitch_images(images,dim,origin="lower right",direction="horizontal",pattern="raster",mm_img=False,load_params=None):

    #microscope specific stuff
    if mm_img:
        images = [image.img for image in images]

    #tile size dimXdim
    dim=int(dim)

    #if haven't stitched with these parameters, then compute them
    if  load_params is None:
        # Align mosaic using 8-bit arrays
        images_8bit = []
        for i,image in enumerate(images):
            image = OpenCVTile(image)
            image.imdata = np.uint8(image.imdata)
            #trying to increase contrast to get image registration to work better
            #image.imdata = cv2.equalizeHist(image.imdata)
            images_8bit.append(image)

        #do the stitching 
        mosaic = StructuredMosaic(images_8bit,
        dim=dim,
        origin=origin,     # position of first tile
        direction=direction,  # primary axis (i.e., the direction to traverse first)
        pattern=pattern          # snake or raster
        )
        mosaic.align()
        mosaic.save_params()
    tiles = []
    for image in images:
        tiles.append(OpenCVTile(image))
    mosaic = create_mosaic(tiles,dim=dim,origin=origin,direction=direction,pattern=pattern)
    mosaic.load_params()
    return mosaic.stitch()

So the code works well on low number of tiles, like 2x2, but on larger tiles, it will sometimes work, and sometimes it will miss images. It seems like it depends on the content of the images. However, even though my images are quite dim, there should be enough features to allow for stitching.

I have uploaded some images I took with my microscope that I am trying to stitch. You can find them here: https://mcgill-my.sharepoint.com/:f:/g/personal/tristan_simas_mail_mcgill_ca/EjyOEoMbsz9EnJ_1ePSjoHgBYHc2v65w-clmCWTHoE7y6g?e=A6qB4S

I took 16 images in a 4x4 grid with 20% overlap for two fluorescent channels, DAPI (contains neuronal nuclei) and TRITC (contains neuron cell bodies and neurites). The DAPI channel mosaic completely fails, showing only 1 tile. The TRITC channel mosaic is better, showing 5 images being tiled.

Since the spacing between each image is constant, shouldn't there be a way to get the stitcher to stitch all the images once it knows the X and Y distance required to get alignment?

I have also tried normalizing the histogram of the 16bit images before turning them into 8 bit images in order to increase contrast to help with image registration. This doesn't work consistently.

Thanks for your help!

[adamancer]

For the TRITC mosaic, you can add mosaic.build_out() to place the tiles that could not be aligned using an average offset between tiles.

The DAPI tiles are more complicated. It looks to me like there's a lot of internal structure to the blobs on those images, and that the blobs look quite different when they repeat in neighboring tiles. That could be tripping up the feature matching algorithm. Blurring and resizing the images to average out the differences didn't work, so I think I'm out of my depth. Feature detection and matching is all handled by the opencv library, so you may have better luck asking about strategies for addressing this on StackOverflow. If you can figure out how to modify the images to look more similar, you would be able to load them back into the imdata attribute and the mosaicker could continue.