The builtin ArrayFuncs.Flatten method is very slow on bigger data.

[laheller]

Hi @rwg0, @elendil-software

I made some tests on the builtin ArrayFuncs.Flatten function and found out that the performance is very slow on FITS images with a higher resolution and 3 image planes, where the 3 AXIS parameters are for example:

NAXIS = 3 / number of axes
NAXIS1 = 2048 / image width
NAXIS2 = 1364 / image height
NAXIS3 = 3 / image planes

I made a simple test code which demonstrates it perfectly and also a possible solution for comparison.
As a test FITS file I used this one.

using nom.tam.fits;
using nom.tam.util;
using System.Diagnostics;

namespace CSharpFits_test {
    internal class Program {
        static void Main(string[] args) {
            Console.WriteLine("CSharpFits ArrayFuncs.Flatten tests.");
            Console.WriteLine();

            var fpath = @"C:\Users\hellerl1\Pictures\Sas.fits";

            var fits = new Fits(fpath);
            var hdu = (ImageHDU)fits.ReadHDU();
            var axes = hdu.Axes;
            if (axes.Length != 3 || hdu.BitPix != BasicHDU.BITPIX_BYTE) {
                Console.WriteLine("For this test an 3D cube FITS image with 8bit data is expected!");
                return;
            }

            var width = axes[2];
            var height = axes[1];

            Console.WriteLine($"Number of axes: {axes.Length}");
            Console.WriteLine($"Number of bits per pixel per plane: {hdu.BitPix}");
            Console.WriteLine($"Image size: {width} x {height}");

            var planes = (Array[])hdu.Kernel;   // get list of planes from 3D cube FITS image HDU
            var plane0 = planes[0]; // the Blue plane from BGR planes

            Console.WriteLine();
            Console.WriteLine("Builtin flatten test:");
            var sw = Stopwatch.StartNew();
            var plane0data = (byte[])ArrayFuncs.Flatten(plane0); // flatten plane0 jagged array to byte array
            sw.Stop();
            Console.WriteLine($"Flatten#1 result: {sw.Elapsed.TotalSeconds} seconds");

            Console.WriteLine();
            Console.WriteLine("Custom flatten test:");
            plane0data = new byte[width * height];
            var c = 0;
            sw.Restart();
            foreach (var row in (Array[])plane0) foreach (var col in (byte[])row) plane0data[c++] = col; // element wise flatten
            sw.Stop();
            Console.WriteLine($"Flatten#2 result: {sw.Elapsed.TotalSeconds} seconds");

            fits.Close();
        }
    }
}

Result - output:

CSharpFits ArrayFuncs.Flatten tests.

Number of axes: 3
Number of bits per pixel per plane: 8
Image size: 2048 x 1364

Builtin flatten test:
Flatten#1 result: 0,1514039 seconds

Custom flatten test:
Flatten#2 result: 0,0061239 seconds

As you can see from the output, the simple element-wise array flattening in the second test is by 2 orders of magnitude faster than the builtin one in the first test. I know the builtin flatten function is a general one for all cases and data formats but for sure it can be improved.

BR,

Ladislav