Memory footprint

[castelao]

Formal and systematic track of memory footprint by:

• input features size;
• input features chunk size;
• number of layers used in the cost function;
• Distance from start to end point. This might be the most critical aspect with the current implementation. We shall have a good estimate how it scales so we can estimate requirements and limitations based on the test cases.

[castelao]

I already started this. I think I found a good approach for this, but I still need to organize it so we can run it easily to keep track of the changes.

[castelao]

For the records, I implemented criterion some PRs ago.

[castelao]

I did several cases manually and I'm convinced that we are fine memory-wise. The indexmap can be large, but everything should be a manageable size. I'll have to leave the systematic tests for another milestone.

[castelao]

I'm downgrading priority to P1. It is still valuable doing it systematically, but manual runs gave enough confidence that it is as we expected.