Issue with High Memory Usage After Lowering Prevalence Threshold for Shortlisting Genomes

[qwenlingli-png]

I'm using MIMIC2 and have adjusted the "Prevalence filtering threshold for shortlisting genomes" to allow specific strains to be included in the selection. However, by lowering the prevalence threshold, I end up with over 70 genomes being selected for the next scoring step. This causes a significant increase in memory usage (%MEM), leading to performance issues.

Is there a way to reduce memory usage while still ensuring that these specific strains are included in the selection, or any other approaches to handle large numbers of genomes without overwhelming the memory?

Thanks in advance for your help!

Best regards

[thh32]

Yeah this is a major issue that occurs when too many species are allowed to pass the filtering. We always tried to keep the number below 30 so the analysis is done in a reasonable timeframe.

To resolve this try increasing the prev filter, I usually use a filter of 25% which works most of the time, but sometimes have to increase it to 30% or decrease it to 10% depending on the data.

There is no way to alter memory usage currently, but I will look into how we could add that in future.

Best,
Tom