v2.1.0

Dolos v2.1.0 is now 3x faster and uses 10x less memory

With people trying out Dolos in increasingly diverse situations, the CLI was often hitting its limits. In our effort to improve the overall performance #974, we've optimized the underlying algorithm to use less memory and run faster. We're very proud of the results: we're seeing speed-ups of around 3 to 4 times faster and a memory footprint that is an order of magnitude smaller. The specific numbers and details of the improvements are available in the PR: #1011.

This means we can now analyze larger datasets. Currently we can handle datasets of more than 1000 files in 30 seconds, using only around one 1000 MB memory.

Next up: improving the Web UI for a smoother experience showing results of very large datasets.

🚀 Optimizations

• Optimize threshold calculation (#1040) @rien
• Optimize memory usage in library (#1011) @rien

🐛 Bug Fixes

• Fix similarity value of NaN when files are empty (#1041) @rien
• Fix --fragment-sort-by not working (#993) @rien
• Fix crash when terminal output is not an interactive console (getColorDepth is not a function ) (#992) @rien

🧰 Maintenance

• Refactor file store to handle labels better (#1052) @rien
• Code cleanup of some stores (#1044) @rien