Skip to content

Single Stream Deflate
Compare
Choose a tag to compare
@sstadick sstadick released this 22 Aug 22:55
· 69 commits to main since this release
v0.5.0
a49c497

v0.5.0 is a large rework of gzp on several fronts. Overall, gzp now behaves more like pigz.

  • Single stream deflate encoding is used instead of Gzip blocks catted together. This means that both Zlib and RawDeflate formats are now supported.
  • The last 32K of the previously encoded block (in sequential order) is used as the dictionary for the current block to compress. This improves overall compression and matches pigz
  • There is now a single "engine" that drives parallel compression for all formats, including Snap. To add a new format you simply need to implement the FormatSpec trait.

Performance stayed about the same through all of this. On toy benchmark programs (examples/test2.rs) gzp appears to be about 30% faster than pigz using equivalent compression levels, buffer sizes, and number of threads. This is very anecdotal and should be taken with many grains of salt.

Assets 2
Loading