Compression levels

[fintelia]

I've been working off-and-on to build better compression for fdeflate. This draft-PR captures the current state of my experimentation. So far, I've mainly focused on the core compression algorithms, so other details aren't handled yet like streaming (at the moment the input is accumulated into a Vec and compressed on flush) or compressing >4GB worth of data (requires a bit of handling code for 32-bit indices rolling over).

Like other compression libraries, this uses different algorithms depending on the compression level:

• Level zero uses a dedicated stored-block encoding path
• Levels 1-2 look for RLE matches of zero bytes and 8-byte matches via a hash table. Based on lz4's compressor.
• Levels 3-6 use a more traditional hash chain match finder with a minimum match size of 4-bytes. Also does greedy matching but with look-ahead to "fizzle" matches that's inspired by zlib-ng.
• Levels 7+ are the least polished. They use a binary tree match finder with lazy matching. Much slower than zlib-rs's level 9, but better compression.

Results on raw PNG IDAT data produced by re-encoding the QOI benchmark suite images, reporting the geometric mean of speed and compression ratio...

Encoder              Speed     Ratio
----------     -----------    ------
fdeflate[0]: 12529.7 MiB/s   100.02%
fdeflate[1]:   396.4 MiB/s    24.77%
fdeflate[2]:   229.6 MiB/s    24.58%
fdeflate[3]:   112.5 MiB/s    24.30%
fdeflate[4]:    71.4 MiB/s    23.94%
fdeflate[5]:    67.5 MiB/s    23.90%
fdeflate[6]:    38.2 MiB/s    23.53%
   ...
fdeflate[9]:    10.3 MiB/s    22.72%

See this comment with comparable measurements from other compressors.