LZ4

LZ4 is lossless compression algorithm, providing compression speed 500 MB/s per core, scalable with multi-cores CPU. It features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems. Speed can be tuned dynamically, selecting an "acceleration" factor which trades compression ratio for faster speed. On the other end, a high compression derivative, LZ4 HC, is also provided, trading CPU time for improved compression ratio. All versions feature the same decompression speed. LZ4 is also compatible with dictionary compression, both at API and CLI levels. It can ingest any input file as dictionary, though only the final 64KB are used. This capability can be combined with the Zstandard Dictionary Builder, in order to drastically improve compression performance on small files. The benchmark uses [lzbench], from @inikep compiled with GCC v8.2.0 on Linux 64-bits (Ubuntu 4.18.0-17). The reference system uses a Core i7-9700K CPU @ 4.9GHz (w/ turbo boost). Benchmark evaluates the compression of reference [Silesia Corpus] in single-thread mode. The raw LZ4 block compression format is detailed within [lz4 Block format]. Arbitrarily long files or data streams are compressed using multiple blocks, for streaming requirements. These blocks are organized into a frame, defined into [lz4 Frame format]. Interoperable versions of LZ4 must also respect the frame format. [lz4 Block format]: doc/lz4 Block format.md [lz4 Frame format]: doc/lz4 Frame format.md Most distributions are bundled with a package manager which allows easy installation of both the liblz4 library and the lz4 command line interface.