I think this should include benchmarks zstd with larger windows and long range mode. I wouldn't be surprised if the window they used is smaller than an individual version tar file, which would prevent useful compression.
The benchmarks are disingenuous, to the point of looking cherry-picked. The block size for bzip3 is set to 512GB, but the window size for zstd is left to its default (8MB I believe for high levels). So in this corpus, which is made up of all versions of Perl source code concatenated, the window is too small to see all the identical files and just match them.
Also corpora made out of very long repetitions are pretty much the best case scenario for BWT-based compressors.
If we match the window size of zstd to that of bzip3 we get dramatically different results:
% gzcat *.gz | time zstd -T8 -16 | wc -c # baseline
2819113884
zstd -T8 -16 2054.50s user 3.47s system 783% cpu 4:22.80 total
% gzcat *.gz | time zstd -T8 -16 --long=29 | wc -c
196405076
zstd -T8 -16 --long=29 1083.06s user 2.41s system 783% cpu 2:18.55 total
Almost 15x smaller than the baseline, and more than 2x smaller than bzip3, also CPU time halves (since long matches are found earlier, so there's less work to do).
(the baseline number is slightly different because I don't have the exact Perl version set used by the author)
Also, in the benchmarks using lrzip, which would make the window size less relevant, zstd is not even compared.
I have not experimented with bzip3 recently, but more than a year ago I have done many tests with it.
Initially I was extremely impressed with it, because in a lot of tests it succeeded to compress hard-to-compress files, like movies, and in many cases it demonstrated a much better compromise between speed and compression ratio than zstd, i.e. depending on the command parameters I could make it either compress better than zstd at similar compression/decompression speed, or compress/decompress faster at a similar compression ratio.
Alas, the initial extremely favorable conclusion was short-lived, because trying later bzip3 on other data files gave worse results than zstd.
So the final conclusion was that the performance of bzip3 was somewhat unpredictable, being highly data dependent. For some files it provided outstanding compression ratio or speed, but for others it was inferior.
The problem was that without doing a compression there was no way to guess whether a file would be among those preferred by bzip3 or by zstd or by xz.
So now I would use it only for a file for which I want maximum compression and which I would compress once and decompress many times, so I can afford a very long compression time, during which I would test multiple compression algorithms, including bzip3 and zstd, with multiple parameter choices, and I would eventually choose the one that offers the best compromise between compression ratio and decompression time, for that particular file.
It certainly is a competitive compression algorithm, but unless it has changed since I last tested it, you cannot guess for which files it would win the compression competition.
Would a multi-stream archive format make sense at this point? I.e. store several compressed streams in the same file and use heuristics to decide where each file (or portion of file) goes.
The benchmark is very rudimentary. It does not test different levels/settings apart from its own -b 256/512 (does it affect decompression?), it doesn't measure compression time and memory usage. It does not specify parallel vs single-threaded (it mentions parallel on the one decoding number but what about the others?).
The lrzip test is interesting but it omits for example zstd and doesn't even have (de-)compression timings.
A lot more numbers are needed to present a fair and informative comparison.
I don't want this to be a swipe against bzip3, I only want to point out the presented benchmarks could be a lot better.
There is a comparison with "zstd -19" on the Silesia corpus, showing better compression ratio for bzip3 (47.2 vs 53MB) while being ~5 times faster (and using only half the memory).
Even if the examples are highly cherry-picked, it is quite suprising to me that such pareto-dominance is possible at all.
having used zstd, it has terrible defaults, optimized for speed and low-memory. You need to change it's params (not just level and dict size) to get high performance.
probably somebody should use a coding agent to do auto-research to optimize params for each compression algo, while matching one fixed goal - time, memory or size
If we match the window size of zstd to that of bzip3 we get dramatically different results:
Almost 15x smaller than the baseline, and more than 2x smaller than bzip3, also CPU time halves (since long matches are found earlier, so there's less work to do).(the baseline number is slightly different because I don't have the exact Perl version set used by the author)
Also, in the benchmarks using lrzip, which would make the window size less relevant, zstd is not even compared.
“Hi, tool author here.” A useful explanation of Burrows-Wheelers transform as used by bzip3: https://news.ycombinator.com/item?id=42902407
“bzip3 is not yet listed on the large text compression benchmark” It is now: https://mattmahoney.net/dc/text.html
(2 years ago, 176 comments) https://news.ycombinator.com/item?id=42899713
(4 years ago, 104 comments) https://news.ycombinator.com/item?id=31324439
Initially I was extremely impressed with it, because in a lot of tests it succeeded to compress hard-to-compress files, like movies, and in many cases it demonstrated a much better compromise between speed and compression ratio than zstd, i.e. depending on the command parameters I could make it either compress better than zstd at similar compression/decompression speed, or compress/decompress faster at a similar compression ratio.
Alas, the initial extremely favorable conclusion was short-lived, because trying later bzip3 on other data files gave worse results than zstd.
So the final conclusion was that the performance of bzip3 was somewhat unpredictable, being highly data dependent. For some files it provided outstanding compression ratio or speed, but for others it was inferior.
The problem was that without doing a compression there was no way to guess whether a file would be among those preferred by bzip3 or by zstd or by xz.
So now I would use it only for a file for which I want maximum compression and which I would compress once and decompress many times, so I can afford a very long compression time, during which I would test multiple compression algorithms, including bzip3 and zstd, with multiple parameter choices, and I would eventually choose the one that offers the best compromise between compression ratio and decompression time, for that particular file.
It certainly is a competitive compression algorithm, but unless it has changed since I last tested it, you cannot guess for which files it would win the compression competition.
The lrzip test is interesting but it omits for example zstd and doesn't even have (de-)compression timings.
A lot more numbers are needed to present a fair and informative comparison.
I don't want this to be a swipe against bzip3, I only want to point out the presented benchmarks could be a lot better.
how is this an honest benchmark:
Even if the examples are highly cherry-picked, it is quite suprising to me that such pareto-dominance is possible at all.
probably somebody should use a coding agent to do auto-research to optimize params for each compression algo, while matching one fixed goal - time, memory or size
The claim “stronger than bzip2” is strange. What does it even mean?
Also, comparing parallel decompression benchmarks with bzip2 instead of pbzip2 seems unfair.