Whole-process gzip command line benchmarks across implementations.
bench.py times whole processes, stdin to stdout, so startup, threading,
and IO all count. Each run covers the level ladder, thread sweeps at
level 6, a decompression grid crossing decoders with producers, and a
deflate block census.
| Variant | Tool | Parallel flag | Requires |
|---|---|---|---|
gzip-ng -p |
gzip-ng | -p |
|
pigz -p |
pigz | -p |
|
pigzpp -p |
pigzpp (-E zlib) |
-p |
|
bgzip -@ |
bgzip (htslib) | -@ |
|
migz |
MiGz (pool of cpus) | always | a JDK |
gzip-ng |
gzip-ng serial | ||
minigzip |
zlib-ng's minigzip | ||
gzip |
gzip |
Tools in tools/bin are preferred, then PATH, missing ones are skipped
and noted. GZIPNG, PIGZ, PIGZPP, GZIP, MINIGZIP, BGZIP, and
JAVA override the lookup. MiGz has no standalone binary, tools/fetch_migz.py
pulls the pinned jars from Maven Central once and compiles a small pipe
CLI, JVM start stays part of its measurements.
tools/build_tools.py builds gzip-ng, minigzip, pigz, pigzpp, and bgzip
against zlib-ng develop into tools/bin, so every zlib-linked tool
measures the same library. It writes tools/zlibng.json, and the graphs put that
zlib-ng version and commit in their footer. GNU gzip carries its own
deflate and MiGz uses the JVM's zlib, so those two stay system provided.
python3 tools/fetch_migz.py
python3 tools/build_tools.py
curl -LO https://mirror.circlestorm.org/silesia.tar
./bench.py silesia.tar -o results/all-tools.json
python3 scripts/graph_runs.py results/all-tools.jsonAny files passed are concatenated into one stream. With no input files
the benchmark instead generates a deterministic synthetic mix of
source-like text, build-log lines, and random blocks, weighted to land
near the 4.8 to 1 of the gzip-ng README corpus under gzip -6.
--levels, --threads, --runs, --size-mb, and --blocks-mb trim or
grow the matrix, ./bench.py --help lists the defaults. Everything needs
only the Python standard library.
python3 scripts/compare_runs.py before.json after.jsonRows match by benchmark name, so two runs of the same matrix, two machines, or two gzip-ng builds, compare directly with time and ratio deltas.
scripts/graph_runs.py turns a run into a speed versus ratio SVG, the
level ladders connected in order, a decompression throughput panel
including the cross-decode pairs, thread scaling line panels, deflate
block census panels, block counts and average block sizes across the
normal, rsyncable, and independent modes, repetition error bars, and
machine specs. An aggregate table prints to stdout.
All tools on the Silesia corpus (silesia.tar, 202 MiB), the common
ground of compression benchmarks, built and run as above: