The speed at which files are compressed with gzip is currently almost always determined by the speed of the CPU. However, standard unix gzip is single- threaded and only uses a single CPU (core).
However, the maintainer of the zlib library has released 'pigz' or 'pig-zee' whichs adds just that: support for parallel compression. This dramatically improves the speed at which a file can be gzipped.
In this example, a 3 GB compressable file is gzipped:
Gzip:
root@Core7i:~# time gzip pigz.bin
real 1m58.994s
user 1m56.480s
sys 0m1.820s
Pigz:
root@Core7i:~# time pigz pigz.bin
real 0m31.524s
user 2m54.890s
sys 0m2.900s
This simple and a bit unscientific example shows a 400% speed improvement. Since the Core i7 has four real cores, this shows that pigz scales nicely.
Comments