Louwrentius

The speed at which files are compressed with gzip is currently almost always determined by the speed of the CPU. However, standard unix gzip is single- threaded and only uses a single CPU (core).

However, the maintainer of the zlib library has released 'pigz' or 'pig-zee' whichs adds just that: support for parallel compression. This dramatically improves the speed at which a file can be gzipped.

In this example, a 3 GB compressable file is gzipped:

Gzip:

root@Core7i:~# time gzip pigz.bin

real 1m58.994s

user 1m56.480s

sys 0m1.820s

Pigz:

root@Core7i:~# time pigz pigz.bin

real 0m31.524s

user 2m54.890s

sys 0m2.900s

This simple and a bit unscientific example shows a 400% speed improvement. Since the Core i7 has four real cores, this shows that pigz scales nicely.

There is a nifty little program called 'iotop'. Iotop is part of Debian or Ubuntu and can be installed with a simple apt-get.

Once you have determined with 'top' that the system is waiting on IO-access, It is nice to know

which process is responsibe for this IO. Therefore, you want a list of processes, just like top provides, but instead of CPU or RAM usage, it shows the IO a process is generating. This is exactly what 'iotop' provides.

This is ideal when troubleshooting performance problems caused by heavy IO.

http://guichaz.free.fr/iotop/