Use pigz for ultra fast, parallel gzipping

gzip can be slow. Painfully slow. If you have a server with a bunch of processors then you can use pigz (pig-zee, or I guess pig-zed if you are Canadian) to utilize those processors.

Installation is easy: aptitude install pigz

Usage is also easy. For example, to tar and gzip /home/jacob using 3 processors, you could use this command: tar cf – -C / /home/jacob | pigz -p 3 > backup.tar.gz

I’m using this for some of my backup tasks and have dramatically cut down the time it takes for my backups to run. I’ve also used this a few times for bundling files up to move between servers.

Featured image creative commons licensed ( BY ) flickr photo shared by MIKI Yoshihito (´・ω・)

Leave a Reply

Your email address will not be published. Required fields are marked *