Comparing experience: xz and gzip

Sum1 · 08-16-2010, 11:11 AM

I've been testing xz compression because I've read many anecdotal reports of improved compression over gzip. I'm careful to maintain complete data backup for all my user data and have, at this time of writing, a 29 Gig. data-set.

I have the following results:

gzip -9 on a 29 gig. .tar file results in a .tar.gz 31% - 32% smaller than the original .tar file.

xz -9 on a 29 gig. .tar file results in a .tar.xz 36% - 38% smaller than the original .tar file.

My 29 gig. data-set contains 717,000 files of widely varying sizes.
All data is stored on ext4 filesystems using kernel 2.6.29.6.

The surprising discovery was the time it took to complete the job using xz. Gzip consistently finished in just under 1 hour -- approx. 59 minutes; however, xz required 4hr38min. - 4hr42min. to complete the same.

For me, the 5%-6% increased compression is definitely not worth the additional 3.5 hr. - 3.75 hr. process time to completion.

I'm interested to hear others' results.

John VV · 08-16-2010, 07:48 PM

just for comparison what dose bz2 & tar.bz2 show for file size and times ?

Sum1 · 08-17-2010, 11:39 AM

Quote:

Originally Posted by John VV

just for comparison what dose bz2 & tar.bz2 show for file size and times ?

test process commenced; will report back upon completion.

Sum1 · 08-17-2010, 01:26 PM

# time bzip2 -9 -v test.tar
test.tar:

1.496:1, 5.348 bits/byte, 33.15% saved, 29319710720 in, 19598812379 out.

real 124m28.091s
user 120m23.832s
sys 0m37.292s

- - - - - - - - -

Very odd, bzip2 takes twice the time as gzip and only produces a 1.15% compression increase over gzip. Also, xz requires more than twice the time of bzip2 and only produces a 3%-5% compression increase over bzip2.

I don't know if there is anything unusual about my data files skewing the results in favor of gzip.

For my needs and environment, xz and bzip2 provide no compelling benefit. I'll continue to use gzip -9.

John VV · 08-17-2010, 03:36 PM

not to odd bz2 normally dose take longer but the compression is normally better
but not on photos , except for RAW files .

p7z ( the fileroller plugin)_ for 7-zip is normally fairly fast -- but i don't use it to much
my MS only friends just say " wtf- windows can not open it" ( default windows )
and are clueless as how to install 7-zip - and it is FREE