TAR

dlugasx · 12-12-2008, 03:08 AM

Hi everyone,

is there any body here which can tell me how can I setup TAR to use all processor cores during zip process ?

All the time is working only one. On the server is fast RAID and it looks like the processor making bottle neck effect for compress.

dlugasx

colucix · 12-12-2008, 03:26 AM

Tar can not work in parallel on multiple cores, but there are two compression utilities which provide this functionality: pbzip2 and mgzip. They are compatible with their "serial" counterparts, for decompression. So you can create a normal archive and then compress it using one of these utilities. Here is an example of performance on a machine with 2 XEON QUAD CORE CPUs (8 cores):

Code:

$ time pbzip2 file.nc

real    0m19.708s
user    2m31.685s
sys     0m1.399s
$
$ time bzip2 file.nc

real    1m57.925s
user    1m56.934s
sys     0m0.762s

dlugasx · 12-12-2008, 03:33 AM

Quote:

Originally Posted by colucix

Tar can not work in parallel on multiple cores, but there are two compression utilities which provide this functionality: pbzip2 and mgzip. They are compatible with their "serial" counterparts, for decompression. So you can create a normal archive and then compress it using one of these utilities. Here is an example of performance on a machine with 2 XEON QUAD CORE CPUs (8 cores):

Code:

$ time pbzip2 file.nc

real    0m19.708s
user    2m31.685s
sys     0m1.399s
$
$ time bzip2 file.nc

real    1m57.925s
user    1m56.934s
sys     0m0.762s

nice Trick

I tested this and its working realy fast

. I must implement this with pipe to the backup scripts.

BIG BEER for Colucix

tahnks man