Share your knowledge at the LQ Wiki.
Go Back > Forums > Linux Forums > Linux - Server
User Name
Linux - Server This forum is for the discussion of Linux Software used in a server related context.


  Search this Thread
Old 02-10-2009, 07:00 AM   #1
Registered: Dec 2008
Location: Germany/Poland
Distribution: CentOS / Debian / Solaris / RedHat
Posts: 266

Rep: Reputation: 18
Backup server - compression with multicore support

Does any body know how can I zip huge amount of data using multicore support ?

I know that tar doesnt support multiprocessing.

Is it possible to handle this problem using bash script ?

When tar is working only one core is runnig. How can I use tar command that it will be use all cores ?
Old 02-10-2009, 08:26 AM   #2
Senior Member
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,257

Rep: Reputation: 296Reputation: 296Reputation: 296

Take a look at pbzip2:

Combined with tar:
Old 02-10-2009, 10:15 AM   #3
Registered: Sep 2006
Location: Orange County, NY
Distribution: Ubuntu, Solaris, Android
Posts: 329
Blog Entries: 21

Rep: Reputation: 52
you can also use a parallel implementation of GZip, nicknamed PIGZ check it out here.



Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Using RSync to backup a secondary off-site backup server pezdspencer Linux - Software 4 06-29-2007 03:40 PM
client server data compression Ammad Linux - Networking 1 10-08-2006 03:48 PM
LXer: Movidis Launches Multicore MIPS-Debian Server LXer Syndicated Linux News 0 08-15-2006 01:21 PM
backup large data - good compression slackman Linux - General 12 04-28-2006 01:01 AM
Is there support for tape backup drives? Belkorin Fedora 0 07-09-2004 10:57 AM

All times are GMT -5. The time now is 05:54 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration