LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 02-10-2009, 08:00 AM   #1
dlugasx
Member
 
Registered: Dec 2008
Location: Germany/Poland
Distribution: CentOS / Debian / Solaris / RedHat
Posts: 250

Rep: Reputation: 18
Backup server - compression with multicore support


Does any body know how can I zip huge amount of data using multicore support ?


I know that tar doesnt support multiprocessing.

Is it possible to handle this problem using bash script ?


When tar is working only one core is runnig. How can I use tar command that it will be use all cores ?
 
Old 02-10-2009, 09:26 AM   #2
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,156

Rep: Reputation: 247Reputation: 247Reputation: 247
Hi

Take a look at pbzip2:

http://compression.ca/pbzip2/

Combined with tar:

http://tldp.org/HOWTO/Bzip2-4.html
 
Old 02-10-2009, 11:15 AM   #3
jkzfixme
Member
 
Registered: Sep 2006
Location: Orange County, NY
Distribution: Ubuntu, Solaris, Android
Posts: 329
Blog Entries: 21

Rep: Reputation: 52
you can also use a parallel implementation of GZip, nicknamed PIGZ check it out here.

Regards
JKZfixme
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Using RSync to backup a secondary off-site backup server pezdspencer Linux - Software 4 06-29-2007 04:40 PM
client server data compression Ammad Linux - Networking 1 10-08-2006 04:48 PM
LXer: Movidis Launches Multicore MIPS-Debian Server LXer Syndicated Linux News 0 08-15-2006 02:21 PM
backup large data - good compression slackman Linux - General 12 04-28-2006 02:01 AM
Is there support for tape backup drives? Belkorin Fedora 0 07-09-2004 11:57 AM


All times are GMT -5. The time now is 12:23 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration