LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 10-01-2009, 12:47 PM   #1
hawk__0
Member
 
Registered: Nov 2008
Posts: 105

Rep: Reputation: 15
Compression/archive tool that utilizes mutliple cores?


I was wondering if there is any software that will compress (preferrably) or archive files, but one that will utilize multiple cores?

Tar uses 1 core per process (yes I know if I spawn 3 tars, I'll use 3 of my 4 cores, but that's not my point).
Rar seems to use 1 core as well.

Help?
 
Old 10-01-2009, 03:58 PM   #2
Oakems
Member
 
Registered: Nov 2008
Posts: 43

Rep: Reputation: 16
I'd say the best way of going about it would be to create a script. You have all the tools you need ready, so it seems needless to download more software. It wouldn't be hard to write a script that would tar and compress files into tar.gz format while utilizing multiple cores.
 
Old 10-01-2009, 07:24 PM   #3
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,873

Rep: Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600
I think tar (archive) and gzip/bzip2/compress (compression) work on streams (typical *nix approach), so they can't multi-process/thread; it wouldn't make sense.
If you are desperate to try and speed things up, you could gzip/bzip2 multiple files separately & simultaneously (thus using multi-cores), then tar up the gzipped files (1 core).
 
Old 10-01-2009, 11:13 PM   #4
manwithaplan
Member
 
Registered: Nov 2008
Location: ~/
Distribution: Arch || Sidux
Posts: 393

Rep: Reputation: 45
I have had great true multi-threaded support with 7zip... Its parallel, and utilizes all cores, and is fast. I dont have any benchmark results, but there has been plenty of tests. Check Phoronix's site for any conclusive tests.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Java Archive Tool Woodsman Slackware 5 07-20-2008 11:31 PM
default KDE compression tool has some problems with ntfs kaiwan Linux - Software 1 07-13-2008 04:02 AM
LXer: Take advantage of multiple CPU cores during file compression LXer Syndicated Linux News 0 02-13-2008 08:40 PM
Difference between Archive and Compression ? tofee Linux - Newbie 1 04-01-2006 01:45 AM
Apache problem with displaying compression/archive files. Akujin Linux - Networking 4 03-09-2005 02:18 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 10:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration