Share your knowledge at the LQ Wiki.
Go Back > Forums > Linux Forums > Linux - Software
User Name
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.


  Search this Thread
Old 10-01-2009, 01:47 PM   #1
Registered: Nov 2008
Posts: 105

Rep: Reputation: 15
Compression/archive tool that utilizes mutliple cores?

I was wondering if there is any software that will compress (preferrably) or archive files, but one that will utilize multiple cores?

Tar uses 1 core per process (yes I know if I spawn 3 tars, I'll use 3 of my 4 cores, but that's not my point).
Rar seems to use 1 core as well.

Old 10-01-2009, 04:58 PM   #2
Registered: Nov 2008
Posts: 43

Rep: Reputation: 16
I'd say the best way of going about it would be to create a script. You have all the tools you need ready, so it seems needless to download more software. It wouldn't be hard to write a script that would tar and compress files into tar.gz format while utilizing multiple cores.
Old 10-01-2009, 08:24 PM   #3
LQ Guru
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 18,120

Rep: Reputation: 2669Reputation: 2669Reputation: 2669Reputation: 2669Reputation: 2669Reputation: 2669Reputation: 2669Reputation: 2669Reputation: 2669Reputation: 2669Reputation: 2669
I think tar (archive) and gzip/bzip2/compress (compression) work on streams (typical *nix approach), so they can't multi-process/thread; it wouldn't make sense.
If you are desperate to try and speed things up, you could gzip/bzip2 multiple files separately & simultaneously (thus using multi-cores), then tar up the gzipped files (1 core).
Old 10-02-2009, 12:13 AM   #4
Registered: Nov 2008
Location: ~/
Distribution: Arch || Sidux
Posts: 393

Rep: Reputation: 45
I have had great true multi-threaded support with 7zip... Its parallel, and utilizes all cores, and is fast. I dont have any benchmark results, but there has been plenty of tests. Check Phoronix's site for any conclusive tests.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Java Archive Tool Woodsman Slackware 5 07-21-2008 12:31 AM
default KDE compression tool has some problems with ntfs kaiwan Linux - Software 1 07-13-2008 05:02 AM
LXer: Take advantage of multiple CPU cores during file compression LXer Syndicated Linux News 0 02-13-2008 09:40 PM
Difference between Archive and Compression ? tofee Linux - Newbie 1 04-01-2006 02:45 AM
Apache problem with displaying compression/archive files. Akujin Linux - Networking 4 03-09-2005 03:18 AM > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 12:52 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration