LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 07-29-2009, 11:08 AM   #1
Z505
LQ Newbie
 
Registered: Mar 2005
Distribution: Custom Debian and Slack
Posts: 16

Rep: Reputation: 0
Unpack resume feature, rar, zip, gz, tar, 7z


Is there any command line unpacker with a resume feature? I am having problems with unpacking some large multi gigabyte files on a shared web server. The server seems to kill the command after a certain amount of time because they monitor CPU usage or have some detection in place. It would be great if I could do a certain amount of unpacking, and then wait for a while, and then do more unpacking.

An example of "resume" feature is FTP servers. When you upload files to FTP, often there is a resume feature in case the connection breaks. What I need is a little different: a simple command line unpacker that allows resuming.

Any unpacker is fine. Rar, gz, tar, 7zip, zip, etc. I just need one with resume!?


Since my problem is CPU usage for a certain amount of time, the last alternative, is to recompile the packer and insert some nanosleep() or sleep() or select() calls in the correct loops to reduce cpu. I would hate to have to do this if there was an easier alternative. Or, as another alternative, I could build a resume feature into some existing unpacker - but I don't want to code that sh*t if I don't have to.

By the way, I have tried using the NICE command to reduce cpu usage, but the process still takes up over 30 percent cpu, so the webserver still doesn't like it for extended periods of time, and still cuts the unpacking process off. I tried cron jobs, and executing it from my web browser through a nifty cgi program. The same thing happens: server cuts off the process after a while before it finishes unpacking.

Note: I will not buy a dedicated server for this small project (so please don't suggest that, I considered it ;-))

Last edited by Z505; 07-29-2009 at 11:27 AM.
 
Old 07-30-2009, 09:11 AM   #2
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,323

Rep: Reputation: 353Reputation: 353Reputation: 353Reputation: 353
Hi

I've never seen any "resume" on unpacking except by unpacking a little by little. (You can usually specify some files to unpack so that not everything is unpacked at once.)

So one solution would be to write a script listing the contents of the file, and then unpacking it file by file. But I guess it won't be easy if some of files are very big.

Another way would be to pipe the output of the unpacker into a very slow reader script. For example:

while read line ; do
echo $line
sleep 10
done

Yet another way would be to not use compression at all - maybe simply tar?. Then it will use a lot more IO time, both for the file transfers and the disk access, but that's maybe what the admins of the server want?
 
Old 07-30-2009, 09:54 AM   #3
nowonmai
Member
 
Registered: Jun 2003
Posts: 481

Rep: Reputation: 48
Because of how pretty much all compressor programs work (variations on Lempel-Ziv) a file can only be decompressed from the top down. Decompression dictionary information is build on the fly by the decompressor as it decompresses, so it would have to start again from the top and rebuild the dictionary every time.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Delete rar files after unpack? Snouser Linux - Software 1 08-06-2008 07:15 PM
Make tar,zip,rar play nice with named pipes? Pi Man Programming 8 10-22-2007 04:14 PM
can't unpack rar files jimjamjahaa Ubuntu 3 01-25-2006 09:41 AM
how to unpack .tar? g-string 3 Debian 8 11-29-2004 08:01 AM
what do i do after i unpack the tar.gz sanjaya Linux - Software 1 02-11-2004 07:58 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 06:50 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration