LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 12-06-2007, 02:54 PM   #1
300zxkyle
LQ Newbie
 
Registered: Jun 2007
Distribution: OpenSuSE 10.2
Posts: 13

Rep: Reputation: 0
Question tar + split question


In a previous thread I asked how to create a tar archive that would allow me to limit the file size and break up automatically into incremental archives.

http://www.linuxquestions.org/questi...volume-603509/

I decided to use the split option as it is meant for this and the -M flag for tar really isn't.

Code:
tar -cvpj 'directory/name'/ | split -d -b 4000m - /name/of/archive.tar.bz2.
which produces something like...

Code:
archive.tar.bz2.00
archive.tar.bz2.01
archive.tar.bz2.02
My question is this. How with the existing file size limit can I extract the contents of one of these archives back to the file system. I can't reassemble the files onto the file system because they will break the 4GB limit. Is there a way that I can pipe cat through tar? Are their other alternatives still?

Any help is appreciated.
 
Old 12-06-2007, 03:15 PM   #2
kilgoretrout
Senior Member
 
Registered: Oct 2003
Posts: 2,310

Rep: Reputation: 139Reputation: 139
I don't think that's possible. Once you use split you have to join the files back together with cat before you can do anything else.
The real solution is to stop taring up your files to a FAT32 filesystem with its 4GB single file size limit. You shouldn't have this problem on any native linux filesystem.
 
Old 12-06-2007, 03:39 PM   #3
David1357
Senior Member
 
Registered: Aug 2007
Location: South Carolina, U.S.A.
Distribution: Ubuntu, Fedora Core, Red Hat, SUSE, Gentoo, DSL, coLinux, uClinux
Posts: 1,302
Blog Entries: 1

Rep: Reputation: 107Reputation: 107
Quote:
Originally Posted by 300zxkyle View Post
My question is this. How with the existing file size limit can I extract the contents of one of these archives back to the file system. I can't reassemble the files onto the file system because they will break the 4GB limit. Is there a way that I can pipe cat through tar? Are their other alternatives still?
Try

# cat *.bz2* | (cd path-to-destination; tar jxv)
That should send a byte-stream through stdin to tar in the destination directory of your choice.
 
Old 12-06-2007, 03:48 PM   #4
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
dar allows you to split archive files and retrieve files from the archive without having to assemble them into one huge archive file. It will even pause while you eject media and mount the next disk in a series.

Dar is a good replacement for tar if you are making archives for storing on removable media, although I think it's not suitable for tape backups (which is tar's job).
 
Old 12-07-2007, 07:52 AM   #5
300zxkyle
LQ Newbie
 
Registered: Jun 2007
Distribution: OpenSuSE 10.2
Posts: 13

Original Poster
Rep: Reputation: 0
Thumbs up worked like a charm.

David,

The solution you provided worked like a charm. Thanks everyone for the input.
 
Old 02-01-2011, 08:56 AM   #6
thiyagi
LQ Newbie
 
Registered: Dec 2010
Posts: 4

Rep: Reputation: 0
thank you guys, nice post..
 
Old 02-01-2011, 09:18 AM   #7
i92guboj
Gentoo support team
 
Registered: May 2008
Location: Lucena, Córdoba (Spain)
Distribution: Gentoo
Posts: 4,040

Rep: Reputation: 373Reputation: 373Reputation: 373Reputation: 373
In the future, you might want to check 7z, which has been ported to a lot of architectures and can split the archives in volumes without needing external tools. With the default options it also provides a better compression ration and a lesser cpu usage, at least that's my perception (I am not really a fan of benchmarks).
 
Old 11-22-2011, 04:47 AM   #8
scramble64
LQ Newbie
 
Registered: Nov 2011
Posts: 3

Rep: Reputation: Disabled
Thanks a lot for the tips!

I recently used the above command to backup about 1TB of data into an external HD that was formatted as FAT32 and created a bz2 tar archive and then split it in 2GB chunks. Now that I am trying to access this data I am realizing that the numbering of the chunks goes from 00 to 99, which got me thinking that actually not all of the original data may have been backed up, given the suffix for the split files can contain only 2 digits.

Is there a way to change the format of the suffix to contain more than two digits?

Many thanks for your help!
 
Old 01-14-2012, 12:39 AM   #9
kssuhesh
LQ Newbie
 
Registered: Aug 2007
Location: Kochi
Distribution: Mandriva
Posts: 23

Rep: Reputation: 15
Hello,

It is a good tip for us to backup files.
 
Old 01-14-2012, 06:19 AM   #10
michaelk
Moderator
 
Registered: Aug 2002
Posts: 11,909

Rep: Reputation: 747Reputation: 747Reputation: 747Reputation: 747Reputation: 747Reputation: 747Reputation: 747
scramble64
You might have already figured this out but AFAIK 100 files is the limit with digits. If the number suffixes was exceeded an error would of been displayed. You can use letters of length N using the -a option.

---------- Post added 01-14-12 at 01:20 PM ----------

scramble64
You might have already figured this out but AFAIK 100 files is the limit with digits. If the number suffixes was exceeded an error would of been displayed. You can use letters of length N using the -a option.
 
  


Reply

Tags
backup, tar


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
a tough question 4 u, problem in extracting tar & tar.gz files p_garg Linux - General 5 11-08-2010 11:02 AM
tar, split, burn to dvd result to tar file corruption fedusr Fedora 8 11-30-2007 04:39 PM
split or tar -M -L? Slacker0815 Linux - Software 3 06-05-2006 04:23 PM
tar problems, trying to pipe through split microsoft/linux Linux - Software 13 03-04-2006 08:20 PM
is it possible to get information from one bad split tar? eantoranz Linux - Software 1 03-30-2005 02:02 PM


All times are GMT -5. The time now is 11:31 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration