tar + split question

300zxkyle · 12-06-2007, 02:54 PM

In a previous thread I asked how to create a tar archive that would allow me to limit the file size and break up automatically into incremental archives.

http://www.linuxquestions.org/questi...volume-603509/

I decided to use the split option as it is meant for this and the -M flag for tar really isn't.

Code:

tar -cvpj 'directory/name'/ | split -d -b 4000m - /name/of/archive.tar.bz2.

which produces something like...

Code:

archive.tar.bz2.00
archive.tar.bz2.01
archive.tar.bz2.02

My question is this. How with the existing file size limit can I extract the contents of one of these archives back to the file system. I can't reassemble the files onto the file system because they will break the 4GB limit. Is there a way that I can pipe cat through tar? Are their other alternatives still?

Any help is appreciated.

kilgoretrout · 12-06-2007, 03:15 PM

I don't think that's possible. Once you use split you have to join the files back together with cat before you can do anything else.
The real solution is to stop taring up your files to a FAT32 filesystem with its 4GB single file size limit. You shouldn't have this problem on any native linux filesystem.

David1357 · 12-06-2007, 03:39 PM

Quote:

Originally Posted by 300zxkyle

My question is this. How with the existing file size limit can I extract the contents of one of these archives back to the file system. I can't reassemble the files onto the file system because they will break the 4GB limit. Is there a way that I can pipe cat through tar? Are their other alternatives still?

Try

# cat *.bz2* | (cd path-to-destination; tar jxv)

That should send a byte-stream through stdin to tar in the destination directory of your choice.

matthewg42 · 12-06-2007, 03:48 PM

dar allows you to split archive files and retrieve files from the archive without having to assemble them into one huge archive file. It will even pause while you eject media and mount the next disk in a series.

Dar is a good replacement for tar if you are making archives for storing on removable media, although I think it's not suitable for tape backups (which is tar's job).

300zxkyle · 12-07-2007, 07:52 AM

David,

The solution you provided worked like a charm. Thanks everyone for the input.

thiyagi · 02-01-2011, 08:56 AM

thank you guys, nice post..

i92guboj · 02-01-2011, 09:18 AM

In the future, you might want to check 7z, which has been ported to a lot of architectures and can split the archives in volumes without needing external tools. With the default options it also provides a better compression ration and a lesser cpu usage, at least that's my perception (I am not really a fan of benchmarks).

scramble64 · 11-22-2011, 04:47 AM

Thanks a lot for the tips!

I recently used the above command to backup about 1TB of data into an external HD that was formatted as FAT32 and created a bz2 tar archive and then split it in 2GB chunks. Now that I am trying to access this data I am realizing that the numbering of the chunks goes from 00 to 99, which got me thinking that actually not all of the original data may have been backed up, given the suffix for the split files can contain only 2 digits.

Is there a way to change the format of the suffix to contain more than two digits?

Many thanks for your help!

kssuhesh · 01-14-2012, 12:39 AM

Hello,

It is a good tip for us to backup files.

michaelk · 01-14-2012, 06:19 AM

scramble64
You might have already figured this out but AFAIK 100 files is the limit with digits. If the number suffixes was exceeded an error would of been displayed. You can use letters of length N using the -a option.

---------- Post added 01-14-12 at 01:20 PM ----------

scramble64
You might have already figured this out but AFAIK 100 files is the limit with digits. If the number suffixes was exceeded an error would of been displayed. You can use letters of length N using the -a option.