Welcome to the most active Linux Forum on the web.
Go Back > Forums > Linux Forums > Linux - Software
User Name
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.


  Search this Thread
Old 09-11-2008, 09:44 PM   #1
Registered: Sep 2008
Posts: 48

Rep: Reputation: 15
Segmenting archives into a variety of smaller archives.

Hey, can one cut an archive into a variety of smaller archives of x size?
Certainly there are many applications that do that, I was just wondering of there was anything that came with basic linux, or, failing that, what is a recommended full-featured, well docced, command-line based app to do just that?
Old 09-11-2008, 09:56 PM   #2
Gentoo support team
Registered: May 2008
Location: Lucena, Córdoba (Spain)
Distribution: Gentoo
Posts: 4,083

Rep: Reputation: 404Reputation: 404Reputation: 404Reputation: 404Reputation: 404
The command line "split" comes with coreutils, so it should be present in most linux, even in a minimal installation or livecds.

It's easy enough to use, and all the info is in "man split".

To join the pieces you can use cat:

cat file1 file2 file3 > big_file
It really can't be any simpler than that.
Old 09-11-2008, 11:41 PM   #3
Registered: Sep 2008
Posts: 48

Original Poster
Rep: Reputation: 15
Thanks. I did a little research before posting but was going at it from the wrong angle it seems.
Old 09-12-2008, 01:06 AM   #4
LQ Guru
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 680Reputation: 680Reputation: 680Reputation: 680Reputation: 680Reputation: 680
I once backed up my home directory before performing a fresh install on my laptop. The external drive available was formatted in FAT32, and the tar archive would be larger than the 2 or 4 GB limit of the filesystem. I used spit.

To restore you don't need to reassemble the parts. You can pipe the output of the pipe command into your tar command.
cat backup* | tar -C /dest/dir/ -xf -


An option of tar allows you to create a multivolume archive at the start. You restore from the first volume and it will prompt you for the second. I don't remember the full details. I think you can use the -f option multiple times for each volume on the command line. If it is a local file and not a tape device, then either you won't be prompted or you can supply a default prompt with < <(yes y) at the end of the command. Please check the info tar manual to make sure.

If you are creating a backup on a local hard drive in order to backup each slice of the backup to a DVD, it can be tricky restoring from DVD's if you can't reassemble the original archive. This can happen if you don't have an external drive, or a spare partition to re-assemble the archive on. The way I've found that works is to create an fifo file for each disc. Then in one terminal cat the file in that dvd into the corresponding fifo. In another shell run a loop cat'ting from each fifo. This way, coming to the end of the file doesn't cause the pipe to be broken.

So for 12 DVDs
Shell 1:
mkfifo fifo-{1..12}
#for each respective disc insert and mount the dvd and cat into the corresponding fifo file interactively
cat /media/dvd/backup_part-1 >fifo-1
cat /media/dvd/backup_part-2 >fifo-2

Shell 2:
for part in fifo-{1..12}; do
cat $part
done | tar -C /dest/dir/ -f -


Forgive me if I went off on a tangent. Given the size of hard disks these days relative to DVDs and the fact that you aren't keeping the archive intact, you might be backing up the slices to a CD or DVD to free up storage.


I'm operating from memory here, from when I tested out the multiple fifo technique on slices of a dd created image. If you may not have the drive space available to reassemble the original, I'd recommend practicing on a smaller archive and slices to perform a practice dry run. Although trying out the multivolume option may be a better idea. Each tar volume is an independent archive unless a large file needed to be split between volumes. So if you can't find disc 3 but you know a program is on disc 4, you can restore from disc 4 directly.


For DVD backups I like the dar command. I cheat by using kdar to set up the backup & restore jobs and then export the jobs as a bash script. If you have dar and libdar installed on a live distro or rescue disk, you can use it and a copy of the bash restore script in an emergency where you con't have X!! access.

Last edited by jschiwal; 09-12-2008 at 01:08 AM.
Old 09-25-2008, 05:47 AM   #5
Registered: Sep 2008
Posts: 48

Original Poster
Rep: Reputation: 15
Thanks for the edifying post, jschiwal.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
how can I create .Z archives zbenjudah Linux - General 1 05-14-2007 04:39 PM
Verifying Archives onelung02 Linux - Software 1 06-26-2006 12:23 PM
Where are the powerswitch archives? yeri63 Linux - General 2 07-08-2003 10:03 AM
Upacking archives Stephanie Linux - General 2 05-23-2003 05:03 PM
archives nautilus_1987 Slackware 4 09-02-2002 04:01 PM > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 03:57 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration