Using (s)tar for back-up to hard disk creating very large files
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Distribution: Debian /Jessie/Stretch/Sid, Linux Mint DE
Posts: 5,195
Rep:
Using (s)tar for back-up to hard disk creating very large files
Hello all,
This week for the umpteenth time my tape streamer broke down. This time it was a Quantum 40 GB DLT. The tape was ruined, and I cannot make backups anymore on our company network.
In the past 10 years, I think I have gone thru 5 tape (professional) tape streamers. Not only does it take a lot of time and hassle to get another tape streamer running, the most important is that all data recovery depends on something flimsy and unreliable as a tape streamer.
I might consider to buy another streamer, but as a second resort (and a temporary solution) I want to put up a Linux machine with some large hard disks and perform backups of my primary file server over the network.
What I would do is implement a multi-level dump strategy, with one regular full backup, and several incremental backups after that, until the next full backup.
I think in that way I have covered off-line storage, and a strategy which let me recover damaged data for a week or two.
I plan to use the 'star' program for that purpose. 'Star' is a 'tar'-like program, but functionality and performance is improved. I know how to use it.
The question is: when I create the backup the first time, I would have to create and write a 40 GB tar file over the network to the backup box. Will that be possible, or does 'tar' choke in such sizes? I do this already at home with 10 GB disks, but then to tape. Is there any fundamental difference in writing to a hard disk as compared to writing to tape?
BTW, the primary file server is a Windoze box (I know, shame on me), and I would have to set up the Linux box as a Samba client. Backups will be initiated from the Linux box.
Re: Using (s)tar for back-up to hard disk creating very large files
Quote:
Originally posted by jlinkels The question is: when I create the backup the first time, I would have to create and write a 40 GB tar file over the network to the backup box. Will that be possible, or does 'tar' choke in such sizes?
tar, was originally created for Tape-ARchiving. I seriously doubt you'll have any problems.
Distribution: Debian /Jessie/Stretch/Sid, Linux Mint DE
Posts: 5,195
Original Poster
Rep:
Just to finish this matter.
This system now has been running ever since January 2005. I make a full backup every two weeks, alternatingly on two hard disks. Additionally, I make a incremental every day and write it to two alternating files on a third disk.
It is not a pefect back-up system, I can only restore from 1 or 2 days before, and then from 2 weeks or 4 weeks before.
In the mean time I have gotten a new tape streamer and use that one for weekly backups.
The system with the disks works very reliably and really is install-and-forget. For quite a number of times I have saved the day of one of our users after he had incidentally overwritten his file, or MS decided to write an empty file to disk instead of the full one.
Restore is easy, just enter the command to restore the file (which is a complicated command, but it still fits on one line ), forget, and some time later the file is back. It might take up to 2 hours for star to find the file from a 40 GB backup on a 1 GHz machine, that is the only drawback. The operation is very reliable however. My script writes a log file with all file names to disk, so it is easy to pick the correct file name. I use this more often than restore from tape.
Just wanted to let you know this, although it is trivial.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.