[SOLVED] large tar nightly + rsync to compare question
Linux - ServerThis forum is for the discussion of Linux Software used in a server related context.
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Introduction to Linux - A Hands on Guide
This guide was created as an overview of the Linux Operating System, geared toward new users as an exploration tour and getting started guide, with exercises at the end of each chapter.
For more advanced trainees it can be a desktop reference, and a collection of the base knowledge needed to proceed with system and network administration. This book contains many real life examples derived from the author's experience as a Linux system and network administrator, trainer and consultant. They hope these examples will help you to get a better understanding of the Linux system and that you feel encouraged to try out things on your own.
Click Here to receive this Complete Guide absolutely free.
I have a few sites that need to be backed up and moved off.
The current solution was a HUGE tar nightly, gzipped, then scp'd over. There is 2 much data to keep things unzipped, as I wanted to just rsync the data.
But, rather than zipping 50+GB nightly, is there a way to tar things, rsync compare them to the current tar, and just rsync the changes inside the tar file?
I'm sure there is a better way to google/ask the question, but I am posting while reading and trying to figure the best way to ask/approach. I simply don't want to copy all that data nightly if 95% is the same.
If you only want to synchronize the changes you could use rsync, but I've had some bad experiences with that. When looking for another solution I came across Unison File Synchronizer. I'm sure it'll be of use.
Thanks, but I didn't want to compare the unzipped storage due to storage limitations. The remote server can't store ALL the data from ALL servers uncompressed. So nightly they backup via tar and copy it over. So rather than taking 50gb, tarring and gzipping nightly to copy over, I wanted to try and tar on the host server, then via rsync or something compare the tarred file and only update the changed file (still inside the tar) but I am not sure if it's possible.
I will chk the provided link, but the key is both the remote backup machine will only have a tar / gzipped copy of the data so the compare has to go into that file to compare and I am not sure if it's possible.
How about 'tarring' your directory(ies)/file(s) directly on the destination server over an ssh tunnel and using the --update or --incremental/--listed-incremental option of tar? Have a look at the following sites, they clarify it and I believe that with a little trial and error you'll be able to avoid having to tar and copy over a big file and instead tar directly on the destination server.