LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-18-2014, 04:22 PM   #1
bluforce
Member
 
Registered: Apr 2012
Posts: 39

Rep: Reputation: Disabled
backup script assistance


Guys,
Trying to create a backup script on 2 machines to back up to an array, which is backed up nightly. The 2 workstation machines are running 32bit RHEL and the version of Netbackup we use here doesn't support 32bit.

So, we had the idea of creating a tarball of the directories my guys need backed up, then setup a cron job to scp those to a directory on a 64bit RHEL server, which is backed up.

What would be the best way to set this up?
I reckon we would need to delete the tarball after it has been scp'ed to the server (for disk space reasons).

So far, my script looks like this:

Code:
TIME=`date +%b-%d-%y`
FILENAME=backup-$TIME.tar.gz
SRCDIR="/build /var/spool/cron /etc/cron*"
DESDIR=/build/backups
tar -cpvz --exclude-from='exclude_list.txt' -f $DESDIR/$FILENAME $SRCDIR
Can you guys offer assistance on how to approach this after the tarball is created? Thanks!

Last edited by bluforce; 12-18-2014 at 04:23 PM.
 
Old 12-18-2014, 04:27 PM   #2
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
scp with passwordless ssh keys set up?

Or you could skip the tarball entirely and just rsync the directories over.

Or you could mount the backup location on the server via NFS and just dump the tarball straight to its final location without having to create an intermediate copy on the local machine.


The problem with tarballs is that you have to redo the ENTIRE backup process EVERY time. With something like rsync you can do incremental backups, so the first one takes forever but all future ones run very quickly. You can even use the --link-dest flag in rsync to make hard links to unchanged files so you can have full, separate backups that only use the space of the files that changed. Plus with rsync your backups are live and navigable, which makes them much more convenient to use.

Last edited by suicidaleggroll; 12-18-2014 at 04:32 PM.
 
Old 12-18-2014, 04:32 PM   #3
bluforce
Member
 
Registered: Apr 2012
Posts: 39

Original Poster
Rep: Reputation: Disabled
Haven't set up keys yet.
Do you think rsync would work better in my situation since disk space would likely be an issue on down the road? The tarball would be ~275gb on a 1TB drive. 275GB is already in use on this drive.
 
Old 12-18-2014, 04:35 PM   #4
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
If you're rsyncing the files straight to the server, then no additional space will be used on the local machine. You only need to worry about available space on the machine when you want to make a giant local tarball (275 GB is insanely unmanageable BTW) and then copy it over, which is a terribly inefficient way of doing things.
 
Old 12-19-2014, 09:43 PM   #5
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
Quote:
Originally Posted by suicidaleggroll View Post
scp with passwordless ssh keys set up?

Or you could skip the tarball entirely and just rsync the directories over.

Or you could mount the backup location on the server via NFS and just dump the tarball straight to its final location without having to create an intermediate copy on the local machine.


The problem with tarballs is that you have to redo the ENTIRE backup process EVERY time.
No - tar supports incremental backups as well. See the -g option in the tar man page.
 
1 members found this post helpful.
Old 12-19-2014, 09:46 PM   #6
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
Quote:
Originally Posted by jpollard View Post
No - tar supports incremental backups as well. See the -g option in the tar man page.
Good to know, thanks.
 
Old 12-19-2014, 09:51 PM   #7
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
I'll add a little more about tar...

It is POSSIBLE to use tar to create a multi-file archive. You have to use the -L option to specify how long you want the file, then use the -F option to run a script at the end of each file. This should allow you to rename the current output, and resume with a new output file being created.

I will admit to having tried this... But if you fear large tarballs this is one way to break them up. Another is to use the split utility (like "tar cxf - . | split ...") which will also work.
 
  


Reply

Tags
backup, scp, script, tar


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Newb to Linux. Need assistance with backup server share bgorgtech Linux - Newbie 2 04-23-2012 05:22 PM
Need assistance on shell script gunturian Linux - General 3 01-08-2010 02:15 PM
HDDTemp : Script Assistance orbit Slackware 5 10-29-2009 03:17 PM
Newbie trying to write a simple backup script to backup a single folder Nd for school stryker759a Linux - Newbie 2 09-16-2009 09:52 AM
need assistance with this script rsmccain Linux - General 2 02-08-2007 11:41 PM


All times are GMT -5. The time now is 06:18 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration