LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 05-30-2009, 12:34 PM   #1
BassKozz
Member
 
Registered: Dec 2008
Distribution: Ubuntu 8.10
Posts: 35

Rep: Reputation: 15
Need a fresh pair of eyes to review my backup+restore scripts


I've created a Backup script and a Restore script to backup my websites on a shared host (dreamhost) that I have SSH access to. The scripts should backup/restore my main website [domain.com], my subdomain [store.domain.com] of the website (which is a Magento Store), and lastly my Magento DB.

Backup Script:
Code:
#!/bin/sh
###
# 5/19/09
# backup.domain.com.sh
# Backup /domain.com & /store.domain.com & mysql.domain.com
###

TODAYSDATE="$(date +"%Y%m%d")"
DOTCOMSOURCE="/home/user/domain.com"
DOTCOMDESTINATION="/home/user/BACKUPS/$TODAYSDATE/domain.com-BACKUP-$TODAYSDATE.tgz"
STORESOURCE="/home/user/store.domain.com"
STOREDESTINATION="/home/user/BACKUPS/$TODAYSDATE/store.domain.com-BACKUP-$TODAYSDATE.tgz"
MYSQLDBDUMP="/home/user/BACKUPS/$TODAYSDATE/mysql.domain.com-BACKUP-$TODAYSDATE.sql"
LOG="/home/user/BACKUPS/$TODAYSDATE/domain.com-BACKUP-$TODAYSDATE.log"

mkdir $TODAYSDATE
echo "Backup .COM Begin: $(date)" >> $LOG
tar cvpzfP "$DOTCOMDESTINATION" "$DOTCOMSOURCE" >> $LOG
echo "Backup .COM End: $(date)" >> $LOG
echo "#######################" >> $LOG
echo "Backup STORE Begin: $(date)" >> $LOG
tar cvpzfP "$STOREDESTINATION" "$STORESOURCE" >> $LOG
echo "Backup STORE End: $(date)" >> $LOG
echo "#######################" >> $LOG
echo "Backup MySQL DB Begin: $(date)" >> $LOG
mysqldump --opt --user=****** --password=****** --host=mysql.domain.com magento_**** > $MYSQLDBDUMP
tar cvpzfP "$MYSQLDBDUMP.tgz" "$MYSQLDBDUMP" >> $LOG
rm $MYSQLDBDUMP
echo "Backup MySQL DB End: $(date)" >> $LOG
Restore Script:
Code:
#!/bin/sh
###
# 5/30/09
# restore.domain.com.sh
# RESTORE /domain.com & /store.domain.com & mysql.domain.com
# You must enter the DATE you want to restore from
###

echo -n "Please Enter the RESTORE DATE you would like to Restore (YYYYMMDD): "
read -e RESTOREDATE

DOTCOMSOURCE="/home/user/BACKUPS/$RESTOREDATE/domain.com-BACKUP-$RESTOREDATE.tgz"
STORESOURCE="/home/user/BACKUPS/$RESTOREDATE/store.domain.com-BACKUP-$RESTOREDATE.tgz"
MYSQLDBDUMP="/home/user/BACKUPS/$RESTOREDATE/mysql.domain.com-BACKUP-$RESTOREDATE.sql"
LOG="/home/user/BACKUPS/$RESTOREDATE/domain.com-RESTORE-$RESTOREDATE.log"

echo "Restore .COM Begin: $(date)" >> $LOG
tar xvzf "$DOTCOMSOURCE" >> $LOG
echo "Restore .COM End: $(date)" >> $LOG
echo "#######################" >> $LOG
echo "Restore STORE Begin: $(date)" >> $LOG
tar xvzf "$STORESOURCE" >> $LOG
echo "Restore STORE End: $(date)" >> $LOG
echo "#######################" >> $LOG
echo "Restore MySQL DB Begin: $(date)" >> $LOG
tar xvzf $MYSQLDBDUMP.tgz >> $LOG
mysql --user=****** --password=****** --host=mysql.domain.com magento_**** < $MYSQLDBDUMP
rm $MYSQLDBDUMP
echo "Restore MySQL DB End: $(date)" >> $LOG
I am new to scripting so I just need a fresh pair of eyes to review this to see if I've missed anything. If it looks good to you please don't hold back from posting "looks fine to me", and of course any/all critiques are greatly appreciated. I just need some outside feedback to make sure I am doing this right before I go and royally mess things up. Oh and did I mention I am new to scripting :-P

Thanks in advance,
-BassKozz
 
Old 05-30-2009, 05:15 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,449
Blog Entries: 54

Rep: Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892
Looks good to me. Only things I'd do is consistent and proper quoting of variables ("${STORESOURCE}" vs $STORESOURCE), making sure you're rooted in the right place (as in 'BACKUPDIR="/home/user/BACKUPS/${TODAYSDATE}"; mkdir "${BACKUPDIR}"; cd "${BACKUPDIR}" || exit 1'), append both stdout and stderr to the log (' 2>&1>> "${LOG}"') and pipe mysqldump output through gzip to get a compressed backup. Optionally you could use MD5 or SHA1 sums to be able to verify file integrity and use "getopts" to combine making, verifying and restoring backups from one script.
 
Old 05-30-2009, 05:44 PM   #3
harry edwards
Member
 
Registered: Nov 2007
Location: Lincolnshire, UK
Distribution: CentOS, Fedora, and Suse
Posts: 365

Rep: Reputation: 48
I would add some sanity checks, like:

Do files and directories exists before I create them. i.e.

Code:
if [ ! -x "$TODAYSDATE" ] then
  mkdir $TODAYSDATE
else
 Do something like email me "Back-up executed twice: Directory already exists $TODAYSDATE?"
fi
Do I have write permissions to the target output directories and folders i.e.

Code:
if [ ! -w "$LOG" ] then
 Do something like email me "Permission denied to $LOG"
fi

Did the result of tar return an error i.e.


Code:
tar cvpzfP "$STOREDESTINATION" "$STORESOURCE" >> $LOG

if [ ! #? -eq 0 ] then
 Do something like email me "tar failed!"
fi

Last edited by harry edwards; 05-30-2009 at 05:47 PM.
 
Old 05-31-2009, 06:45 PM   #4
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,287

Rep: Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034
1. -x = file is executable: you mean -e (exists) or -d (is a dir)
http://www.tldp.org/LDP/abs/html/fto.html
2. #? : meaningless; try $?
 
Old 05-31-2009, 10:42 PM   #5
BassKozz
Member
 
Registered: Dec 2008
Distribution: Ubuntu 8.10
Posts: 35

Original Poster
Rep: Reputation: 15
Thanks for all the feedback everyone,

Quote:
Originally Posted by unSpawn View Post
Looks good to me. Only things I'd do is consistent and proper quoting of variables ("${STORESOURCE}" vs $STORESOURCE)
For my own education, what's the difference?
And do I have to put quotes around it ( is it "${STORESOURCE}" or ${STORESOURCE} ) ?
Quote:
Originally Posted by unSpawn View Post
, making sure you're rooted in the right place (as in 'BACKUPDIR="/home/user/BACKUPS/${TODAYSDATE}"; mkdir "${BACKUPDIR}"; cd "${BACKUPDIR}" || exit 1')
Doesn't the script already handle this by having the path's defined in the variables (DESTINATION for backup script, and SOURCE for restore script)?
Quote:
Originally Posted by unSpawn View Post
, append both stdout and stderr to the log (' 2>&1>> "${LOG}"')
How would that look on my script?
I am confused about how to output the stdout and stderr to the log, and what is "stdout" and "stderr" ?
Quote:
Originally Posted by unSpawn View Post
and pipe mysqldump output through gzip to get a compressed backup.
Again, not sure how to do this?
Sorry, I am new at this :-P
Quote:
Originally Posted by unSpawn View Post
Optionally you could use MD5 or SHA1 sums to be able to verify file integrity
Interesting, I'll have to research this one, any chance I could get a nudge in the right direction?
Would this be able to verify the integrity of the MySQL DB also?
Quote:
Originally Posted by unSpawn View Post
and use "getopts" to combine making, verifying and restoring backups from one script.
Time for some googling :-P
...
Quote:
Originally Posted by harry edwards View Post
I would add some sanity checks, like:

Do files and directories exists before I create them. i.e.

Code:
if [ ! -x "$TODAYSDATE" ] then
  mkdir $TODAYSDATE
else
 Do something like email me "Back-up executed twice: Directory already exists $TODAYSDATE?"
fi
Do I have write permissions to the target output directories and folders i.e.

Code:
if [ ! -w "$LOG" ] then
 Do something like email me "Permission denied to $LOG"
fi

Did the result of tar return an error i.e.


Code:
tar cvpzfP "$STOREDESTINATION" "$STORESOURCE" >> $LOG

if [ ! #? -eq 0 ] then
 Do something like email me "tar failed!"
fi
Thanks for the idea's I will look into adding these to the scripts...
Quote:
Originally Posted by chrism01 View Post
1. -x = file is executable: you mean -e (exists) or -d (is a dir)
http://www.tldp.org/LDP/abs/html/fto.html
2. #? : meaningless; try $?
Thanks for the fixes :-)

Last edited by BassKozz; 05-31-2009 at 10:51 PM.
 
Old 06-01-2009, 12:36 AM   #6
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,287

Rep: Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034
The difference between

${var}

and

$var

is that if $var is embedded in another string eg string$varstring2, the interpreter can't figure out where the varname ends, so use string${var}string2 and the interpreter knows the varname starts at '${' and stops at '}' (ignoring quote marks).
Bookmark my hyperlink above and read it.


stdin = standard input channel (aka '0' ie zero)
stdout = standard output channel (aka '1')
stderr = standard error channel (aka '2')

By default, these 3 are automatically assigned to each process at creation. To capture stdout, stderr to same file

/path/to/program 1> program.log 2>&1

The first mention of '1' can be assumed, so we get

/path/to/program > program.log 2>&1


ie send output to channel 1 (log file), send chan 2 to chan 1 => logfile also

Last edited by chrism01; 06-01-2009 at 12:40 AM.
 
Old 06-01-2009, 04:51 AM   #7
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,449
Blog Entries: 54

Rep: Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892Reputation: 2892
...in addition, wrt "${var}" vs $var it's not only embedding but also safeguarding against default IFS probs, names with spaces.

Compressing a backup here is as simple as running 'mysqldump $youroptions | gzip > "${MYSQLDBDUMP}.gz"'.

Hashing files can be done by looping over a list of files with md5sum or sha1sum, as in 'find /path/to/files -type f -print0|xargs -0 -iF md5sum 'F' > /path/of/files.md5', or recursively hashing them with md5deep or sha1deep: 'sha1deep -krs /path/to/files > /path/of/files.sha1'. Hashing a database dump doesn't make much sense unless you would want to verify the integrity of the package moving it between systems. I don't think MySQL supports internal hashing like Oracle does, maybe Maatkit's 'mk-table-checksum' can help. Checking integrity by cronjobbing 'mysqlcheck -s -u root -p --all-databases' could be a way except this locks the db while it is checking but unfortunately I'm not that deep in database forensics to suggest better/more efficient ways to hash or validate stuff.

For getopts type 'help getopts' if you run BaSH, else the http://www.tldp.org/LDP/abs/ will definately help.
 
  


Reply

Tags
backup, bash, restore, shell


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
I am just not seeing this error kinda need an extra pair of eyes please (C++) jus71n742 Programming 3 03-01-2009 04:24 PM
how to restore the startup scripts after removal kushalkoolwal Debian 6 05-10-2006 01:06 PM
i need a second pair of eyes spooge Linux - Networking 1 03-16-2005 06:25 PM
fresh eyes: site faqs abbynormal LQ Suggestions & Feedback 12 01-07-2005 01:37 AM
RH 7.3 Backup and Restore MrJoshua Linux - General 1 06-18-2003 11:39 AM


All times are GMT -5. The time now is 04:33 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration