LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (http://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Need a fresh pair of eyes to review my backup+restore scripts (http://www.linuxquestions.org/questions/linux-newbie-8/need-a-fresh-pair-of-eyes-to-review-my-backup-restore-scripts-729532/)

BassKozz 05-30-2009 12:34 PM
Need a fresh pair of eyes to review my backup+restore scripts
 
I've created a Backup script and a Restore script to backup my websites on a shared host (dreamhost) that I have SSH access to. The scripts should backup/restore my main website [domain.com], my subdomain [store.domain.com] of the website (which is a Magento Store), and lastly my Magento DB.

Backup Script:
Code:
#!/bin/sh
###
# 5/19/09
# backup.domain.com.sh
# Backup /domain.com & /store.domain.com & mysql.domain.com
###

TODAYSDATE="$(date +"%Y%m%d")"
DOTCOMSOURCE="/home/user/domain.com"
DOTCOMDESTINATION="/home/user/BACKUPS/$TODAYSDATE/domain.com-BACKUP-$TODAYSDATE.tgz"
STORESOURCE="/home/user/store.domain.com"
STOREDESTINATION="/home/user/BACKUPS/$TODAYSDATE/store.domain.com-BACKUP-$TODAYSDATE.tgz"
MYSQLDBDUMP="/home/user/BACKUPS/$TODAYSDATE/mysql.domain.com-BACKUP-$TODAYSDATE.sql"
LOG="/home/user/BACKUPS/$TODAYSDATE/domain.com-BACKUP-$TODAYSDATE.log"

mkdir $TODAYSDATE
echo "Backup .COM Begin: $(date)" >> $LOG
tar cvpzfP "$DOTCOMDESTINATION" "$DOTCOMSOURCE" >> $LOG
echo "Backup .COM End: $(date)" >> $LOG
echo "#######################" >> $LOG
echo "Backup STORE Begin: $(date)" >> $LOG
tar cvpzfP "$STOREDESTINATION" "$STORESOURCE" >> $LOG
echo "Backup STORE End: $(date)" >> $LOG
echo "#######################" >> $LOG
echo "Backup MySQL DB Begin: $(date)" >> $LOG
mysqldump --opt --user=****** --password=****** --host=mysql.domain.com magento_**** > $MYSQLDBDUMP
tar cvpzfP "$MYSQLDBDUMP.tgz" "$MYSQLDBDUMP" >> $LOG
rm $MYSQLDBDUMP
echo "Backup MySQL DB End: $(date)" >> $LOG
Restore Script:
Code:
#!/bin/sh
###
# 5/30/09
# restore.domain.com.sh
# RESTORE /domain.com & /store.domain.com & mysql.domain.com
# You must enter the DATE you want to restore from
###

echo -n "Please Enter the RESTORE DATE you would like to Restore (YYYYMMDD): "
read -e RESTOREDATE

DOTCOMSOURCE="/home/user/BACKUPS/$RESTOREDATE/domain.com-BACKUP-$RESTOREDATE.tgz"
STORESOURCE="/home/user/BACKUPS/$RESTOREDATE/store.domain.com-BACKUP-$RESTOREDATE.tgz"
MYSQLDBDUMP="/home/user/BACKUPS/$RESTOREDATE/mysql.domain.com-BACKUP-$RESTOREDATE.sql"
LOG="/home/user/BACKUPS/$RESTOREDATE/domain.com-RESTORE-$RESTOREDATE.log"

echo "Restore .COM Begin: $(date)" >> $LOG
tar xvzf "$DOTCOMSOURCE" >> $LOG
echo "Restore .COM End: $(date)" >> $LOG
echo "#######################" >> $LOG
echo "Restore STORE Begin: $(date)" >> $LOG
tar xvzf "$STORESOURCE" >> $LOG
echo "Restore STORE End: $(date)" >> $LOG
echo "#######################" >> $LOG
echo "Restore MySQL DB Begin: $(date)" >> $LOG
tar xvzf $MYSQLDBDUMP.tgz >> $LOG
mysql --user=****** --password=****** --host=mysql.domain.com magento_**** < $MYSQLDBDUMP
rm $MYSQLDBDUMP
echo "Restore MySQL DB End: $(date)" >> $LOG
I am new to scripting so I just need a fresh pair of eyes to review this to see if I've missed anything. If it looks good to you please don't hold back from posting "looks fine to me", and of course any/all critiques are greatly appreciated. I just need some outside feedback to make sure I am doing this right before I go and royally mess things up. Oh and did I mention I am new to scripting :-P

Thanks in advance,
-BassKozz

unSpawn 05-30-2009 05:15 PM
Looks good to me. Only things I'd do is consistent and proper quoting of variables ("${STORESOURCE}" vs $STORESOURCE), making sure you're rooted in the right place (as in 'BACKUPDIR="/home/user/BACKUPS/${TODAYSDATE}"; mkdir "${BACKUPDIR}"; cd "${BACKUPDIR}" || exit 1'), append both stdout and stderr to the log (' 2>&1>> "${LOG}"') and pipe mysqldump output through gzip to get a compressed backup. Optionally you could use MD5 or SHA1 sums to be able to verify file integrity and use "getopts" to combine making, verifying and restoring backups from one script.

harry edwards 05-30-2009 05:44 PM
I would add some sanity checks, like:

Do files and directories exists before I create them. i.e.

Code:
if [ ! -x "$TODAYSDATE" ] then
  mkdir $TODAYSDATE
else
 Do something like email me "Back-up executed twice: Directory already exists $TODAYSDATE?"
fi
Do I have write permissions to the target output directories and folders i.e.

Code:
if [ ! -w "$LOG" ] then
 Do something like email me "Permission denied to $LOG"
fi

Did the result of tar return an error i.e.


Code:
tar cvpzfP "$STOREDESTINATION" "$STORESOURCE" >> $LOG

if [ ! #? -eq 0 ] then
 Do something like email me "tar failed!"
fi

chrism01 05-31-2009 06:45 PM
1. -x = file is executable: you mean -e (exists) or -d (is a dir)
http://www.tldp.org/LDP/abs/html/fto.html
2. #? : meaningless; try $?

BassKozz 05-31-2009 10:42 PM
Thanks for all the feedback everyone,

Quote:
Originally Posted by unSpawn (Post 3557723)
Looks good to me. Only things I'd do is consistent and proper quoting of variables ("${STORESOURCE}" vs $STORESOURCE)
For my own education, what's the difference?
And do I have to put quotes around it ( is it "${STORESOURCE}" or ${STORESOURCE} ) ?
Quote:
Originally Posted by unSpawn (Post 3557723)
, making sure you're rooted in the right place (as in 'BACKUPDIR="/home/user/BACKUPS/${TODAYSDATE}"; mkdir "${BACKUPDIR}"; cd "${BACKUPDIR}" || exit 1')
Doesn't the script already handle this by having the path's defined in the variables (DESTINATION for backup script, and SOURCE for restore script)?
Quote:
Originally Posted by unSpawn (Post 3557723)
, append both stdout and stderr to the log (' 2>&1>> "${LOG}"')
How would that look on my script?
I am confused about how to output the stdout and stderr to the log, and what is "stdout" and "stderr" ?
Quote:
Originally Posted by unSpawn (Post 3557723)
and pipe mysqldump output through gzip to get a compressed backup.
Again, not sure how to do this?
Sorry, I am new at this :-P
Quote:
Originally Posted by unSpawn (Post 3557723)
Optionally you could use MD5 or SHA1 sums to be able to verify file integrity
Interesting, I'll have to research this one, any chance I could get a nudge in the right direction?
Would this be able to verify the integrity of the MySQL DB also?
Quote:
Originally Posted by unSpawn (Post 3557723)
and use "getopts" to combine making, verifying and restoring backups from one script.
Time for some googling :-P
...
Quote:
Originally Posted by harry edwards (Post 3557731)
I would add some sanity checks, like:

Do files and directories exists before I create them. i.e.

Code:
if [ ! -x "$TODAYSDATE" ] then
  mkdir $TODAYSDATE
else
 Do something like email me "Back-up executed twice: Directory already exists $TODAYSDATE?"
fi
Do I have write permissions to the target output directories and folders i.e.

Code:
if [ ! -w "$LOG" ] then
 Do something like email me "Permission denied to $LOG"
fi

Did the result of tar return an error i.e.


Code:
tar cvpzfP "$STOREDESTINATION" "$STORESOURCE" >> $LOG

if [ ! #? -eq 0 ] then
 Do something like email me "tar failed!"
fi
Thanks for the idea's I will look into adding these to the scripts...
Quote:
Originally Posted by chrism01 (Post 3558451)
1. -x = file is executable: you mean -e (exists) or -d (is a dir)
http://www.tldp.org/LDP/abs/html/fto.html
2. #? : meaningless; try $?
Thanks for the fixes :-)

chrism01 06-01-2009 12:36 AM
The difference between

${var}

and

$var

is that if $var is embedded in another string eg string$varstring2, the interpreter can't figure out where the varname ends, so use string${var}string2 and the interpreter knows the varname starts at '${' and stops at '}' (ignoring quote marks).
Bookmark my hyperlink above and read it.


stdin = standard input channel (aka '0' ie zero)
stdout = standard output channel (aka '1')
stderr = standard error channel (aka '2')

By default, these 3 are automatically assigned to each process at creation. To capture stdout, stderr to same file

/path/to/program 1> program.log 2>&1

The first mention of '1' can be assumed, so we get

/path/to/program > program.log 2>&1


ie send output to channel 1 (log file), send chan 2 to chan 1 => logfile also

unSpawn 06-01-2009 04:51 AM
...in addition, wrt "${var}" vs $var it's not only embedding but also safeguarding against default IFS probs, names with spaces.

Compressing a backup here is as simple as running 'mysqldump $youroptions | gzip > "${MYSQLDBDUMP}.gz"'.

Hashing files can be done by looping over a list of files with md5sum or sha1sum, as in 'find /path/to/files -type f -print0|xargs -0 -iF md5sum 'F' > /path/of/files.md5', or recursively hashing them with md5deep or sha1deep: 'sha1deep -krs /path/to/files > /path/of/files.sha1'. Hashing a database dump doesn't make much sense unless you would want to verify the integrity of the package moving it between systems. I don't think MySQL supports internal hashing like Oracle does, maybe Maatkit's 'mk-table-checksum' can help. Checking integrity by cronjobbing 'mysqlcheck -s -u root -p --all-databases' could be a way except this locks the db while it is checking but unfortunately I'm not that deep in database forensics to suggest better/more efficient ways to hash or validate stuff.

For getopts type 'help getopts' if you run BaSH, else the http://www.tldp.org/LDP/abs/ will definately help.


All times are GMT -5. The time now is 04:02 AM.