LinuxQuestions.org - help with modifying a backup script

- Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)

- - help with modifying a backup script (https://www.linuxquestions.org/questions/linux-general-1/help-with-modifying-a-backup-script-486921/)

help with modifying a backup script

hi all,

i have a backup script i aquired from somewhere (credit where credit is due).
i would like to modify it a little.What it does is backs up daily (/etc/cron.daily) what ever directories you enter into it.
e.g. BACKUPTHESE="/home /root/"

it backs up to directories i have created.

e.g on a monday it backsup everything inside /home & /root and places it in /media/other/backup/daily.On Tuesday it first moves everything from /dailt into /yesterday an so on. (see the script below)

Code:

#!/bin/bash

# A Simple (Poor Man) disk based backup script

# Will backup all user home directories in seperate archives

# as a single /home backup file can be quite large and unwieldly.

# Does not need to be modified if users are added/deleted



# Step - 1 Create Timestamp and set up variables and functions

# BUDTSTAMP = Backup Date/Time Stamp

BUTDSTAMP=$(date +%Y%m%d)



# variable holding directories containing files to backup eg: BACKUPTHESE="/home /root /etc"

BACKUPTHESE="/home"



#directory containing today's backup

BKUPDIR="/media/other/backup/daily/home"

#directory containing yesterday's backup

YDBKUPDIR="/media/other/backup/yesterday/home"

#directory containing the day before yesterday's backup

DBBKUPDIR="/media/other/backup/daybefore/home"



# log files

LOG="mail.txt"

ERRORLOG="ERROR.txt"



# tag for backup file name

TAG="home-backup"



# function that quits and logs on error

exiterror()

{

# use this function by supplying $LINENO as first arg

echo "Fatal error caused by line ${1} of ${0}" >> $ERRORLOG

mail somebody@somedomain.com -s "Backup Job ERROR" -v < $ERRORLOG

mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG

exit 1

}



# Step - 2 Start Email Message To Be Sent



# Remove mail message from previous backup

# I do this at the beginning of the script instead of the end

# in case the mail does not send for whatever reason or

# I need to debug it



rm $LOG > /dev/null 2>&1

echo "System Backup $BUTDSTAMP" >> $LOG



# The email sends a user friendly note showing the start and

# end time/dates. This is important so you can compare logs

# and see if a backup ran ok.



echo "Backup Began $(date)" >> $LOG



# Step - 3 Rotate Backups

# Simple three backupset rotation, keeps only last three

# Use directory /media/other/backup as an example, make sure you change this path

# to fit your local settings. Have three directories in /media/other/backup named daily,

# yesterday and daybefore.

# Make sure /media/other/backup is on a different disk (preferably a different machine)

# than the files you are backing up



#looks specifically for backup files in case other files are kept in these directories

for bkfile in $DBBKUPDIR/*; do

echo $bkfile | grep $TAG >/dev/null &&

(rm $bkfile || exiterror $LINENO )

done

for bkfile in $YDBKUPDIR/*; do

echo $bkfile | grep $TAG >/dev/null &&

(mv $bkfile $DBBKUPDIR/ || exiterror $LINENO )

done

for bkfile in $BKUPDIR/*; do

echo $bkfile | grep $TAG >/dev/null &&

(mv $bkfile $YDBKUPDIR/ || exiterror $LINENO )

done



# Step - 4 Archive Home Directories

# Creates a seperate tar file for each directory in the directories in $BACKUPTHESE





# change to dir to backup. Check in case cd failed for some reason.

cd $BACKUPTHIS || exiterror $LINENO

#backup all files listed in BACKUPTHESE

for DIRTOBACKUP in $BACKUPTHESE; do

for FOLDERNAME in $DIRTOBACKUP/*

do

# Archives are created in the format someuser-$TAG-datetime.tar.gz

# basename is used here so as not to include absolute paths

# -p preserves permissions

echo -e "-------------------\n>>>taring ${FOLDERNAME}\n\n" >> $LOG

tar -czvpf ${BKUPDIR}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} >> $LOG || exiterror $LINENO

done

done



# Step - 5 Finish Email Report and Send



echo "Backup ended $(date)" >> $LOG



# df -h includes a human readable disk usage report of the media that /media/other/backup

# is mounted on. Good to now if your backup disk is running out of space.

# Of course /dev/hdb1 is the device I use, modify it for your local settings



df -h /dev/hdb1 >> $LOG

mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG



# Step - 6 All Done

The trouble is you end up with a folder full of individual archives, and not knowing where they come from.If you also add /etc to the backups it starts to.....well i think you can see what happens.Is there a way to maybe have it put backups from each folder e.g. /home & /etc into maybe say /media/other/backup/daily/home.

Or any other ideas on a better setup.

Thanks

bump

anyone got any ideas

hello. what's BACKUPTHIS in 'cd $BACKUPTHIS || exiterror $LINENO'?

i'm not really sure it looks like it just checks if it changed directory alright.
e.g. if you didn't create the directory your changing to, maybe?

The script works great though.

Code:

#!/bin/sh



backupdir="/path/to/backup/dir"

backupthese="/foo/bar /baz/qux"



day=`date +%Y%m%d`

dir="$backupdir/$day"; mkdir $dir

for d in $backupthese; do

        tar cfz "$dir/`echo $d | tr / !`.tar.gz" $d

done

rm -rf "$backupdir/`ls -1 $backupdir | head -1`"

This script backups /foo/bar und /baz/qux to /path/to/backup/dir/20060927/!foo!bar.tar.gz and /path/to/backup/dir/20060927/!baz!qux.tar.gz (since we are the 27th of september).

The names of the tar.gz files let you know where they come from.

The script also removes the oldest backup: if you want to give it a try, make sure to create /path/to/backup/dir/20060926 and newer dummy backup directories before running it.

An advice: NEVER run a script if you do not understand each line of it.

Bye!

Quote:

Originally Posted by dfwcomputer

i'm not really sure it looks like it just checks if it changed directory alright.
e.g. if you didn't create the directory your changing to, maybe?

The script works great though.

it appears that BACKUPTHIS doesn't have any value. it's not BACKUPTHESE right? so cd to where? is it really included in the script?

Quote:

it appears that BACKUPTHIS doesn't have any value. it's not BACKUPTHESE right? so cd to where? is it really included in the script?

Sorry yes your right it is BACKUPTHESE

Quote:

Originally Posted by kloss

Code:

#!/bin/sh



backupdir="/path/to/backup/dir"

backupthese="/foo/bar /baz/qux"



day=`date +%Y%m%d`

dir="$backupdir/$day"; mkdir $dir

for d in $backupthese; do

    tar cfz "$dir/`echo $d | tr / !`.tar.gz" $d

done

rm -rf "$backupdir/`ls -1 $backupdir | head -1`"

Thanks i will try it out.

Quote:

Originally Posted by kloss

An advice: NEVER run a script if you do not understand each line of it.

the system i run is just one i draged out of the shed to muck around on so i not really concerned even if the pc self destructed.

But i here the warning ;)

What do you prefer to identify the folders, tags or directories?

not quite sure of the difference, but i would imagine that it would not really bother me either way.

please find a way to test this mod. if you don't like each pair to be separated in a different directory just tell me.

Code:

#!/bin/bash



# A Simple (Poor Man) disk based backup script

# Will backup all user home directories in seperate archives

# as a single /home backup file can be quite large and unwieldly.

# Does not need to be modified if users are added/deleted



# Step - 1 Create Timestamp and set up variables and functions

# BUDTSTAMP = Backup Date/Time Stamp

BUTDSTAMP=$(date +%Y%m%d)



# variable holding directories containing files to backup with tags

# format: "DIR:TAG ..."

# eg: BACKUPPAIRS="/homehome-backup /root:root-backup /etc:etc-backup"

BACKUPPAIRS="/home:home-backup"



#directory containing today's backup

BKUPDIR="/media/other/backup/daily/home"

#directory containing yesterday's backup

YDBKUPDIR="/media/other/backup/yesterday/home"

#directory containing the day before yesterday's backup

DBBKUPDIR="/media/other/backup/daybefore/home"



# log files

LOG="mail.txt"

ERRORLOG="ERROR.txt"



# function that quits and logs on error

exiterror()

{

        # use this function by supplying $LINENO as first arg

        echo "Fatal error caused by line ${1} of ${0}" >> $ERRORLOG

        mail somebody@somedomain.com -s "Backup Job ERROR" -v < $ERRORLOG

        mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG

        exit 1

}



# Step - 2 Start Email Message To Be Sent



# Remove mail message from previous backup

# I do this at the beginning of the script instead of the end

# in case the mail does not send for whatever reason or

# I need to debug it



rm $LOG > /dev/null 2>&1

echo "System Backup $BUTDSTAMP" >> $LOG



# The email sends a user friendly note showing the start and

# end time/dates. This is important so you can compare logs

# and see if a backup ran ok.



echo "Backup Began $(date)" >> $LOG



# Step - 3 Rotate Backups

# Simple three backupset rotation, keeps only last three

# Use directory /media/other/backup as an example, make sure you change this path

# to fit your local settings. Have three directories in /media/other/backup named daily,

# yesterday and daybefore.

# Make sure /media/other/backup is on a different disk (preferably a different machine)

# than the files you are backing up



#looks specifically for backup files in case other files are kept in these directories

for PAIR in $BACKUPPAIRS; do

        TAG=${PAIR/*:}

        [ -d $DBBKUPDIR/$TAG ] && \

        ( rm -fr $DBBKUPDIR/$TAG || exiterror $LINENO )

        [ -d $YDBKUPDIR/$TAG ] && \

        ( mv $YDBKUPDIR/$TAG $DBBKUPDIR/ || exiterror $LINENO )

        [ -d $BKUPDIR/$TAG ] && \

        ( mv $BKUPDIR/$TAG $YDBKUPDIR/ || exiterror $LINENO )

done



# Step - 4 Archive Home Directories

# Creates a seperate tar file for each directory in the directories in $BACKUPPAIRS



# change to dir to backup. Check in case cd failed for some reason.

# cd $BACKUPPAIRS || exiterror $LINENO

#backup all files listed in BACKUPPAIRS

for PAIR in $BACKUPPAIRS; do

        DIRTOBACKUP=${PAIR/:*}

        TAG=${PAIR/*:}

        [ -d $DIRTOBACKUP ] || exiterror $LINENO

        for FOLDERNAME in $DIRTOBACKUP/*

        do

                # Archives are created in the format someuser-$TAG-datetime.tar.gz

                # basename is used here so as not to include absolute paths

                # -p preserves permissions

                echo -e "-------------------\n>>>taring ${FOLDERNAME}\n\n" >> $LOG

                mkdir -p ${BKUPDIR}/${TAG} || exiterror $LINENO

                tar -czvpf ${BKUPDIR}/${TAG}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} >> $LOG || exiterror $LINENO

        done

done



# Step - 5 Finish Email Report and Send



echo "Backup ended $(date)" >> $LOG



# df -h includes a human readable disk usage report of the media that /media/other/backup

# is mounted on. Good to now if your backup disk is running out of space.

# Of course /dev/hdb1 is the device I use, modify it for your local settings



df -h /dev/hdb1 >> $LOG

mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG



# Step - 6 All Done

edit: enclosed some of the commands

o.k gave it a go but i get errors.Not sure what this means (see below)

Code:

root@server:~# sh backup

: command not found

: command not found

: command not found

: command not found

: command not found

'ackup: line 28: syntax error near unexpected token `

'ackup: line 28: `exiterror()

root@server:~#

any ideas

Text format error. Sorry about that. I tested the code myself and made new changes. Please try again.

Code:

#!/bin/bash



# A Simple (Poor Man) disk based backup script

# Will backup all user home directories in seperate archives

# as a single /home backup file can be quite large and unwieldly.

# Does not need to be modified if users are added/deleted



# Step - 1 Create Timestamp and set up variables and functions

# BUDTSTAMP = Backup Date/Time Stamp

BUTDSTAMP=$(date +%Y%m%d)



# variable holding directories containing files to backup with tags

# format: "DIR:TAG ..."

# eg: BACKUPPAIRS="/homehome-backup /root:root-backup /etc:etc-backup"

BACKUPPAIRS="/home:home-backup /etc:etc-backup"



#directory containing today's backup

BKUPDIR="/media/other/backup/daily"

#directory containing yesterday's backup

YDBKUPDIR="/media/other/backup/yesterday"

#directory containing the day before yesterday's backup

DBBKUPDIR="/media/other/backup/daybefore"



# log files

LOG="mail.txt"

ERRORLOG="ERROR.txt"



# function that quits and logs on error

exiterror()

{

        # use this function by supplying $LINENO as first arg

        echo "Fatal error caused by line ${1} of ${0}" >> $ERRORLOG

        mail somebody@somedomain.com -s "Backup Job ERROR" -v < $ERRORLOG

        mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG

        exit 1

}



# Step - 2 Start Email Message To Be Sent



# Remove mail message from previous backup

# I do this at the beginning of the script instead of the end

# in case the mail does not send for whatever reason or

# I need to debug it



rm $LOG > /dev/null 2>&1

echo "System Backup $BUTDSTAMP" >> $LOG



# The email sends a user friendly note showing the start and

# end time/dates. This is important so you can compare logs

# and see if a backup ran ok.



echo "Backup Began $(date)" >> $LOG



# Step - 3 Rotate Backups

# Simple three backupset rotation, keeps only last three

# Use directory /media/other/backup as an example, make sure you change this path

# to fit your local settings. Have three directories in /media/other/backup named daily,

# yesterday and daybefore.

# Make sure /media/other/backup is on a different disk (preferably a different machine)

# than the files you are backing up



# create the backup directories if they don't exist

[ ! -d $DBBKUPDIR ] && ( mkdir -p $DBBKUPDIR || exiterror $LINENO )

[ ! -d $YDBKUPDIR ] && ( mkdir -p $YDBKUPDIR || exiterror $LINENO )

[ ! -d $BKUPDIR ] && ( mkdir -p $BKUPDIR || exiterror $LINENO )



#looks specifically for backup files in case other files are kept in these directories

for PAIR in $BACKUPPAIRS; do

        TAG=${PAIR/*:}

        [ -d $DBBKUPDIR/$TAG ] && \

        ( rm -fr $DBBKUPDIR/$TAG || exiterror $LINENO )

        [ -d $YDBKUPDIR/$TAG ] && \

        ( mv $YDBKUPDIR/$TAG $DBBKUPDIR/ || exiterror $LINENO )

        [ -d $BKUPDIR/$TAG ] && \

        ( mv $BKUPDIR/$TAG $YDBKUPDIR/ || exiterror $LINENO )

done



# Step - 4 Archive Home Directories

# Creates a seperate tar file for each directory in the directories in $BACKUPPAIRS



# change to dir to backup. Check in case cd failed for some reason.

# cd $BACKUPPAIRS || exiterror $LINENO

#backup all files listed in BACKUPPAIRS

for PAIR in $BACKUPPAIRS; do

        DIRTOBACKUP=${PAIR/:*}

        TAG=${PAIR/*:}

        echo "Doing backups for $DIRTOBACKUP using TAG $TAG" >> $LOG

        [ -d $DIRTOBACKUP ] || exiterror $LINENO

        # if we don't want to include DIRTOBACKUP from the archive, we should

        # note the annoying 'removing trailing lines' tar messages so i included 2>/dev/null

        # cd $DIRTOBACKUP || exiterror $LINENO

        # for FOLDERNAME in *

        for FOLDERNAME in $DIRTOBACKUP/*

        do

                # Archives are created in the format someuser-$TAG-datetime.tar.gz

                # basename is used here so as not to include absolute paths

                # -p preserves permissions

                echo -e "-------------------\n>>> tarring ${FOLDERNAME}" >> $LOG

                mkdir -p ${BKUPDIR}/${TAG} || exiterror $LINENO

                tar -czvpf ${BKUPDIR}/${TAG}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} >> $LOG 2>/dev/null || exiterror $LINENO

                echo >> $LOG

        done

done



# Step - 5 Finish Email Report and Send



echo "Backup ended $(date)" >> $LOG



# df -h includes a human readable disk usage report of the media that /media/other/backup

# is mounted on. Good to now if your backup disk is running out of space.

# Of course /dev/hdb1 is the device I use, modify it for your local settings



df -h /dev/hdb1 >> $LOG

mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG



# Step - 6 All Done

Works great, thanks for spending the time to do that it's much appreciated :D

is there a simple thing i can change to make the script verbose.Not a big issue just wondering.

Also any ideas on the best way to restore these backup's if failure did happen to occur.

Once again thanks for the script (much appreciated)

Dean

added verbosity (script.sh -v)

Code:

#!/bin/bash



# A Simple (Poor Man) disk based backup script

# Will backup all user home directories in seperate archives

# as a single /home backup file can be quite large and unwieldly.

# Does not need to be modified if users are added/deleted



# Some variables

# Set to 1 for verbose mode by default (might not work with cron).

VERBOSE=



# Some messaging functions

log() {

        [ "$VERBOSE" ] && echo -e "$@"

        echo -e "$@" >> $LOG

}



log_error() {

        [ "$VERBOSE" ] && echo -e "$@"

        echo -e "$@" >> $LOG

}



# Check args

for a in $*; do

        case $a in

        -v)

                VERBOSE=1

                ;;

        *)

                ;;

        esac

done



# Step - 1 Create Timestamp and set up variables and functions

# BUDTSTAMP = Backup Date/Time Stamp

BUTDSTAMP=$(date +%Y%m%d)



# variable holding directories containing files to backup with tags

# format: "DIR:TAG ..."

# eg: BACKUPPAIRS="/homehome-backup /root:root-backup /etc:etc-backup"

BACKUPPAIRS="/home:home-backup /etc:etc-backup"



#directory containing today's backup

BKUPDIR="/media/other/backup/daily"

#directory containing yesterday's backup

YDBKUPDIR="/media/other/backup/yesterday"

#directory containing the day before yesterday's backup

DBBKUPDIR="/media/other/backup/daybefore"



# log files

LOG="mail.txt"

ERRORLOG="ERROR.txt"



# function that quits and logs on error

exiterror()

{

        # use this function by supplying $LINENO as first arg

        log_error "Fatal error caused by line ${1} of ${0}"

        mail somebody@somedomain.com -s "Backup Job ERROR" -v < $ERRORLOG

        mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG

        exit 1

}



# Step - 2 Start Email Message To Be Sent



# Remove mail message from previous backup

# I do this at the beginning of the script instead of the end

# in case the mail does not send for whatever reason or

# I need to debug it



rm $LOG > /dev/null 2>&1

log "System Backup $BUTDSTAMP"



# The email sends a user friendly note showing the start and

# end time/dates. This is important so you can compare logs

# and see if a backup ran ok.



log "Backup Began $(date)"



# Step - 3 Rotate Backups

# Simple three backupset rotation, keeps only last three

# Use directory /media/other/backup as an example, make sure you change this path

# to fit your local settings. Have three directories in /media/other/backup named daily,

# yesterday and daybefore.

# Make sure /media/other/backup is on a different disk (preferably a different machine)

# than the files you are backing up



# create the backup directories if they don't exist

[ ! -d $DBBKUPDIR ] && ( mkdir -p $DBBKUPDIR || exiterror $LINENO )

[ ! -d $YDBKUPDIR ] && ( mkdir -p $YDBKUPDIR || exiterror $LINENO )

[ ! -d $BKUPDIR ] && ( mkdir -p $BKUPDIR || exiterror $LINENO )



#looks specifically for backup files in case other files are kept in these directories

for PAIR in $BACKUPPAIRS; do

        TAG=${PAIR/*:}

        [ -d $DBBKUPDIR/$TAG ] && \

        ( rm -fr $DBBKUPDIR/$TAG || exiterror $LINENO )

        [ -d $YDBKUPDIR/$TAG ] && \

        ( mv $YDBKUPDIR/$TAG $DBBKUPDIR/ || exiterror $LINENO )

        [ -d $BKUPDIR/$TAG ] && \

        ( mv $BKUPDIR/$TAG $YDBKUPDIR/ || exiterror $LINENO )

done



# Step - 4 Archive Home Directories

# Creates a seperate tar file for each directory in the directories in $BACKUPPAIRS



# change to dir to backup. Check in case cd failed for some reason.

# cd $BACKUPPAIRS || exiterror $LINENO

# check verbosity

if [ "$VERBOSE" ]; then

        TAROUTPUT="| tee -a $LOG"

else

        TAROUTPUT=">> $LOG 2>/dev/null"

fi

# backup all files listed in BACKUPPAIRS

for PAIR in $BACKUPPAIRS; do

        DIRTOBACKUP=${PAIR/:*}

        TAG=${PAIR/*:}

        log "Doing backups for $DIRTOBACKUP using TAG $TAG"

        [ -d $DIRTOBACKUP ] || exiterror $LINENO

        # if we don't want to include DIRTOBACKUP from the archive, we should

        # note the annoying 'removing trailing lines' tar messages so i included 2>/dev/null

        # cd $DIRTOBACKUP || exiterror $LINENO

        # for FOLDERNAME in *

        for FOLDERNAME in $DIRTOBACKUP/*

        do

                # Archives are created in the format someuser-$TAG-datetime.tar.gz

                # basename is used here so as not to include absolute paths

                # -p preserves permissions

                log "-------------------\n>>> tarring ${FOLDERNAME}"

                mkdir -p ${BKUPDIR}/${TAG} || exiterror $LINENO

                # tar -czvpf ${BKUPDIR}/${TAG}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} >> $LOG 2>/dev/null || exiterror $LINENO

                eval "tar -czvpf ${BKUPDIR}/${TAG}/$( basename $FOLDERNAME )-${TAG}-${BUTDSTAMP}.tar.gz ${FOLDERNAME} ${TAROUTPUT} || exiterror $LINENO"

                log

        done

done



# Step - 5 Finish Email Report and Send



log "Backup ended $(date)"



# df -h includes a human readable disk usage report of the media that /media/other/backup

# is mounted on. Good to now if your backup disk is running out of space.

# Of course /dev/hdb1 is the device I use, modify it for your local settings



df -h /dev/hdb1 >> $LOG

mail somebody@somedomain.com -s "Backup Job Report" -v < $LOG



# Step - 6 All Done