I need a backup to USB script with rotating drives

MikeP · 06-23-2013, 05:38 PM

I have a bash script that uses rsync to backup data on a daily basis to a USB hard disk and it's been working fine because the USB drive is always connected, however, the volume of data we need to backup has grown and I've had to split it across 4 x 2TB drives set as 2 spanned drives internally.

The first set is called FS1 and the second FS2, which I mount into the following two folders and make them available to our network
/srv/FS1
/srv/FS2
Both have 4TB capacity.

What I am trying to achieve is to create an automated backup script that runs from a cron job which checks what USB drive is available, mounts it into a matching mount point based on its name, then runs a rsync based backup that matches the connected drives name and exits gracefully.

I have created the mount point directories
/media/USB_FS1
/media/USB_FS2
and I have 2 drives named USB_FS1 & USB_FS2 respectively.

Mounting, rsyncing and umnounting the drives from a script, that I can do, but I can't get my head around the way to check for a drives presence first and exit gracefully if the drive is not connected, and of course what I would like to achieve with the 2 drives in rotation.

What I need help with is the portion of the script that checks for a drives presence and mounts it, then continues to the rsync line that matches the connected drive and its mount point.

Anybody care to lend a hand? I am running Linux Mint

chrism01 · 06-23-2013, 06:17 PM

You can check the output or rtn code of the mount cmd

Beryllos · 06-23-2013, 11:23 PM

2 drive rotation: Do you mean that only one drive will be present at any time? If both drives are present, do you want to detect which was used most recently, and use the other?

lleb · 06-23-2013, 11:27 PM

MikeP check your e-mail from LQ...

MikeP · 06-24-2013, 12:21 AM

Thanks for your replies.

@ chrism01 - Umm, ok, not quite sure what you mean by that. but I think I have a smattering of an idea. Do you mean like this?

Code:

if ! mount | grep -q '/media/USB_FS1'; then
    echo "/media/USB_FS1 not mounted"
    exit 1
else
    echo "blah blah not really necessary"
    rsync command here
fi

The above is very incomplete and I struggled to get even that far, but it has no graceful exit strategy, no reporting capability and it doesn't satisfy the query for which drive by name is available.

@ Beryllos - only 1 drive will be connected at a time because the backups will be taken off-site and copied to another storage facility. Due to the cost of internet here, its very slow upload speed (768kbps) and the volume of data that needs to be backed up, this is the most effective solution we can muster, though we did consider leaving both drives plugged in and purchasing another two so we can do rotations at a slower rate.

@ lleb - Can you please re-send, I recently changed my email address and hadn't updated it here. I just updated and verified the new email address with LQ. Please do accept my apologies

Thank you

lleb · 06-24-2013, 09:47 AM

sent again.

Beryllos · 06-24-2013, 10:50 AM

I would code it something like this:

Code:

drive_list="/media/USB_FS1 /media/USB_FS2 /media/USB_FS3"
drives_available=""

for drive in $drive_list; do
# test whether $drive is already mounted
    if mount | grep -q $drive; then
        echo "already mounted: $drive"
        drives_available="$drives_available $drive"
    else
# try to mount $drive
        if mount $drive >> /dev/null 2>&1; then
            echo "mount success: $drive"
            drives_available="$drives_available $drive"
        else
            echo "mount failure: $drive"
        fi
    fi
done

echo "drives available: $drives_available"
number_of_drives_available=$(echo $drives_available | wc -w)
echo "number of drives available: $number_of_drives_available"

# Here add code to select the backup target drive
if [ some user logic based on $drives_available ]; then
    target_drive=$drive
fi

# Then use $target_drive in your backup procedure:
/usr/bin/rsync -a $source_dir/ $target_drive/$backup_dir/ >> $log_file 2>&1

This checks the availability of both drives (or all drives if there are more than two). There follows some logic for selecting the drive to be the target of the backup procedure. If no drive is available, log that and exit. If only one drive is available, select it by setting the target_drive variable. If two or more drives are available, you may wish to determine which one contains the oldest backup set, select that drive, and umount the others.

Beryllos · 06-24-2013, 02:47 PM

Quote:

Originally Posted by Beryllos

Code:

if [ some user logic based on $drives_available ]; then
    target_drive=$drive
fi

We could replace the above sketchy code with this:

Code:

if [ $number_of_drives_available -eq 0 ]; then
    echo "$(date), Error: no target drive is available"  >> $log_file
    exit 1
elif [ $number_of_drives_available -eq 1 ]; then
    target_drive=$drives_available
else
# Two or more drives; what do we do?
# Until we think of something more clever, let's just pick the first one in the list
    target_drive=$(echo $drives_available | cut -d " " -f 1)
fi