LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Ubuntu (http://www.linuxquestions.org/questions/ubuntu-63/)
-   -   Backup Script (http://www.linuxquestions.org/questions/ubuntu-63/backup-script-848238/)

vaniaspeedy 12-03-2010 06:52 PM

Backup Script
 
Hey, I hope this is the right thread, if not my apologies, mods feel free to move it.

I had some extra time today, and decided to write a small script to backup up an ubuntu installation. I know many people have problems backing up their systems often enough, so I hope this will help.

This is a three part script;
baksys.sh is the actual backup solution
watch.ls.sh is a tiny script that will monitor the growth of the tgz file
restoresys.sh will restore your system

-----------------------------------------------------------------------------------------
baksys.sh

Code:

#!/bin/sh
## part one of a three script pack, backup restore and watch.

##Backup script by Vania S.

##calling variables to create filename.
 
TODAY=$(date +"%m-%d-%y")

HOST=$(hostname)

DISTRO=$(
cat /etc/*-release > name.txt
cut -d'=' -f2- name.txt >name2.txt
sed '3!d' name2.txt
rm name.txt name2.txt
)

filename="$HOST.$DISTRO.$TODAY.tgz"

## tellling user what's up
echo "---------------------------------------------------------------"
echo "Date: $TODAY    Distro: $DISTRO                Host:$HOST      "
echo "Creating a backup tgz file in the same location as this script."
echo "---------------------------------------------------------------"
echo "run watch.ls.sh in a new window to monitor the file growth."
echo "This script will resume in 15 seconds."
sleep 15

## actual backup command
tar cvpzf $filename --exclude=/proc --exclude=$filename --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /
echo "---------------------------------------------------------------"
echo "Done!"
echo "When you want to restore, put the tgz in the same folder as the"
echo "scripts, and run restoresys.sh"

--------------------------------------------------------------------------------------
watch.ls.sh
Code:

#!/bin/sh
## part two of a three script pack, backup restore and watch.

##Watch script by Vania S.

HOST=$(hostname)
watch -d -n 1 -d ls -lh $HOST*.tgz

--------------------------------------------------------------------------------------
restoresys.sh
Code:

#! /bin/sh
## part three of a three script pack, backup restore and watch.

##Restore script by Vania S.


HOST=$(hostname)

## start script

su
cd /
tar xvpfz $HOST*.tgz -C /
mkdir proc media lost+found mnt sys

--------------------------------------------------------------------------------------

Additional info
You can integrate these scripts into your system, and turn them into commands.
1. Move them to /bin
2. Remove the .sh extension
3. Make them executable
Code:

chmod +x baksys restoresys watch.ls
4. Voila!

Optional
Change the backup script so that the file is saved on a separate partition, example:
Original:
Code:

tar cvpzf $filename --exclude=/proc --exclude=$filename --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /
New:
Code:

tar cvpzf /YOUR/MNT/POINT/$filename --exclude=/proc --exclude=$filename --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /
Please post comments and suggestions, I'm curious to see how it goes.
Vania S.

unSpawn 12-04-2010 05:09 AM

Quote:

Originally Posted by vaniaspeedy (Post 4180005)
Code:

TODAY=$(date +"%m-%d-%y")

Purely cosmetic but the way sorting works this isn't as optimal as "%Y%m%d".


Quote:

Originally Posted by vaniaspeedy (Post 4180005)
Code:

DISTRO=$(
cat /etc/*-release > name.txt
cut -d'=' -f2- name.txt >name2.txt
sed '3!d' name2.txt
rm name.txt name2.txt
)


Code reuse isn't to be frowned upon and for getting various release files there's solutions already. For instance have a look at rkh_dat_get_os_info() starting at line 3548 in http://rkhunter.cvs.sourceforge.net/...pathrev=rel138 .


Quote:

Originally Posted by vaniaspeedy (Post 4180005)
Code:

## tellling user what's up

IMHO making backups should be done unobtrusive, regularly and unattended. Unless one wants to watch *everything* that happens on the system sending an email after the backup job completed with the location of the file and it's MD5 or SHA1 hash would be more efficient.


Quote:

Originally Posted by vaniaspeedy (Post 4180005)
Code:

## actual backup command
tar cvpzf $filename --exclude=/proc --exclude=$filename --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys


If you make a configuration item for exclusions then it is easier for users to exclude items.


Quote:

Originally Posted by vaniaspeedy (Post 4180005)
Code:

mkdir proc media lost+found mnt sys

If you run 'stat -c "%a:%u:%g"' on these directories at backup time and record details you ensure they are created again with the right permissions and ownership data.


Quote:

Originally Posted by vaniaspeedy (Post 4180005)
Optional
Change the backup script so that the file is saved on a separate partition,

This shouldn't be optional but a default configuration item. At least let backups be placed in a default directory you exclude on backup. And then the backup files should eventually be moved off the system in case of file system or hardware failure.


It's cool you took the time to create a backup script but I suggest you do look at existing solutions for their approach and features. Creating a backup of the whole system will be slow, includes a lot of files that do not change often (/usr/sbin versus /home) or are uninteresting to the user (/var/cache versus /etc) and might DoS the system if not enough disk space is available, so be careful.

vaniaspeedy 12-04-2010 03:07 PM

Wow, thank you so much for taking the time to write such a thorough and detailed response, I really appreciate it.

Quote:

Originally Posted by unSpawn (Post 4180321)
Purely cosmetic but the way sorting works this isn't as optimal as "%Y%m%d".

I changed the date sort, Thanks for catching that. I think that the hyphens make it more visually appealing, but the YY-MM-DD is definitely the way to go.


Quote:

Originally Posted by unSpawn (Post 4180321)
Code reuse isn't to be frowned upon and for getting various release files there's solutions already. For instance have a look at rkh_dat_get_os_info() starting at line 3548 in http://rkhunter.cvs.sourceforge.net/...pathrev=rel138 .

I took a look, it's a hefty chunk of code but I'll try to use it. So from what I gather that particular bit of code will create a rkhunter.dat file with the relevant inf. What would be the best way to extract the distro codename? I was thinking of using cut, since I could have the colon be the delineator.

Quote:

Originally Posted by unSpawn (Post 4180321)
IMHO making backups should be done unobtrusive, regularly and unattended. Unless one wants to watch *everything* that happens on the system sending an email after the backup job completed with the location of the file and it's MD5 or SHA1 hash would be more efficient.

What I can do is create a readme with the relevant info, and remove all of the echo commands from the script. I'll have the md5sum be put in the same place as the backup. What would be the best way to email the user the info? I'd have to ask for his email and input it into the configuration file with the exclusion items. Should I find some mail script to include in this set, or write my own? Would there be any dependency issues, for example if the user doesn't have sendmail installed?

Quote:

Originally Posted by unSpawn (Post 4180321)
If you make a configuration item for exclusions then it is easier for users to exclude items.

Could you point me to a link to learn how to do that? How would I ask for the user's input?

Quote:

Originally Posted by unSpawn (Post 4180321)
run 'stat -c "%a:%u:%g"'

What exactly does this do? Should I have it be
Code:

stat -c "%a:%u:%g" > permissions.dat

Quote:

Originally Posted by unSpawn (Post 4180321)
This shouldn't be optional but a default configuration item. At least let backups be placed in a default directory you exclude on backup.

A possible solution would be to place the backup in /home/backup, and have the folder be excluded, instead of placing the file in the same location as the script.
Quote:

Originally Posted by unSpawn (Post 4180321)
And then the backup files should eventually be moved off the system in case of file system or hardware failure.

The script creates a full system backup, and it's up to the user to be prepared and move it to a network drive or some other computer or offsite location.

Quote:

Originally Posted by unSpawn (Post 4180321)
Creating a backup of the whole system will be slow, includes a lot of files that do not change often (/usr/sbin versus /home) or are uninteresting to the user (/var/cache versus /etc) and might DoS the system if not enough disk space is available, so be careful.

My main reason for creating this script is because many people I know are new to linux, and they often do something extremely stupid like delete the bin directory or rm -rf their system. The main idea is that this creates a complete backup, from which the users could extract files that they need or just restore the whole archive to their system. It's also meant to help out in the event of a system disk failure, since everything could just be copied back onto a new partition. The only thing that would have to be tweaked is the UUID of / in the fstab so that the system boots.
There are plenty of tools that backup just /home/user and etc, but when I implement the configuration file you mentioned the user can choose to exclude those folders on his own.

Thank you for your post, if you could help me out on the questions I had, I hope to make this script even better.
How is the restoresys.sh, is it reasonable?

unSpawn 12-05-2010 08:37 AM

Quote:

Originally Posted by vaniaspeedy (Post 4180757)
I took a look, it's a hefty chunk of code but I'll try to use it. So from what I gather that particular bit of code will create a rkhunter.dat file with the relevant inf. What would be the best way to extract the distro codename? I was thinking of using cut, since I could have the colon be the delineator.

The only thing rkh_dat_get_os_info() does is show you there are different file names (RELEASE variable), usually something along the .*version.* or .*release.*.. As you can see getting OSNAME per distro is easy but it needs some treatment afterwards. BTW I've only shown that as an example. You don't have to use it if you don't want to.



Quote:

Originally Posted by vaniaspeedy (Post 4180757)
What I can do is create a readme with the relevant info, and remove all of the echo commands from the script.

That would be a good idea because a backup job should really not be left to the user but run regularly from cron (discipline issue).


Quote:

Originally Posted by vaniaspeedy (Post 4180757)
What would be the best way to email the user the info? I'd have to ask for his email and input it into the configuration file with the exclusion items. Should I find some mail script to include in this set, or write my own? Would there be any dependency issues, for example if the user doesn't have sendmail installed?

You could make it a configuration switch? For instance with "MAIL=yes" you could opt for sending email (just make it the users responsibility to have their mail server configured well) and with "MAIL=no" you could log output to syslog (try "logger -t test "hello world", then check your syslog).


Quote:

Originally Posted by vaniaspeedy (Post 4180757)
Could you point me to a link to learn how to do that? How would I ask for the user's input?

Well it's not asking but telling the user to configure things before running it:
Code:

# This allows the user to have the configuration file in another place:
CONF="/etc/myconfigfile.conf"
# If the file exists we can use it:
if [ -f "${CONF}" ]; then
 # ...but use a safeguard for those that didn't bother to configure things:
 grep -q "^IDIDNOTCONFIGURETHIS$" "${CONF}" && { echo "You didn't configure my app, exiting."; exit 127; }
 # ...so at this point all should be well and we read in the values by sourcing it with either "source" or "." in bash:
 source "${CONF}"
else
 # Woops, no config:
 echo "You didn't set a configuration file, exiting."; exit 127;
fi
# Commence with whatever you want to


Quote:

Originally Posted by vaniaspeedy (Post 4180757)
What exactly does this do?

You could run it yourself and see the output: 'stat -c "%a %u %g" /proc'.


Quote:

Originally Posted by vaniaspeedy (Post 4180757)
Should I have it be
Code:

stat -c "%a:%u:%g" > permissions.dat

Code:

# Given a list of directory names in the configuration file or script:
DIRS="/proc /media /lost+found /mnt /sys"
# ..and a configured backup or temporary directory:
BACKUPDIR="/backups"
# ...you could save permissions like this but note you only do that once if the "${BACKUPS}/permissions.dat" file doesn't exist:
[ -e "${BACKUPS}/permissions.dat" ] || for DIR in $DIRS; do echo "${DIR} $(stat -c "%a %u %g" "${DIR}")"; done > "${BACKUPS}/permissions.dat"
# ...but remember each mounted partition has its own "lost+found" (as in 'find / -maxdepth 2 -type d -name lost+found').

Code:

# ..and restore them like this (forget about IFS and spaces for now):
[ -e "${BACKUPS}/permissions.dat" ] && cat "${BACKUPS}/permissions.dat" | while read DIR MODE USER GROUP; do install -m $MODE -o $USER -g $GROUP $DIR; done



Quote:

Originally Posted by vaniaspeedy (Post 4180757)
The script creates a full system backup, and it's up to the user to be prepared and move it to a network drive or some other computer or offsite location.

Sure, that's OK, but the user should be aware there's no "magic" meaning that if she/he tries to back up an installation of say 40GB on say a 60GB disk that things will fail (horribly, depending on partition scheme). The way I look at it most SOHO installations probably are ninety per cent inert, meaning the majority of files on the system belong to an installed package. So if you would create a list of installed packages the only thing you would need to backup are /etc (or modified configuration files), /home and /var (if one wants to keep logs and say caches). On restore you would run a networked installation, retrieve packages from the list and after that dump the tarball over / (another thing to look at could be incremental backups) but maybe all of that is a wee bit to advanced for now...


Quote:

Originally Posted by vaniaspeedy (Post 4180757)
How is the restoresys.sh, is it reasonable?

Looks OK, not much that can go wrong there, but I wouldn't use "-v" because dumping a lot to the console tends to slow things down. At least for me it does.


All in all a nice effort. I think it's a good start for enhancing your scripting skills if this is something you will use yourself. If there's anything I should share with you about shell scripting it's that you should never script as root (especially when testing potentially destructive features) until you made certain that things work out OK, test things before using them ('man test'), know how to debug ('/bin/bash -vxe /path/to/scriptname.sh') things and most of all: have fun!


Code:

function help() { echo "Bash scripting guides:
http://www.tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
http://www.tldp.org/LDP/Bash-Beginners-Guide/html/index.html
http://wooledge.org/mywiki/BashFAQ?action=show&redirect=BashFaq
http://www.tldp.org/LDP/abs/html/ ; }



All times are GMT -5. The time now is 11:01 AM.