LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Ubuntu
User Name
Password
Ubuntu This forum is for the discussion of Ubuntu Linux.

Notices



Reply
 
Search this Thread
Old 12-03-2010, 07:52 PM   #1
vaniaspeedy
LQ Newbie
 
Registered: Dec 2010
Location: Silicon Valley, California
Distribution: Ubuntu 10.04, 9.10 in a VM,
Posts: 6

Rep: Reputation: 0
Backup Script


Hey, I hope this is the right thread, if not my apologies, mods feel free to move it.

I had some extra time today, and decided to write a small script to backup up an ubuntu installation. I know many people have problems backing up their systems often enough, so I hope this will help.

This is a three part script;
baksys.sh is the actual backup solution
watch.ls.sh is a tiny script that will monitor the growth of the tgz file
restoresys.sh will restore your system

-----------------------------------------------------------------------------------------
baksys.sh

Code:
#!/bin/sh
## part one of a three script pack, backup restore and watch.

##Backup script by Vania S.

##calling variables to create filename.
 
TODAY=$(date +"%m-%d-%y")

HOST=$(hostname)

DISTRO=$(
cat /etc/*-release > name.txt
cut -d'=' -f2- name.txt >name2.txt
sed '3!d' name2.txt
rm name.txt name2.txt
)

filename="$HOST.$DISTRO.$TODAY.tgz"

## tellling user what's up
echo "---------------------------------------------------------------"
echo "Date: $TODAY    Distro: $DISTRO                Host:$HOST      "
echo "Creating a backup tgz file in the same location as this script."
echo "---------------------------------------------------------------"
echo "run watch.ls.sh in a new window to monitor the file growth."
echo "This script will resume in 15 seconds."
sleep 15

## actual backup command
tar cvpzf $filename --exclude=/proc --exclude=$filename --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /
echo "---------------------------------------------------------------"
echo "Done!"
echo "When you want to restore, put the tgz in the same folder as the"
echo "scripts, and run restoresys.sh"
--------------------------------------------------------------------------------------
watch.ls.sh
Code:
#!/bin/sh
## part two of a three script pack, backup restore and watch.

##Watch script by Vania S.

HOST=$(hostname)
watch -d -n 1 -d ls -lh $HOST*.tgz
--------------------------------------------------------------------------------------
restoresys.sh
Code:
#! /bin/sh
## part three of a three script pack, backup restore and watch.

##Restore script by Vania S.


HOST=$(hostname)

## start script

su
cd /
tar xvpfz $HOST*.tgz -C /
mkdir proc media lost+found mnt sys
--------------------------------------------------------------------------------------

Additional info
You can integrate these scripts into your system, and turn them into commands.
1. Move them to /bin
2. Remove the .sh extension
3. Make them executable
Code:
chmod +x baksys restoresys watch.ls
4. Voila!

Optional
Change the backup script so that the file is saved on a separate partition, example:
Original:
Code:
tar cvpzf $filename --exclude=/proc --exclude=$filename --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /
New:
Code:
tar cvpzf /YOUR/MNT/POINT/$filename --exclude=/proc --exclude=$filename --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /
Please post comments and suggestions, I'm curious to see how it goes.
Vania S.
 
Old 12-04-2010, 06:09 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,699
Blog Entries: 54

Rep: Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962
Quote:
Originally Posted by vaniaspeedy View Post
Code:
TODAY=$(date +"%m-%d-%y")
Purely cosmetic but the way sorting works this isn't as optimal as "%Y%m%d".


Quote:
Originally Posted by vaniaspeedy View Post
Code:
DISTRO=$(
cat /etc/*-release > name.txt
cut -d'=' -f2- name.txt >name2.txt
sed '3!d' name2.txt
rm name.txt name2.txt
)
Code reuse isn't to be frowned upon and for getting various release files there's solutions already. For instance have a look at rkh_dat_get_os_info() starting at line 3548 in http://rkhunter.cvs.sourceforge.net/...pathrev=rel138 .


Quote:
Originally Posted by vaniaspeedy View Post
Code:
## tellling user what's up
IMHO making backups should be done unobtrusive, regularly and unattended. Unless one wants to watch *everything* that happens on the system sending an email after the backup job completed with the location of the file and it's MD5 or SHA1 hash would be more efficient.


Quote:
Originally Posted by vaniaspeedy View Post
Code:
## actual backup command
tar cvpzf $filename --exclude=/proc --exclude=$filename --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys
If you make a configuration item for exclusions then it is easier for users to exclude items.


Quote:
Originally Posted by vaniaspeedy View Post
Code:
mkdir proc media lost+found mnt sys
If you run 'stat -c "%a:%u:%g"' on these directories at backup time and record details you ensure they are created again with the right permissions and ownership data.


Quote:
Originally Posted by vaniaspeedy View Post
Optional
Change the backup script so that the file is saved on a separate partition,
This shouldn't be optional but a default configuration item. At least let backups be placed in a default directory you exclude on backup. And then the backup files should eventually be moved off the system in case of file system or hardware failure.


It's cool you took the time to create a backup script but I suggest you do look at existing solutions for their approach and features. Creating a backup of the whole system will be slow, includes a lot of files that do not change often (/usr/sbin versus /home) or are uninteresting to the user (/var/cache versus /etc) and might DoS the system if not enough disk space is available, so be careful.
 
1 members found this post helpful.
Old 12-04-2010, 04:07 PM   #3
vaniaspeedy
LQ Newbie
 
Registered: Dec 2010
Location: Silicon Valley, California
Distribution: Ubuntu 10.04, 9.10 in a VM,
Posts: 6

Original Poster
Rep: Reputation: 0
Wow, thank you so much for taking the time to write such a thorough and detailed response, I really appreciate it.

Quote:
Originally Posted by unSpawn View Post
Purely cosmetic but the way sorting works this isn't as optimal as "%Y%m%d".
I changed the date sort, Thanks for catching that. I think that the hyphens make it more visually appealing, but the YY-MM-DD is definitely the way to go.


Quote:
Originally Posted by unSpawn View Post
Code reuse isn't to be frowned upon and for getting various release files there's solutions already. For instance have a look at rkh_dat_get_os_info() starting at line 3548 in http://rkhunter.cvs.sourceforge.net/...pathrev=rel138 .
I took a look, it's a hefty chunk of code but I'll try to use it. So from what I gather that particular bit of code will create a rkhunter.dat file with the relevant inf. What would be the best way to extract the distro codename? I was thinking of using cut, since I could have the colon be the delineator.

Quote:
Originally Posted by unSpawn View Post
IMHO making backups should be done unobtrusive, regularly and unattended. Unless one wants to watch *everything* that happens on the system sending an email after the backup job completed with the location of the file and it's MD5 or SHA1 hash would be more efficient.
What I can do is create a readme with the relevant info, and remove all of the echo commands from the script. I'll have the md5sum be put in the same place as the backup. What would be the best way to email the user the info? I'd have to ask for his email and input it into the configuration file with the exclusion items. Should I find some mail script to include in this set, or write my own? Would there be any dependency issues, for example if the user doesn't have sendmail installed?

Quote:
Originally Posted by unSpawn View Post
If you make a configuration item for exclusions then it is easier for users to exclude items.
Could you point me to a link to learn how to do that? How would I ask for the user's input?

Quote:
Originally Posted by unSpawn View Post
run 'stat -c "%a:%u:%g"'
What exactly does this do? Should I have it be
Code:
stat -c "%a:%u:%g" > permissions.dat

Quote:
Originally Posted by unSpawn View Post
This shouldn't be optional but a default configuration item. At least let backups be placed in a default directory you exclude on backup.
A possible solution would be to place the backup in /home/backup, and have the folder be excluded, instead of placing the file in the same location as the script.
Quote:
Originally Posted by unSpawn View Post
And then the backup files should eventually be moved off the system in case of file system or hardware failure.
The script creates a full system backup, and it's up to the user to be prepared and move it to a network drive or some other computer or offsite location.

Quote:
Originally Posted by unSpawn View Post
Creating a backup of the whole system will be slow, includes a lot of files that do not change often (/usr/sbin versus /home) or are uninteresting to the user (/var/cache versus /etc) and might DoS the system if not enough disk space is available, so be careful.
My main reason for creating this script is because many people I know are new to linux, and they often do something extremely stupid like delete the bin directory or rm -rf their system. The main idea is that this creates a complete backup, from which the users could extract files that they need or just restore the whole archive to their system. It's also meant to help out in the event of a system disk failure, since everything could just be copied back onto a new partition. The only thing that would have to be tweaked is the UUID of / in the fstab so that the system boots.
There are plenty of tools that backup just /home/user and etc, but when I implement the configuration file you mentioned the user can choose to exclude those folders on his own.

Thank you for your post, if you could help me out on the questions I had, I hope to make this script even better.
How is the restoresys.sh, is it reasonable?
 
Old 12-05-2010, 09:37 AM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,699
Blog Entries: 54

Rep: Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962Reputation: 2962
Quote:
Originally Posted by vaniaspeedy View Post
I took a look, it's a hefty chunk of code but I'll try to use it. So from what I gather that particular bit of code will create a rkhunter.dat file with the relevant inf. What would be the best way to extract the distro codename? I was thinking of using cut, since I could have the colon be the delineator.
The only thing rkh_dat_get_os_info() does is show you there are different file names (RELEASE variable), usually something along the .*version.* or .*release.*.. As you can see getting OSNAME per distro is easy but it needs some treatment afterwards. BTW I've only shown that as an example. You don't have to use it if you don't want to.



Quote:
Originally Posted by vaniaspeedy View Post
What I can do is create a readme with the relevant info, and remove all of the echo commands from the script.
That would be a good idea because a backup job should really not be left to the user but run regularly from cron (discipline issue).


Quote:
Originally Posted by vaniaspeedy View Post
What would be the best way to email the user the info? I'd have to ask for his email and input it into the configuration file with the exclusion items. Should I find some mail script to include in this set, or write my own? Would there be any dependency issues, for example if the user doesn't have sendmail installed?
You could make it a configuration switch? For instance with "MAIL=yes" you could opt for sending email (just make it the users responsibility to have their mail server configured well) and with "MAIL=no" you could log output to syslog (try "logger -t test "hello world", then check your syslog).


Quote:
Originally Posted by vaniaspeedy View Post
Could you point me to a link to learn how to do that? How would I ask for the user's input?
Well it's not asking but telling the user to configure things before running it:
Code:
# This allows the user to have the configuration file in another place:
CONF="/etc/myconfigfile.conf"
# If the file exists we can use it:
if [ -f "${CONF}" ]; then
 # ...but use a safeguard for those that didn't bother to configure things:
 grep -q "^IDIDNOTCONFIGURETHIS$" "${CONF}" && { echo "You didn't configure my app, exiting."; exit 127; }
 # ...so at this point all should be well and we read in the values by sourcing it with either "source" or "." in bash:
 source "${CONF}"
else
 # Woops, no config:
 echo "You didn't set a configuration file, exiting."; exit 127;
fi
# Commence with whatever you want to

Quote:
Originally Posted by vaniaspeedy View Post
What exactly does this do?
You could run it yourself and see the output: 'stat -c "%a %u %g" /proc'.


Quote:
Originally Posted by vaniaspeedy View Post
Should I have it be
Code:
stat -c "%a:%u:%g" > permissions.dat
Code:
# Given a list of directory names in the configuration file or script:
DIRS="/proc /media /lost+found /mnt /sys"
# ..and a configured backup or temporary directory:
BACKUPDIR="/backups"
# ...you could save permissions like this but note you only do that once if the "${BACKUPS}/permissions.dat" file doesn't exist:
[ -e "${BACKUPS}/permissions.dat" ] || for DIR in $DIRS; do echo "${DIR} $(stat -c "%a %u %g" "${DIR}")"; done > "${BACKUPS}/permissions.dat"
# ...but remember each mounted partition has its own "lost+found" (as in 'find / -maxdepth 2 -type d -name lost+found').
Code:
# ..and restore them like this (forget about IFS and spaces for now):
[ -e "${BACKUPS}/permissions.dat" ] && cat "${BACKUPS}/permissions.dat" | while read DIR MODE USER GROUP; do install -m $MODE -o $USER -g $GROUP $DIR; done


Quote:
Originally Posted by vaniaspeedy View Post
The script creates a full system backup, and it's up to the user to be prepared and move it to a network drive or some other computer or offsite location.
Sure, that's OK, but the user should be aware there's no "magic" meaning that if she/he tries to back up an installation of say 40GB on say a 60GB disk that things will fail (horribly, depending on partition scheme). The way I look at it most SOHO installations probably are ninety per cent inert, meaning the majority of files on the system belong to an installed package. So if you would create a list of installed packages the only thing you would need to backup are /etc (or modified configuration files), /home and /var (if one wants to keep logs and say caches). On restore you would run a networked installation, retrieve packages from the list and after that dump the tarball over / (another thing to look at could be incremental backups) but maybe all of that is a wee bit to advanced for now...


Quote:
Originally Posted by vaniaspeedy View Post
How is the restoresys.sh, is it reasonable?
Looks OK, not much that can go wrong there, but I wouldn't use "-v" because dumping a lot to the console tends to slow things down. At least for me it does.


All in all a nice effort. I think it's a good start for enhancing your scripting skills if this is something you will use yourself. If there's anything I should share with you about shell scripting it's that you should never script as root (especially when testing potentially destructive features) until you made certain that things work out OK, test things before using them ('man test'), know how to debug ('/bin/bash -vxe /path/to/scriptname.sh') things and most of all: have fun!


Code:
function help() { echo "Bash scripting guides:
http://www.tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html 
http://www.tldp.org/LDP/Bash-Beginners-Guide/html/index.html 
http://wooledge.org/mywiki/BashFAQ?action=show&redirect=BashFaq
http://www.tldp.org/LDP/abs/html/ ; }
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
What happens with backup-manager.org (very useful backup script)? Murz Linux - Software 3 07-27-2010 07:46 AM
Newbie trying to write a simple backup script to backup a single folder Nd for school stryker759a Linux - Newbie 2 09-16-2009 09:52 AM
Need help with script to organise files into folders as part of DVD backup script jasybee2000 Linux - Newbie 5 06-15-2009 08:29 PM
how to create backup MYSQL Script to backup my database for every 1hour RMLinux Linux - Newbie 3 11-20-2008 11:13 AM
Need a backup script enygma Linux - General 5 11-04-2004 04:49 PM


All times are GMT -5. The time now is 08:44 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration