LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 01-07-2005, 12:16 AM   #1
it_slut
LQ Newbie
 
Registered: Jan 2005
Posts: 2

Rep: Reputation: 0
backup questions


Hi Guys,

I'm relatively new to the Linux scene, got a dedicated server to host some sites and have now setup RHEL 3 at my home/office for testing websites and for doing backups of my other computers.

I have just spent the past few day writing the following backup shell script (see end of post) to backup all my pc's running winblows. The script backups all the computers on a samba network to my linuxserver and then another script rsync's a copy to an external Hard Drive that will be stored offsite and rotated weekly.

I have a number of questions so I figured this would be a great place to ask..... so here goes.....

1. How would I create a separate partition on my hard drive called "backup" where I can store all my backups, without deleting any existing data? There is about 180MB free at the mo.

2. I would like to allocate about 120MB for this partition and will probably need to defrag it every 6 months or so, how do you do this in linux?

3. Rather than using cygwin, as I did want to have to install on all my networked computers, I am mounting my networked computers from my linuxserver via samba and then rsyncing the files from there. Is this a bad idea?

4. When using rsync I have tested with and without using compression (z). The compression seems to make the whole process take a lot longer, is there any benefit of using this on a home/small office network?

Thank you so much for reading this post and if you have any advice or suggestions it would be much appreciated,

Thanks

PS: If see a way to improve the code below then please let me know.

the config file

workstation_backup.config file

Code:
#!/bin/bash
# config file for workstation backup
# this file will be imported by tar_rsync_backup.sh

####### MAIN CONFIG START ##################

#create variable for computername
COMPUTERNAME="workstation"

#username to access remote computer
USERNAME="Administrator"
#password to access remote computer
PASSWORD="*********"

#most of the time you will want to use the $COMPUTERNAME but if you want it in a different directory name it here
BACKUPNAME=$COMPUTERNAME

#set up the directories
ARCHIVEDIR="/backup/$BACKUPNAME/archives"
LOGSDIR="/backup/$BACKUPNAME/logs"

#destination directory where the backup will be saved
DESTINATIONDIR="/backup/$BACKUPNAME/current_backup"

#mount directory
MOUNTDIR=/mnt/computers

#mountpoint directory reference to computer on samba network
MOUNTPOINT=$MOUNTDIR/$COMPUTERNAME

#base source directory on remote computer
BASESOURCEDIR="c_drive"

#create an array with all the directories to be backed up
#directory to be backed up on the computer use " "quotes for the directory name as this
#allows for spaces in the directory names 
SOURCEDIRS[0]="$MOUNTPOINT/Documents and Settings/master/My Documents"
SOURCEDIRS[1]="$MOUNTPOINT/Documents and Settings/master/Favorites"
SOURCEDIRS[2]="$MOUNTPOINT/Documents and Settings/master/Local Settings/Application Data/Identities/{2C7A8CD0-78FA-427F-BF86-AE333A20DC52}/Microsoft/Outlook Express"
SOURCEDIRS[3]="$MOUNTPOINT/Documents and Settings/master/Local Settings/Application Data/Microsoft/Outlook"
SOURCEDIRS[4]="$MOUNTPOINT/code"
SOURCEDIRS[5]="$MOUNTPOINT/Nokia"
SOURCEDIRS[6]="$MOUNTPOINT/tutorials & notes"
#SOURCEDIRS[7]="$MOUNTPOINT/Program Files/Macromedia/Flash MX/Configuration/Include
#SOURCEDIRS[8]="$MOUNTPOINT/Program Files/Macromedia/Flash MX 2004/en/First Run/Include"

#maximum archive number to keep before deleting old ones
MAX_ARCHIVES=3


####### MAIN CONFIG END #######################################

main script that tars old version and rsync new info

tar_rsync_backup.sh


Code:
#!/bin/bash
# backup script

# This script works as follows;
# 1. Import the main config variables from eg. workstation_backup.config file.
# 1. Archives the current_backup folder as a tar.gz and stores it in /archives/ directory eg. /archives/2005-01-12.tar.gz
# 2. Creates a mount point for the computer using samba and places it in /mnt/computers/
# 3. Run rsync on source directories array and transfer the lastest changes to current_backup directory

#import all the main config variables for doing the backup
#the backup or computer name is passed as a the 1st command line argument eg. tar_rsync_back.sh workstation
#this would then import eg. "workstation_backup.config"
source "/backup/sh/$1_backup.config"

echo "Backing up: $BACKUPNAME"

#command line variable to enable testing and debugging without transferring all the files each time
#rsync options from bash varible
if [ "$2" = "rsync_test" ]; then
 echo "rsync test run" 
 RSYNC_OPTIONS="n"
else
 echo "rsync real"
 RSYNC_OPTIONS=""
fi

#tar options from bash variable
if [ "$3" = "tar_false" ]; then
 echo "tar is disabled"
 TAR="false"
else
 echo "tar is enabled"
 TAR="true"
fi

echo "desination directory: $DESTINATIONDIR"

#check if the $BACKUPNAME is a directory in /backup/ if not create one and a logs, and archives folder
if [ ! -d "/backup/$BACKUPNAME" ]; then
 mkdir "/backup/$BACKUPNAME"
 mkdir "$LOGSDIR"
 mkdir "$ARCHIVEDIR"
fi

ARCHIVESOURCEDIR=$DESTINATIONDIR

#log file name will use the current date eg. 2005-01-05.log
LOGFILENAME=`date +"%F"`.log
echo "$LOGFILENAME"

echo "checking for last backup ....."
echo "archive source directory: $ARCHIVESOURCEDIR"
echo "archive destination directory: $ARCHIVEDIR"

#check if the destination directory exists otherwise make one
if [ ! -d $DESTINATIONDIR ]; then
 mkdir $DESTINATIONDIR
 echo "no backup found"
 echo "destination directory doesn't exist so created: $DESTINATIONDIR"
else

 #the destination directory exists so now we must archive it 
 
 echo "last backup found"
 echo "archiving last backup ......"
 
 #get the list of all archives from the archives directory and reserve sort via time
 #eg make the oldest appear first in the array[0]
 ARCHIVES_ARRAY=(`ls -rt $ARCHIVEDIR`)
 #do the same for the log files
 LOGFILE_ARRAY=(`ls -rt $LOGSDIR`)

 #total number of archives in the archive directory
 TOTAL_ARCHIVES=${#ARCHIVES_ARRAY[@]}
 echo "TOTAL_ARCHIVES: $TOTAL_ARCHIVES"
  
 #check if the total achives are equal to the max archive number
 #if so delete the oldest archive so we can have room for a new one
 if [ $TOTAL_ARCHIVES -eq $MAX_ARCHIVES ]; then
  echo "deleting oldest achive: ${ARCHIVES_ARRAY[0]}"
  rm -f $ARCHIVEDIR/${ARCHIVES_ARRAY[0]}
  
  #also delete its log file
  rm -f $LOGSDIR/${LOGFILES_ARRAY[0]}

 fi
 
 #get the date of the last backup from the last_backup.log file eg. 2005-01-04
 LASTBACKUPDATE=`cat $LOGSDIR/last_backup.log`
 echo "Logs Directory: $LOGSDIR"

 #archive name using the last backup date eg. 2005-01-04.tar.gz
 ARCHIVENAME=$LASTBACKUPDATE.tar.gz
 
 #check if tar has been disabled from the command line
 if [ $TAR = "true" ]; then

  #now create a tar archive with the date and time and compress with gzip
  tar -zcvf $ARCHIVEDIR/$ARCHIVENAME $ARCHIVESOURCEDIR 
  echo "created: $ARCHIVEDIR/$ARCHIVENAME"

  #IMPORTANT
  #pehaps in future remove gzip compression from the tar achive command (z)  and do it here instead
  #this way by using gzip on its own we can make it work with rsync friendly
  #add this in later. Though it has been said that rsyncing tar.gz files is not reliable.

 fi

fi


#set up the samba mount

echo "mountpoint: $MOUNTPOINT"
echo "checking mount point directory..."
#check if the computer's mount point directory exists in /mnt/computers
if [ ! -d $MOUNTPOINT ]; then
  #make the mount point directory in /mnt/computers
  mkdir $MOUNTPOINT 
  echo "creating mountpoint"
else
 echo "mount point directory exists"
fi

#mount the computer from samba
echo "creating mountpoint"
mount -t smbfs -o username="$USERNAME",password="$PASSWORD" //$COMPUTERNAME/$BASESOURCEDIR $MOUNTPOINT

#loop through all the source directorys and rsych them
#put "" quotes around the directory names variables so they won't spilt
for i in "${SOURCEDIRS[@]}"
do
 echo "backing up directory: $i"
 
 #get the name of the current source dir without all the other stuff eg.Outlook Express
 SOURCEDIRNAME="${i##*/}"

 echo "destination name: $DESTINATIONDIR/$SOURCEDIRNAME"
  
 #rsync the current source directory to the destination directory
 rsync -"$RSYNC_OPTIONS"va --delete "$i" "$DESTINATIONDIR"

done

#un mount the mount point
umount $MOUNTPOINT

#write the today's date to last_backup.log so we know the last date a backup was made
#we will use this date for archiving when this script is run again
date +"%F" > $LOGSDIR/last_backup.log

exit
 
Old 01-07-2005, 03:54 AM   #2
rjlee
Senior Member
 
Registered: Jul 2004
Distribution: Ubuntu 7.04
Posts: 1,990

Rep: Reputation: 67
Re: backup questions

Quote:
Originally posted by it_slut
Hi Guys,

I'm relatively new to the Linux scene, got a dedicated server to host some sites and have now setup RHEL 3 at my home/office for testing websites and for doing backups of my other computers.

I have just spent the past few day writing the following backup shell script (see end of post) to backup all my pc's running winblows. The script backups all the computers on a samba network to my linuxserver and then another script rsync's a copy to an external Hard Drive that will be stored offsite and rotated weekly.

I have a number of questions so I figured this would be a great place to ask..... so here goes.....

1. How would I create a separate partition on my hard drive called "backup" where I can store all my backups, without deleting any existing data? There is about 180MB free at the mo.
Be careful here. If you back up all the data from a disk onto the same disk, then this won't help you when the disk dies. Also consider what would happen in the event of, say, a fire taking down your LAN.

You can create, resize and move partitions using gnuparted (http://www.gnu.org/software/parted/)

Quote:
2. I would like to allocate about 120MB for this partition and will probably need to defrag it every 6 months or so, how do you do this in linux?
Defragmentation is one of the most annoying features of some old, particularly FAT-based, filesystems. If you use a modern filesystem, like ext2fs, ext3fs, reiserfs or JFS then they tend to avoid fragmenting during use.

If you really find that you need to defragment files on the drive, you can do this by moving all the files off the drive (onto another partition of the same type, so that file attributes are not lost), and then moving them back. I've run a Linux machine off an ext2 partition for several years of quite heavy use without ever needing to do this.

Quote:
3. Rather than using cygwin, as I did want to have to install on all my networked computers, I am mounting my networked computers from my linuxserver via samba and then rsyncing the files from there. Is this a bad idea?
This is a valid solution to the problem. The only thing that you need to be aware of is that SMB (the Windows sharing protocol) was designed to use FAT32 devices; in particular, these are case-insensitive (it doesn't do very well when you have a file called a.txt and another called A.TXT in the same directory) and they do not have all of the permissions information that a modern UNIX-type filesystem would assign to them.

Quote:
4. When using rsync I have tested with and without using compression (z). The compression seems to make the whole process take a lot longer, is there any benefit of using this on a home/small office network?
Compression reduces the network bandwidth used by the computers. It sounds like bandwidth is not your bottleneck, so I would avoid the z option unless you want to use the bandwidth for something else, like playing networked games or downloading big files, and you are more interested in the lag that your backups are inducing.

Last edited by rjlee; 01-10-2005 at 04:56 AM.
 
Old 01-10-2005, 04:52 AM   #3
it_slut
LQ Newbie
 
Registered: Jan 2005
Posts: 2

Original Poster
Rep: Reputation: 0
Hi rjlee,

Thanks for the quality advice.

From your own experiences it sounds like the Linux file systems keep them selves in good order. Must say that's a nice change from windows!

Thank you again for your help,

much appreciated
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Partimage backup questions jrdioko Linux - Newbie 11 09-24-2004 04:22 PM
CD-RW/ Verify backup questions. PraxisArch Linux - Hardware 1 08-06-2004 06:57 PM
Doin a weekly tape backup - multiple questions - VERY important - Please help WorldBuilder Linux - General 10 04-25-2003 03:12 PM
Backup questions. cottonmouth Linux - General 1 11-21-2002 02:58 PM
2 questions(Re: backup to scsi tape drive) ascii2k Linux - General 5 07-16-2001 05:47 PM


All times are GMT -5. The time now is 09:58 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration