LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Other *NIX Forums > *BSD
User Name
Password
*BSD This forum is for the discussion of all BSD variants.
FreeBSD, OpenBSD, NetBSD, etc.

Notices

Reply
 
Search this Thread
Old 07-30-2008, 12:56 PM   #1
Leon W. Malinofsky
LQ Newbie
 
Registered: Aug 2007
Posts: 23

Rep: Reputation: 15
Smile Shell scripting question (simple)


Please excuse the beginner question. I am new to shell
scripting. I have written a script to automate backups
of our FreeBSD server. This is the part of the script to
back up the root directory.


************************************************************
cd /mnt/root
echo "Backing up root directory-- / "
rm -Rf *
cd ..
rmdir /mnt/root/.*
rm -Rf /mnt/root
cd /mnt/root
dump -0 -a -C16 -L -f - /dev/da0s1a | restore -x -f -
************************************************************

"dump" finishes with this output, seeking a "y" or "n":

set owner/mode for '.'? [yn]

Now my question is how do I have the script notice this output
and respond with a "y" and a carriage return?

Also would like to know where to look up the answers to
similar questions. Many thanks.
 
Old 07-30-2008, 02:57 PM   #2
chort
Senior Member
 
Registered: Jul 2003
Location: Silicon Valley, USA
Distribution: OpenBSD 4.6, OS X 10.6.2, CentOS 4 & 5
Posts: 3,660

Rep: Reputation: 69
Why on Earth are you wiping out your previous backup before verifying that the new one completed successfully? If there's a power failure during backup, you're screwed.

As for dump, you could try prefixing it with
Code:
echo 'y' |
 
Old 07-30-2008, 03:27 PM   #3
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
You are also wasting a lot of time. You basically have a complete copy of your filesystem. You then remove the copy and create another. Unless EVERYTHING on your filesystem changes, this isn't the best way to do it. You can do the same with rsync much more quickly by running the following:

Code:
rsync -ax --delete / /mnt/root/
As far as finding answers to your questions, I suggest Google.

HTH

Forrest
 
Old 07-30-2008, 03:59 PM   #4
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
A general answer on recognizing prompts and replying to them in a script is to use "expect".
If you simply want to reply "y" to any response add "< <(yes y)" to the end of the command.

For moving files, the "--reply=no" option is depreciated so "< <(yes n)" is used instead.

However, it would be better to perform incremental backups instead full backup every time.
 
Old 07-30-2008, 04:11 PM   #5
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
@Leon W. Malinofsky: Please take the time to learn shell scripting well before taking on a project as critical as backups.

I agree with the suggestions that precede my post, but I'd add that you seem to be missing some key fundamentals (e.g. you cycle through several redundant rm operations, adding cruft and confusion to your script).

A great, free, online resource is: Advanced Bash-Scripting Guide

If you're writing scripts for FreeBSD, you're probably more likely to be using Bourne (sh), so keep in mind that some bash functionality may not present in sh.

A good book for learning Bourne shell scripting is: Sams Teach Yourself Shell Programming in 24 Hours by Sriranga Veeraraghavan

Read 'em, practice, and understand what you're doing.
 
Old 07-30-2008, 04:21 PM   #6
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 59
Quote:
Originally Posted by Leon W. Malinofsky View Post
cd /mnt/root
echo "Backing up root directory-- / "
rm -Rf *
Let's also comment on how dangerous this is!

If /mnt/root either does not exist, or the user running the script does not have permission to cd into /mnt/root, ALL files and directories from the current directory will be removed recursively. If you are in /, say goodbye to your system. If you are in your home directory, say goodbye to all your files and directories.

NEVER use rm -rf * in an uncontrolled fashion. Better here to use one of:

rm -rf /mnt/root/*
find -depth /mnt/root | xargs rm -rf
cd /mnt/root && rm -rf *

There are other approaches too.
 
Old 07-30-2008, 11:46 PM   #7
frob23
Senior Member
 
Registered: Jan 2004
Location: Roughly 29.467N / 81.206W
Distribution: Ubuntu, FreeBSD, NetBSD
Posts: 1,449

Rep: Reputation: 47
Long ago, in the before time, I wrote a quick and dirty backup script for my server. These backups are only the system critical areas (/home and /tmp are ignored but backed up elsewhere). In this case, /usr/data is on a secondary drive. Every so often, I would create a restore CD for this system and part of that CD would be these dumps (the level 0 dumps would all fit on the main CD and the secondary dumps would go on a second if needed although they usually fit too).

This quickly became less quick and more dirty than I liked, but it's what it is. It is still being run every night... although I haven't used these backups in ages. This is for a NetBSD machine but could easily be modified for other machines (with a change in the device names).

Code:
#!/bin/sh

FILESYSTEMS="wd0a wd0f wd0g"  # /, /var, /usr
LOCATION="/usr/data/dmps"

perfdump() {
        for file in $FILESYSTEMS
        do
                DUMPFILE="${LOCATION}/L${LEVEL}.N${NUMBER}.$file"
                echo "Dumping ${file}: /dev/$file to $DUMPFILE"
                dump -${LEVEL}au -h 0 -f $DUMPFILE /dev/$file
        done
}

dayofmonth=`date | awk 'BEGIN{FS=" "}{print $3}' -`
if [ dayofmonth = "1" ] ;then
        LEVEL="0"
        NUMBER="0"
else
        case `date | awk 'BEGIN{FS=" "}{print $1}' -` in
                Sat)    LEVEL="9"
                        NUMBER="1"
                        ;;
                Sun)    LEVEL="9"
                        NUMBER="2"
                        ;;
                Mon)    LEVEL="9"
                        NUMBER="3"
                        ;;
                Tue)    LEVEL="9"
                        NUMBER="4"
                        ;;
                Wed)    LEVEL="9"
                        NUMBER="5"
                        ;;
                Thu)    LEVEL="9"
                        NUMBER="6"
                        ;;
                Fri)    LEVEL="5"
                        NUMBER=`dc << END
                                $dayofmonth 1-7/pq
                                END`
                        ;;
                *)      echo "Error: something went wrong with the date!"
                        exit 1
                        ;;
        esac 
fi

perfdump
I admit it's not perfect and it's ugly. But it basically works like so:

First the day of the month is checked... if it's the 1st... a new level 0 dump is done.
If not, then the day of the week is checked. Saturday through Thursday an incremental dump is done for any files that changed in the last 24 hours. On Friday, a weekly dump is done for any files that changed in the last week. Basically, it performs a bunch of incremental dumps. The filenames are ugly and hard to comprehend. I have only had to restore from these once and it was manageable but not fun.

Basically, the weekly dumps are like L5.N0.wd0a (week 1) -> L5.N1.wd0a (week 2) -> L5.N2.wd0a (week 3)

I've since taken to using rsnapshot (which uses rsync) to backup all my desktop machines and user directories... but this one machine still does these dumps every night. Although it's not excessively tested, it does work. And it has worked without fail for years.

At one point, I must have been planning to allow user input because I had a function:
Code:
usage() {
        echo "Usage: $0 {0-9} {number}"
}
I cut this out of the above code because it's pointless. You know, I think I might have started with manual dump level selection (forcing cron to include the dump level and number) but later automated that. And the "usage()" function was never noticed until just now and has remained littering the code for years.

Note: I was reading through this and have no idea why I have a trailing '-' at the end of the awk statements since awk appears to default to stdin if no file is specified. I can only imagine that it was habit or perhaps a previous version was more picky (this code had been in place since NetBSD 1.5 or sometime around then).


-----

It should be noted: although this system still uses dump it corrects some of the problems with the script in the original post. We've removed all the destructive "rm" commands... which are really scary to have in a backup script. It would be much better, if they are required, to use something like Mr. C recommended for safety.

We also have incremental backups.

I also see absolutely no reason to have the backups unpacked on the backup disk. Just leave them in the dump form. restore(8) has an interactive mode. For example, if you wipe out all of the files that start with s in /etc (tsk, tsk... you shouldn't drink and rm... although it happens to the worst of us):
Code:
cd /
restore -i -f /usr/data/dmps/L0.N0.wd0a
restore > cd etc
restore > add s*
restore > extract
You have not read any tapes yet.
Unless you know which volume your file(s) are on you should start
with the last volume and work towards the first.
(Use 1 for the first volume/tape, etc.) 
Specify next volume #: 1
restore > quit
So you wouldn't need to have them unpacked to get at specific files. Although you will need to check the incremental dumps as well as the level 0 to make sure none of these files have changed recently... start with the weeks... and then the days.

Edit: The intent of these dumps was to create a timely and up-to-date recovery method should the drive in this server fail or the whole server go down. Since this server was acting as the gateway for the entire house as well as had some other semi-critical tasks. With these dumps, I could restore the old system and configuration in a matter of minutes (well under an hour in a test run), and have very minor or no tweaking to get the new machine in place. This is why it's only the critical areas of the drive. I used dump and restore mainly because these are available on the NetBSD install kernel image (which is what I was using on my recovery CD).

What you intend to do with your dumps and how critical they are will effect what sort of script you desire. But hopefully this gives you a few ideas.

Last edited by frob23; 07-31-2008 at 12:03 AM.
 
Old 07-31-2008, 12:44 AM   #8
frob23
Senior Member
 
Registered: Jan 2004
Location: Roughly 29.467N / 81.206W
Distribution: Ubuntu, FreeBSD, NetBSD
Posts: 1,449

Rep: Reputation: 47
Oh, and to note: in the worst case scenario, you're restoring from a Thursday near the end of the month. So it's possible to have something like.

restore -rf L0.N0.wd0a
restore -rf L5.N0.wd0a
restore -rf L5.N1.wd0a
restore -rf L5.N2.wd0a
restore -rf L5.N3.wd0a
restore -rf L9.N1.wd0a
restore -rf L9.N2.wd0a
restore -rf L9.N3.wd0a
restore -rf L9.N4.wd0a
restore -rf L9.N5.wd0a
restore -rf L9.N6.wd0a

Now that's ugly. I would usually cheat in this case:
Code:
foo() {
while read LINE
do
    restore -rf $LINE
done
}
ls *.wd0a | foo
Of course, I would check how that expanded before I ran it... and also check the dates and rename or remove all old dumps (like L5.N4.wd0a if it's hanging around from a couple months ago -- which does happen)... but this should do all the heavy lifting. The majority of restores are much simpler than that... but anything can happen.
 
Old 07-31-2008, 09:48 AM   #9
Leon W. Malinofsky
LQ Newbie
 
Registered: Aug 2007
Posts: 23

Original Poster
Rep: Reputation: 15
Thanks to all

THANK YOU sincerely to all who responded. I didn't think my system had rsync but when I saw it was available in /usr/ports/net I installed it immediately and Life Is Good. And now I have the titles of some good
reference books. All comments were **most** useful.

Again, my thanks.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
simple scripting question jerf Linux - Newbie 4 09-19-2007 03:12 AM
simple question regarding shell scripting branche_dude Linux - Newbie 8 07-21-2006 02:35 AM
simple shell scripting problem noir911 Programming 11 03-14-2006 01:27 AM
Simple Shell Scripting Question hellomynameisphil Programming 2 08-27-2005 03:41 AM
simple scripting question wedgeworth Programming 6 02-04-2004 10:30 AM


All times are GMT -5. The time now is 05:50 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration