LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   *BSD (http://www.linuxquestions.org/questions/%2Absd-17/)
-   -   Shell scripting question (simple) (http://www.linuxquestions.org/questions/%2Absd-17/shell-scripting-question-simple-659305/)

Leon W. Malinofsky 07-30-2008 01:56 PM

Shell scripting question (simple)
 
Please excuse the beginner question. I am new to shell
scripting. I have written a script to automate backups
of our FreeBSD server. This is the part of the script to
back up the root directory.


************************************************************
cd /mnt/root
echo "Backing up root directory-- / "
rm -Rf *
cd ..
rmdir /mnt/root/.*
rm -Rf /mnt/root
cd /mnt/root
dump -0 -a -C16 -L -f - /dev/da0s1a | restore -x -f -
************************************************************

"dump" finishes with this output, seeking a "y" or "n":

set owner/mode for '.'? [yn]

Now my question is how do I have the script notice this output
and respond with a "y" and a carriage return?

Also would like to know where to look up the answers to
similar questions. Many thanks.

chort 07-30-2008 03:57 PM

Why on Earth are you wiping out your previous backup before verifying that the new one completed successfully? If there's a power failure during backup, you're screwed.

As for dump, you could try prefixing it with
Code:

echo 'y' |

forrestt 07-30-2008 04:27 PM

You are also wasting a lot of time. You basically have a complete copy of your filesystem. You then remove the copy and create another. Unless EVERYTHING on your filesystem changes, this isn't the best way to do it. You can do the same with rsync much more quickly by running the following:

Code:

rsync -ax --delete / /mnt/root/
As far as finding answers to your questions, I suggest Google.

HTH

Forrest

jschiwal 07-30-2008 04:59 PM

A general answer on recognizing prompts and replying to them in a script is to use "expect".
If you simply want to reply "y" to any response add "< <(yes y)" to the end of the command.

For moving files, the "--reply=no" option is depreciated so "< <(yes n)" is used instead.

However, it would be better to perform incremental backups instead full backup every time.

anomie 07-30-2008 05:11 PM

@Leon W. Malinofsky: Please take the time to learn shell scripting well before taking on a project as critical as backups.

I agree with the suggestions that precede my post, but I'd add that you seem to be missing some key fundamentals (e.g. you cycle through several redundant rm operations, adding cruft and confusion to your script).

A great, free, online resource is: Advanced Bash-Scripting Guide

If you're writing scripts for FreeBSD, you're probably more likely to be using Bourne (sh), so keep in mind that some bash functionality may not present in sh.

A good book for learning Bourne shell scripting is: Sams Teach Yourself Shell Programming in 24 Hours by Sriranga Veeraraghavan

Read 'em, practice, and understand what you're doing.

Mr. C. 07-30-2008 05:21 PM

Quote:

Originally Posted by Leon W. Malinofsky (Post 3231213)
cd /mnt/root
echo "Backing up root directory-- / "
rm -Rf *

Let's also comment on how dangerous this is!

If /mnt/root either does not exist, or the user running the script does not have permission to cd into /mnt/root, ALL files and directories from the current directory will be removed recursively. If you are in /, say goodbye to your system. If you are in your home directory, say goodbye to all your files and directories.

NEVER use rm -rf * in an uncontrolled fashion. Better here to use one of:

rm -rf /mnt/root/*
find -depth /mnt/root | xargs rm -rf
cd /mnt/root && rm -rf *

There are other approaches too.

frob23 07-31-2008 12:46 AM

Long ago, in the before time, I wrote a quick and dirty backup script for my server. These backups are only the system critical areas (/home and /tmp are ignored but backed up elsewhere). In this case, /usr/data is on a secondary drive. Every so often, I would create a restore CD for this system and part of that CD would be these dumps (the level 0 dumps would all fit on the main CD and the secondary dumps would go on a second if needed although they usually fit too).

This quickly became less quick and more dirty than I liked, but it's what it is. It is still being run every night... although I haven't used these backups in ages. This is for a NetBSD machine but could easily be modified for other machines (with a change in the device names).

Code:

#!/bin/sh

FILESYSTEMS="wd0a wd0f wd0g"  # /, /var, /usr
LOCATION="/usr/data/dmps"

perfdump() {
        for file in $FILESYSTEMS
        do
                DUMPFILE="${LOCATION}/L${LEVEL}.N${NUMBER}.$file"
                echo "Dumping ${file}: /dev/$file to $DUMPFILE"
                dump -${LEVEL}au -h 0 -f $DUMPFILE /dev/$file
        done
}

dayofmonth=`date | awk 'BEGIN{FS=" "}{print $3}' -`
if [ dayofmonth = "1" ] ;then
        LEVEL="0"
        NUMBER="0"
else
        case `date | awk 'BEGIN{FS=" "}{print $1}' -` in
                Sat)    LEVEL="9"
                        NUMBER="1"
                        ;;
                Sun)    LEVEL="9"
                        NUMBER="2"
                        ;;
                Mon)    LEVEL="9"
                        NUMBER="3"
                        ;;
                Tue)    LEVEL="9"
                        NUMBER="4"
                        ;;
                Wed)    LEVEL="9"
                        NUMBER="5"
                        ;;
                Thu)    LEVEL="9"
                        NUMBER="6"
                        ;;
                Fri)    LEVEL="5"
                        NUMBER=`dc << END
                                $dayofmonth 1-7/pq
                                END`
                        ;;
                *)      echo "Error: something went wrong with the date!"
                        exit 1
                        ;;
        esac
fi

perfdump

I admit it's not perfect and it's ugly. But it basically works like so:

First the day of the month is checked... if it's the 1st... a new level 0 dump is done.
If not, then the day of the week is checked. Saturday through Thursday an incremental dump is done for any files that changed in the last 24 hours. On Friday, a weekly dump is done for any files that changed in the last week. Basically, it performs a bunch of incremental dumps. The filenames are ugly and hard to comprehend. I have only had to restore from these once and it was manageable but not fun.

Basically, the weekly dumps are like L5.N0.wd0a (week 1) -> L5.N1.wd0a (week 2) -> L5.N2.wd0a (week 3)

I've since taken to using rsnapshot (which uses rsync) to backup all my desktop machines and user directories... but this one machine still does these dumps every night. Although it's not excessively tested, it does work. And it has worked without fail for years.

At one point, I must have been planning to allow user input because I had a function:
Code:

usage() {
        echo "Usage: $0 {0-9} {number}"
}

I cut this out of the above code because it's pointless. You know, I think I might have started with manual dump level selection (forcing cron to include the dump level and number) but later automated that. And the "usage()" function was never noticed until just now and has remained littering the code for years.

Note: I was reading through this and have no idea why I have a trailing '-' at the end of the awk statements since awk appears to default to stdin if no file is specified. I can only imagine that it was habit or perhaps a previous version was more picky (this code had been in place since NetBSD 1.5 or sometime around then).


-----

It should be noted: although this system still uses dump it corrects some of the problems with the script in the original post. We've removed all the destructive "rm" commands... which are really scary to have in a backup script. It would be much better, if they are required, to use something like Mr. C recommended for safety.

We also have incremental backups.

I also see absolutely no reason to have the backups unpacked on the backup disk. Just leave them in the dump form. restore(8) has an interactive mode. For example, if you wipe out all of the files that start with s in /etc (tsk, tsk... you shouldn't drink and rm... although it happens to the worst of us):
Code:

cd /
restore -i -f /usr/data/dmps/L0.N0.wd0a
restore > cd etc
restore > add s*
restore > extract
You have not read any tapes yet.
Unless you know which volume your file(s) are on you should start
with the last volume and work towards the first.
(Use 1 for the first volume/tape, etc.)
Specify next volume #: 1
restore > quit

So you wouldn't need to have them unpacked to get at specific files. Although you will need to check the incremental dumps as well as the level 0 to make sure none of these files have changed recently... start with the weeks... and then the days.

Edit: The intent of these dumps was to create a timely and up-to-date recovery method should the drive in this server fail or the whole server go down. Since this server was acting as the gateway for the entire house as well as had some other semi-critical tasks. With these dumps, I could restore the old system and configuration in a matter of minutes (well under an hour in a test run), and have very minor or no tweaking to get the new machine in place. This is why it's only the critical areas of the drive. I used dump and restore mainly because these are available on the NetBSD install kernel image (which is what I was using on my recovery CD).

What you intend to do with your dumps and how critical they are will effect what sort of script you desire. But hopefully this gives you a few ideas.

frob23 07-31-2008 01:44 AM

Oh, and to note: in the worst case scenario, you're restoring from a Thursday near the end of the month. So it's possible to have something like.

restore -rf L0.N0.wd0a
restore -rf L5.N0.wd0a
restore -rf L5.N1.wd0a
restore -rf L5.N2.wd0a
restore -rf L5.N3.wd0a
restore -rf L9.N1.wd0a
restore -rf L9.N2.wd0a
restore -rf L9.N3.wd0a
restore -rf L9.N4.wd0a
restore -rf L9.N5.wd0a
restore -rf L9.N6.wd0a

Now that's ugly. I would usually cheat in this case:
Code:

foo() {
while read LINE
do
    restore -rf $LINE
done
}
ls *.wd0a | foo

Of course, I would check how that expanded before I ran it... and also check the dates and rename or remove all old dumps (like L5.N4.wd0a if it's hanging around from a couple months ago -- which does happen)... but this should do all the heavy lifting. The majority of restores are much simpler than that... but anything can happen.

Leon W. Malinofsky 07-31-2008 10:48 AM

Thanks to all
 
THANK YOU sincerely to all who responded. I didn't think my system had rsync but when I saw it was available in /usr/ports/net I installed it immediately and Life Is Good. And now I have the titles of some good
reference books. All comments were **most** useful.

Again, my thanks.


All times are GMT -5. The time now is 07:12 PM.