shell backup script help needed

Quakeboy02 · 04-25-2007, 08:55 PM

I have a simple backup script that I use for my root drive. I don't like the way the for loop is setup. Is there a way to put these excluded directories into an array to clean it up? It would also clean up where I make the dummy directories. Heck, maybe the whole thing can be improved, I dunno? It does require a lot of disk space, but space is cheap these days.

Code:

#!/bin/sh
rm -rf tmp
mkdir tmp
cd tmp
for i in `ls / |grep -vw data |grep -vw data2 |grep -vw proc |grep -vw dev |grep -vw sys |grep -vw musem | grep -vw wendy`
do
echo /$i
cp -axu /$i .
done
mkdir data
mkdir data2
mkdir proc
mkdir dev
mkdir sys
mkdir musem
mkdir wendy
echo "tar --bzip2 -cf ../backup.tar.bz2 ."
tar --bzip2 -cf ../backup.tar.bz2 .
cd ..

jlinkels · 04-25-2007, 09:45 PM

I don't quite get it.

It seems that you list a lot of path/file names, copy them to directories in your /tmp and then tarring them. Right.

You also say that you have an exclude list, but I only see files which are INcluded.

Anyway... if the files you want to include are easier to specify than the files you want to exclude, specify the files to backup, otherwise use --exclude for exluding patterns.

Also, it seems easier to simply specify what you want to backup:
tar -Cf mytarfile.tar /data/ /data2/ ....

The entire `ls` operation can be skipped.

You might also want to take a look at the "star" backup program, which is a sort of extended "tar". It comes with about 2654 different options to specify what you want to backup, log or skip.

Last but not least, bzipping an archive makes the process *much* slower.

jlinkels

Quakeboy02 · 04-25-2007, 09:59 PM

Quote:

You also say that you have an exclude list, but I only see files which are INcluded.

The -vw option to grep excludes whatever is listed, so that the script copies everything except the /data, /data2, /proc, /dev, /sys, /musem, and /wendy directories.

Quote:

Anyway... if the files you want to include are easier to specify than the files you want to exclude, specify the files to backup, otherwise use --exclude for exluding patterns.

I didn't have any luck excluding whole directory trees with the --exclude option, but maybe I wasn't using it right. For example, how would I setup a tar to exclude the directory "/data" but not to exclude the directory "/home/data"? I just didn't understand this bit.

Quote:

Last but not least, bzipping an archive makes the process *much* slower.

Yeah, I know. The next time I run it, I'll probably switch to gzip.

The idea behind what I've done is that I can use a mindi boot CD and then just mount a backup CD and untar it onto the disk to get my running system back. This is for my test mule and I manage to bork it badly every few weeks; the latest being when I had problems with a Promise driver that wasn't recoverable.

jlinkels · 04-26-2007, 05:35 AM

Quote:

Originally Posted by Quakeboy02

The -vw option to grep excludes whatever is listed, so that the script copies everything except the /data, /data2, /proc, /dev, /sys, /musem, and /wendy directories.

Ah, never too old too learn.

Quote:

Originally Posted by Quakeboy02

I didn't have any luck excluding whole directory trees with the --exclude option, but maybe I wasn't using it right. For example, how would I setup a tar to exclude the directory "/data" but not to exclude the directory "/home/data"? I just didn't understand this bit.

Yes it works, but you have to put the quotes right. To the best of my knowledge it is:

tar --exclude="/data/*" --exclude="/data2/*" -cvf mytar.tar /

Using this command. you would skip /data, but not /home/data. I did not try it, but I don't think the "/" has to be escaped. (Like in \/home\/data\/*) I am extremely bad in remembering such details, but it becomes obvious fast enough when you try. I have backed up many root drives using the --exclude option though.

jlinkels

Quakeboy02 · 04-26-2007, 05:20 PM

Code:

tar --exclude="/data/*" --exclude="/data2/*" -cvf mytar.tar /

Thanks. I'll have to try it. It's clearly Much quicker than the way I do it, but mine creates /proc etc as dummies that then get put in tar. This clearly requires more thought on my part.

In case you're interested, here's what I wound up with where there's a clear exclusion list that's easily manipulated on a case by case basis. The resulting tar file has empty directories for each of the excluded ones.

Code:

#!/bin/sh
rm -rf tmp
mkdir tmp
cd tmp
exclude="data data2 proc dev sys musem wendy"
dirs=""
first=1
for thisdir in $exclude
do
if test $first -eq 1
then
first=0
dirs="$thisdir"
fi
dirs="$echo $dirs\|$thisdir"
done
for thisdir in `ls / | grep -vw $dirs`
do
echo /$thisdir
cp -axu /$thisdir .
done
for thisdir in $exclude
do
mkdir $thisdir
done
echo "tar --gzip -cf ../backup.tar.gz ."
tar --gzip -cf ../backup.tar.gz .
cd ..