Slackware total Backup and recovery DIY HOWTO

SCerovec · 09-26-2007, 07:47 AM

This one is intended for Slackware specific locations and tools.

I distinguish:

1. Distro data (system):

Quote:

The data than be recovered from the installation media, but without the default configuration.
Additional media repositories with unsupported packages also.

If I wanted the default configurtion I would use clean install and not recovery.

2. Distro settings & config (config)

Quote:

The data in /etc and in /var to some extent and /dev static entryes.
I could loose eg. the old logs in the backup.

3. Tunning & add ons (custom)

Quote:

The software sources and the software derived from the sources.
Usually /usr/local/
and
/lib/modules + /boot + /usr/src
for the custom kernels & paches

4. User data (payload)

Quote:

The actuall payload of the system.
This includes /home /srv /var/www /var/named

5. The data that is regenereted automatically (state)

Quote:

/tmp
/sys
This has nothing of value for backup.

So, the actual backup should take:
2. (config)
3. (custom)
4. (payload)

The recovery should deliver:
1. (system)
2. (config)
3. (custom)
4. (payload)

Now the main question:
How could it be automated, so an advanced user could choose to only restore a specific category.
For example, if I knew my payload is intact, the settings are ok but some files of the system are missing-damaged (eg. FSCK-ed), I would to only restore missing and differing files back to the filesystem.

How would I sweep the whole FStree for missing files (bash command) and how would I replace them from the source media?

Bruce Hill · 10-26-2007, 06:46 PM

When I got here:

Code:

The data in [red]/etc[/red] and in [red]/var[/red] to some extent and [red]/dev[/red] static entryes.

it just became too confusing to read. You must use color= to turn it on, and /color to turn it off, eg: duh

SCerovec · 10-27-2007, 07:55 AM

Quote:

Originally Posted by Bruce Hill

When I got here:

Code:

The data in [red]/etc[/red] and in [red]/var[/red] to some extent and [red]/dev[/red] static entryes.

it just became too confusing to read. You must use color= to turn it on, and /color to turn it off, eg: duh

Sorry to see You had any trouble. I updated the tags accordinly. Thanks for the note.
I started to feel none's slackin' any more :-(

H_TeXMeX_H · 10-27-2007, 08:37 AM

Well, I'm sure it's useful, it's just that I don't back up my system files, only my personal files. When a new Slackware comes out, I wipe the drive and re-install. This also forces me to make more backups ... which is a good thing.

rkelsen · 10-27-2007, 08:50 AM

Quote:

Originally Posted by SCerovec

How would I sweep the whole FStree for missing files (bash command) and how would I replace them from the source media?

There is no easy way of doing this. A couple of years ago, I tried to do exactly the same thing and ended up writing a custom script. Apologies in advance.

Code:

#!/bin/bash
echo "Creating lists from physical files" 
find / |sort |sed 's:^/::' |sed 's/\/$//' > a
find /home |sort |sed 's:^/::' |sed 's/\/$//' > b
find /etc |sort |sed 's:^/::' |sed 's/\/$//' > c
find /var |sort |sed 's:^/::' |sed 's/\/$//' > d
find /lib/modules |sort |sed 's:^/::' |sed 's/\/$//' > e
find /dev |sort |sed 's:^/::' |sed 's/\/$//' > f
find /proc |sort |sed 's:^/::' |sed 's/\/$//' > g
find /sys |sort |sed 's:^/::' |sed 's/\/$//' > h
find /tmp |sort |sed 's:^/::' |sed 's/\/$//' > i
find /mnt |sort |sed 's:^/::' |sed 's/\/$//' > j
find /boot |sort |sed 's:^/::' |sed 's/\/$//' > k
find /root |sort |sed 's:^/::' |sed 's/\/$//' > l
find /usr/local |sort |sed 's:^/::' |sed 's/\/$//' > m
find / -type l |sort |sed 's:^/::' |sed 's/\/$//' > n
find / -type d |sort |sed 's:^/::' |sed 's/\/$//' > o
find /usr/src |sort |sed 's:^/::' |sed 's/\/$//' > p
echo "Creating list from package database"
cat /var/log/packages/* |sort -u |sed 's:^/::' |sed 's/\/$//' > z
echo "Removing physical but unpackageable items from lists"
comm -2 -3 a b > 1
comm -2 -3 1 c > 2
comm -2 -3 2 d > 3
comm -2 -3 3 e > 4
comm -2 -3 4 f > 5
comm -2 -3 5 g > 6
comm -2 -3 6 h > 7
comm -2 -3 7 i > 8
comm -2 -3 8 j > 9
comm -2 -3 9 k > 10
comm -2 -3 10 l > 11
comm -2 -3 11 m > 12
comm -2 -3 12 n > 13
comm -2 -3 13 o > 14
comm -2 -3 14 p > x
echo "Comparing physical to packaged"
comm -2 -3 x z > unpackaged
comm -2 -3 z a > y
comm -2 -3 y o > missing
#rm a b c d e f g h i j k l m n o 1 2 3 4 5 6 7 8 9 10 11 12 13 x y z
echo "All done"
echo ""
echo "View 'unpackaged' and 'missing' files for details"

I'd suggest putting this script in it's own directory and running it from there, since it creates a few files.

It ain't perfect, but it got me close enough to what I wanted.

Bruce Hill · 10-27-2007, 08:55 AM

Thanks for fixing your code tags. One other thing worth mentioning. Reading your thread title, "Slackware total Backup and recovery DIY HOWTO", along with the message icon, led me to believe you'd written a HOW-TO. It never occurred to me that you might be asking for advice.

We don't backup like that, either, but will show you what we do.

A good friend and Master Slacker (TM) has helped me immeasurably. He helped me setup my LAN, and helped with scripts. We have a small server here, and every night it backs up the user's home directories on the comps in the LAN with a cron job. Here is the line in the cron job for this computer:

Code:

#This is to backup silas/mingdao every morning at 1:00 a.m.
00 1 * * * rsync -va --exclude=WinXP.img --exclude=scratch.img -e "ssh -o BatchMode=yes -o user=root -i /root/.ssh/identity.wired_silas"    wired_silas:~mingdao /backup2/silas/

You can write a similar cron job to the backup destination of your choice for your computer(s). A script would be better ... and there are some old Slackers here who can help with that, although you might be better off in the Programming Forum for that kind of advice.

Edit: You might want to consider Eric's rsnapshot backup solution.

reikyv · 10-27-2007, 09:19 AM

I would agree that it is more easy and simple to just run a rsync script to backup all your necessary files and folders. This is what I did also, just write a simple shell script and define all your important files and folders there, and let rsync do the job :-p.
And also, there's some open source gui backup tool available, e.g. backuppc. Anyway, to me, command line tool is more fast and easy to deploy and maintain. :-)

SCerovec · 10-29-2007, 04:24 AM

Quote:

Originally Posted by Bruce Hill

Thanks for fixing your code tags. One other thing worth mentioning. Reading your thread title, "Slackware total Backup and recovery DIY HOWTO", along with the message icon, led me to believe you'd written a HOW-TO. It never occurred to me that you might be asking for advice.

Bruce, sorry to dissapoint You in the 1st pass, but as far as I can see, this thread actually IS rolling toward a 'HOWTOISH' one...

Thanks for the other posters too, for the contributed expamples.

@rkelsen:
That's the way I was thinking toward, Yet I hoped that there might be less time-consuming option (I think like @450MHz => 1h solution) for the system.

SCerovec · 10-29-2007, 04:36 AM

I would have a backup that is flexible. The typical scenarios for recovery mostly include:

1. Total disaster + clen install (most case for 1st time payload 100% lost)
2. Disaster + Clean install + payload restoration (my favorite)
3. Partial disater + clean install (/home was separate from / )
4. Partial disaster + partial restoration (state is or is not restored, all else IS restored)
5. Partial disaster + Clean install + total restortion (the clean system is overrun with the backup)
6. Partial disaster + inteligent restoration (system, config and payload restored where missing only (the aim of this howtoish thread

),
7. Micro disaster + inteligent restoration (intelligent restoratio of few missing files (rcinit, rc.local or /etc/hosts and similar...)

The above scenarios have different properties. What I'm thinking about is:
6. and 7. must be faster than 3. 2. or it makes no sense.

So our benchmark is restoration time compared to clen-install time.

shadowsnipes · 10-30-2007, 12:30 PM

I also use rsync for my backups. I simply wrote a script that accepts some args and has some interactivity that lets me select what I want to backup, etc. A general overview of my method is below.

I backup my computer to an external HD so the first part of my script mounts the file systems I need. With no command line args the script simply does a dry run for all backups and logs the results. I then check the logs to make sure I am happy with the results. If I am I run the -all switch to back everything up. I can also enter a different mode with the -s switch where I can select only certain things to backup. At the end of the script I give the option to umount the partitions that were previously mounted. All results are saved to a log file with the date at the front of the filename. Other files are used to list exclusions. I keep the script and all related files (exclusions, logs) in a tiny little folder in /root (root should be the only one running it in my opinion anyways).

I haven't yet written a restore routine, but it would basically be the same thing in reverse.

My solution is not something I would publish for everyone to use as it is specifically written for my system needs and preferences. However, it could easily be modified for use on someone else's system. As such, it could be a good starting point for playing around with rsync based backups.

SCerovec · 05-09-2008, 09:37 AM

Anyone saw a OS/X doing checkup of system?
It has a sort of a database of files and their respective MD5 sums (or SHA1?)
It just runs over most files and compares their md5 sums with a file-list from the install media...

I thought it was impressive - simple, accurate yet fast...

shadowsnipes · 05-09-2008, 01:39 PM

Quote:

Originally Posted by SCerovec

Anyone saw a OS/X doing checkup of system?
It has a sort of a database of files and their respective MD5 sums (or SHA1?)
It just runs over most files and compares their md5 sums with a file-list from the install media...

I thought it was impressive - simple, accurate yet fast...

like tripwire for linux...

SCerovec · 05-12-2008, 03:09 AM

then I shall try and learn tripwire now ...

off I go for homework...
:-)