Shell Script: Cross-platform backup for distributed servers
 

I admin a series of servers scattered around the internet. Each one has user data on it which needs to be backed up onto other systems. This is not an enterprise level arrangement at all, it's just something I've set up for my family so we can store those important files and documents we wouldn't want to lose in the case of a fire, for example.

I would like to write a shell script that can run on each machine and pull backups of all the other machine's data. Hard coding it would be very simple, but I would like it to be dynamic and automatically add new users as they appear.

Here is what I have in mind. Each user has a directory in a special folder, say /home/backup/<user>. In their home directory they could have a file (say .server) that lists the IP or DNS name of the server that contains their master data. So then I would need to write a script that scans the /home/backup folder (with ls for example) to determine which users to backup and then runs an rsync from <user>@<.server> to localhost.

I thought this would be something simple I could write and execute using a combination of bash and awk scripting, but my skills are weak, and I haven't been able to pull it off. My (first) two problems are that I can't seem to draw in the ls output and the contents of .server in a way that keeps them coordinated. I.e. I can get both stored in an awk variable, but they do not relate to one another. The other is that I have no way for the machine to realize that it is .server and skip the backup.

There's probably a vastly simpler way to do this, and I'd love to hear it. Also any scripting suggestions would be great!

Thanks,
Ted

Certainly sounds reasonable.
Seems like you'd want to run it locally ie from local box and fetch (via rsync+ssh) from the remote server. That'd be simplest.
(As opposed to push from server; server will prob always be up, but local box will vary ...)
Can you show us your code so far?

Hello tedcox. That's quite an interesting plan. You can perhaps send only the comments or outline of the scripts if you like. Then perhaps we can make some suggestions. Though I still prefer to see the whole code.

Sorry for the vast time delay here. I must have the wrong email on my LQ account, I didn't know there'd been any replies, and <shame> forgot about launching this thread. As I mentioned below, my shell scripting is awful. I'll try not to be too hurt if you laugh. :)

This was just my feeble attempt to get an ordered list user - server, but it doesn't come anywhere close. Right now, there's actually an awk error, but even at it's best I've never managed better than all the users and all the servers listed out separately.

Your thoughts are appreciated.

If you look at the bottom of this page you'll notice the "Similar Threads" feature. It could help you save time exploring wheels already invented. Sourceforge and Freshmeat also are a true cornucopia of production-ready wheels, partially invented wheels and concepts of flat round objects that could be tweaked to serve as wheels in no time. I am confident we can make your script work RSN, just wanted to mention wheels and reinventing them.