Visit Jeremy's Blog.
Go Back > Forums > Non-*NIX Forums > Programming
User Name
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.


  Search this Thread
Old 07-29-2006, 11:32 AM   #1
LQ Newbie
Registered: Jan 2004
Distribution: Fedora Core
Posts: 23

Rep: Reputation: 15
Question Shell Script: Cross-platform backup for distributed servers

I admin a series of servers scattered around the internet. Each one has user data on it which needs to be backed up onto other systems. This is not an enterprise level arrangement at all, it's just something I've set up for my family so we can store those important files and documents we wouldn't want to lose in the case of a fire, for example.

I would like to write a shell script that can run on each machine and pull backups of all the other machine's data. Hard coding it would be very simple, but I would like it to be dynamic and automatically add new users as they appear.

Here is what I have in mind. Each user has a directory in a special folder, say /home/backup/<user>. In their home directory they could have a file (say .server) that lists the IP or DNS name of the server that contains their master data. So then I would need to write a script that scans the /home/backup folder (with ls for example) to determine which users to backup and then runs an rsync from <user>@<.server> to localhost.

I thought this would be something simple I could write and execute using a combination of bash and awk scripting, but my skills are weak, and I haven't been able to pull it off. My (first) two problems are that I can't seem to draw in the ls output and the contents of .server in a way that keeps them coordinated. I.e. I can get both stored in an awk variable, but they do not relate to one another. The other is that I have no way for the machine to realize that it is .server and skip the backup.

There's probably a vastly simpler way to do this, and I'd love to hear it. Also any scripting suggestions would be great!

Old 07-31-2006, 01:29 AM   #2
LQ Guru
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,300

Rep: Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358
Certainly sounds reasonable.
Seems like you'd want to run it locally ie from local box and fetch (via rsync+ssh) from the remote server. That'd be simplest.
(As opposed to push from server; server will prob always be up, but local box will vary ...)
Can you show us your code so far?
Old 07-31-2006, 05:40 AM   #3
Senior Member
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
Hello tedcox. That's quite an interesting plan. You can perhaps send only the comments or outline of the scripts if you like. Then perhaps we can make some suggestions. Though I still prefer to see the whole code.
Old 09-05-2006, 06:48 PM   #4
LQ Newbie
Registered: Jan 2004
Distribution: Fedora Core
Posts: 23

Original Poster
Rep: Reputation: 15
Sorry for the vast time delay here. I must have the wrong email on my LQ account, I didn't know there'd been any replies, and <shame> forgot about launching this thread. As I mentioned below, my shell scripting is awful. I'll try not to be too hurt if you laugh.


ls -1 /home/vmail > /tmp/backup-ls.delete-me

cat /tmp/backup-ls.delete-me | awk -F'\n' '{

printf("SERVER=`cat /home/vmail/%s/.server` ; echo \"%s $SERVER\" >> /tmp/backup-servers.delete-me\n" $1, $1);


cat /tmp/backup-servers.delete-me | awk '{

printf("scp -rp /home/vmail/%s/* root@%s:/home/vmail/%s/\n", $1, $2, $1);

This was just my feeble attempt to get an ordered list user - server, but it doesn't come anywhere close. Right now, there's actually an awk error, but even at it's best I've never managed better than all the users and all the servers listed out separately.

Your thoughts are appreciated.
Old 09-05-2006, 07:36 PM   #5
Registered: May 2001
Posts: 29,335
Blog Entries: 55

Rep: Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535
If you look at the bottom of this page you'll notice the "Similar Threads" feature. It could help you save time exploring wheels already invented. Sourceforge and Freshmeat also are a true cornucopia of production-ready wheels, partially invented wheels and concepts of flat round objects that could be tweaked to serve as wheels in no time. I am confident we can make your script work RSN, just wanted to mention wheels and reinventing them.


rsync, scripting, shell

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell Script for backup BBQ_Matt Linux - Software 7 06-30-2005 05:19 PM
Cross Platform Shell narusegawa Linux - Software 2 06-24-2004 08:50 AM
Incremental Backup shell script. datadriven Linux - General 3 06-02-2004 09:19 AM
writting a backup shell script yenonn Slackware 2 03-18-2004 07:49 PM
bash shell backup script turnip Programming 0 04-03-2003 06:00 PM

All times are GMT -5. The time now is 10:03 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration