LinuxQuestions.org - [SOLVED] Automated remote backup of laptop using RSnapshot

- Slackware (https://www.linuxquestions.org/questions/slackware-14/)

- - Automated remote backup of laptop using RSnapshot (https://www.linuxquestions.org/questions/slackware-14/automated-remote-backup-of-laptop-using-rsnapshot-810009/)

Automated remote backup of laptop using RSnapshot

Hi! I'm searching for real Linux Gurus! I have a machine on my network and that machine is a mass storage server that I will eventually use as a media server (to stream movies, videoclips and music on my home theater system). I use slackware 13 on ALL of my machines.

I am trying to automate the backup of the "/home" folder of my laptop onto the mass storage server. I currently use rsnapshot and it works great, but I would like to automate the whole process, even if I am not home or in front of my machines... basically, I assume that is possible and I am sure some of you guys already doing this!!!

Here's what I imagined (in pseudo code):

1) Poll if server is active (up);
1.1) If not:
1.1.1) Wake up the server (WOL);
1.1.2) Wait for the server to boot;
1.1.3) Confirm the server has made it to the login prompt (normal boot);
1.1.3.1) If not, send an alarm via email;
1.1.3.2) If so, continue to step 1.2.1 (below)
1.2) If so:
1.2.1) Mount NFS share on local directory;
1.2.2) Rsync /home folder on NFS share;
1.2.3) Unmount NFS share;
1.2.4) Generate confirmation of successful completion on syslog server (same machine as NFS share)
2) If currently between 10PM and 5PM:
2.1) Shutdown server (means that I'm either sleeping or not home);
else
2.2) If server was down;
2.2.1) shutdown server (means that I'm not home);
else
2.3) If server was up:
2.3.1) Keep it up! (means that I'm using it for media streaming);

Any idea or recommendations, please share with me!!

Thanks!!

Real Linux gurus can be found here :)

Meanwhile, the high-level pseudo-code looks do-able and bash script would be suitable (as would several other scripting languages -- which do you know best?).

Regards "Poll if server is active" it might be simpler just to try mounting the NFS share because that is the functionality you want. By its network activity trying to mount the NFS share would effect "1.1.1) Wake up the server (WOL)" so this method would save a specific step to do so (KISS).

The script could sit in a time delayed loop trying mount the NFS share. If successful immediately note that server was not down. If not successful after a reasonable elapsed time do "1.1.3.1) If not, send an alarm via email" and give up.

For each part of the script I would become familiar with the necessary commands by running them at the command line and then program them into the script.

No point in writing more until you tell us which parts you know how to do. For the rest we can recommend some approaches and give you some keywords to search on and learn as required.

LQ seldom writes scripts to order; we prefer to help people do it themselves. Give a man a fish ...

Quote:

Originally Posted by lpallard (Post 3980750)

catkin, many thanks for your reply!

I guess I posted without indicating some valuable info.... I'm sorry about that. Yes I was refering to a bash script, basically a .sh file. Except that, in linux I dont know any other languages and in fact I'm pretty poor at bash scripting. I have no idea how to program logic loops such as if conditions, else, and simple loops (while)... So this is going to be a big, fun adventure for you! :) I hope you are up for spending a fair bit of time on it because it is a non-trivial script even for someone who is fluent in bash. There are a lot of good bash intros on the net. Maybe this one and more links here. This one looks a bit strange (I never heard of UDVs!) but might be useful.

As for the individual commands and their usage, here's what I know (at least entering these in a terminal ):

Poll if server is active = I guess a simple ping would do? The thing I dont know is how to interpret the answer received from that command (no ping VS ping ?) It would do. You could capture the ping output and use a bash case statement to find telltale strings in it. But why not simply try the NFS mount? How likely is it that the server would respond to ping but not be serving NFS?

Wake up the server (WOL) = I have installed WOL on my laptop so a simple wol MAC_ADDRESS_OF_SERVER would do? Dunno. I guess it has to be MAC address because the OS hasn't configured its NIC with an IP address. Have to do some netsearching on this to find how people have done it.

Wait for the server to boot = Sleep ?? Yes -- the sleep command.

Confirm the server has made it to the login prompt (normal boot) = I have absolutely no idea how to do that! What I'm trying to do here is to make sure the server booted up normally and has made it to the login prompt (in case it freeze during kernel load, kernel panics, corrupted partitions, etc, only god knows what can go wrong!) 99 times out of 100 it should just boot and start serving NFS. If you do find that some problems happen regularly then you could script solutions but better to do so later when you have found out what those problems are? Too much to analyse from the script and certainly for the version one script?

If not, send an alarm via email = No idea!! A whole new project! For version one the script could just write to a log file. Later you could mail locally (mail, mailx, mutt or pine depending on what is already intalled) and finally email to the Internet (sendmail, postfix ... complex beasts!).

Mount NFS share on local directory = I know how to do that (mount -t nfs IP_ADDRESS_OF_SERVER:/path/to/mount/directory Cool! Good that bit is familiar!

Rsync /home folder on NFS share = rsnapshot will take care of that!

Unmount NFS share = umount /local/mount/point

Generate confirmation of successful completion on syslog server (same machine as NFS share) = No idea!! Again version one could simply write to a log file and later the whizzy syslog stuff.

That is, at least for me, the black magic section :
==> If currently between 10PM and 5PM: = No idea!! The date command will be the one to use with a format string so it writes only the required data. Perfect example of something to familiarise yourself with at the command line before programming into the script.

Shutdown server = halt? Any better way to shutdown a slackware machine? halt is a bit vicious. shutdown -h now is kinder. You will probably use ssh to run that command on the server.

If server was down = No idea!! How do you validate if the machine was initially powerred up through step WOL or was already up? You could use a "flag" variable. The term "flag" refers to how you use it -- logic true or false; it would be an ordinary bash variable. Set it while bringing up the server and use it at this stage.

shutdown server = same as above

If server was up = No idea!! How do you validate if the machine was initially powered up through step WOL or was already up? "Flag" variable as above.

Keep it up! = I should figure out this one! ;) Ah reckons you can do that!

I also forgot to mention that I dont know how to handle the potential prompts for passwords (either locally or remotely) from for example umount... normally only root can umount. Running this script as root would do it right? Right. And that is what I would do but there are people who would argue against for security. Does mean taking special care when writing the script. If anything does go wrong it is an ideal opportunity to test your disaster recovery plan!
Thanks!

Reply by annotation above.

Overall, I suggest an iterative approach. Version one of the script could be just the basics, without bells and whistles. Once that is working, progressively add refinements in order of importance as determined by experience using the script. Two exceptions to that: error trapping and logging. Firstly developing a good error trapping and logging framework from the beginning will aid in debugging and troubleshooting. A clean implementation can be easily extended to emailing later. Secondly using a single function to exit the script. That would allow the script to undo changes it has made that should be undone and facilitates logging a final message; it is also useful during debugging when you will be running the script interactively and may want to stop it using Ctrl+C.

catkin, I am building the script right now. I will let you know what happens soon!
Thanks!

catkin, I have a hardcopy of Advanced Linux Programming by "New riders" do you know this book? is tha any good?

Quote:

Originally Posted by lpallard (Post 3984823)

catkin, I have a hardcopy of Advanced Linux Programming by "New riders" do you know this book? is tha any good?

I do not know it but found it online here and downloaded the Processes chapter. Nice clear introductory text but does not explain what a process is from the kernel's side which is important for understanding environment variables, security contexts, open file descriptors ... . More importantly from your point of view it is for C and C++ programmers, not for shellscript and the task you have set yourself is better done in shellscript. It could be done in C but would be harder to write and debug. Why? Because you are primarily issuing commands, and that is what shellscipt is optimised for -- the "shell" in shellscript is a command shell.

Yeah I never really opened that book before, I tough it was for shell scripting.... sorry about that!

Hey catkin! Like I said, I am writing my script right now, and so far, I am pretty satisfied! some stuff works, some not ( was to be expected right!?)

I have a problem with polling the server to see if it is alive. Like you said in one of your earlier posts, I can ping the server and grab what the command returns and interpret in a case statement or an "if" ...

Basically, I am using the following IF block to see if the machine is up & running (please note that $IP_BACKUP_SRV is a variable declared at the very beginning of the script and it is the IP of the server):

Code:

if  [ ping -c1 -w1 -q $IP_BACKUP_SRV >/dev/null = 1 ]; then

  echo "the server is alive"

else

  echo "the server is down"

fi

Whenever I run this block of code, I get:

Code:

./rsnapshot_script.sh: line 20: [: too many arguments

If I run the command above as a standalone command in a terminal, I get an answer (1). So it works.

??? Whats going on? If I understand correctly, this is a string comparison (not arithmetic right?) and a equal sign is supposed to do? If it was an arithmetical comparison, I would have to use -eq ?

Thanks !!!

Try using the exit status of the command.

Code:

ping -c1 -w1 -q $IP_BACKUP_SRV &>/dev/null

        exstat=$?

if [ ${exstat} -eq 0 ]; then

    echo "the server is alive"

else

  echo "the server is down"

fi

it works perfectly! I am still workingon the script but so far it looks good!

Hey guys!

I have a problem with a portion of my backup script. The following code should verify what time it is (on the local machine) and then if the time is between 4PM (16:00:00) and 10PM (22:00:00) keep the server alive, if not, shut it down... When I run the code, it always decide to shut down the server like if it was either before 4pm or past 10pm... Why? for example, I tried the code at 8pm and it didnt work.

Code:

date +%H%M%S  &>/dev/null

current_time=$?    



if [ $current_time -gt 160000 -a $current_time -lt 220000 ]; then



echo "between 4pm & 10pm so keeping the server up"

else

echo "outside the operation hours so shutting down"



fi

date +%H%M%S output the correct info (at the moment I tried, it was 8:42:36PM so the command output 204236

Thanks!

Quote:

Originally Posted by lpallard (Post 4010840)

Code:

date +%H%M%S  &>/dev/null

current_time=$?    



if [ $current_time -gt 160000 -a $current_time -lt 220000 ]; then



echo "between 4pm & 10pm so keeping the server up"

else

echo "outside the operation hours so shutting down"



fi

date +%H%M%S output the correct info (at the moment I tried, it was 8:42:36PM so the command output 204236

Thanks!

$? gets the return code from the last program -- usually 0 if it exited cleanly, and a non-zero value if there was an error. It does *not* get the *output* from the last program that ran. What you would really want is the following:

Code:

current_time=$(date +%H%M%S)

$(cmd) or `cmd` runs the command within parentheses or backticks (I usually use parentheses just because you cannot nest backticks). See "Command Substitution" in `man bash`.

I kinda have more questions that are related to this thread and did not want to open a new thread for the same stuff, so I marked this thread Unsolved.

My new question is pretty straight forward. I have 3X 5.25 SATA removable bays with hard drives in it. I am working on a new script to manually backup some very important data to those drives. Basically, to eliminate risks of burning the HDD's, or accidentally (either by me or my scripts) destroy data, I normally keep those drives offline, i.e. unpowered. The enclosures are Vantec EZ swap and require a key to turn them on/off. Once it is turned on, Slackware see them without a glitch. My script will basically display a message to warn that the backup is about to happen, then once the user click the OK button, mount the partition to a local folder and then backup the data. The partition will then be unmounted and a new message will be displayed to ask the user to "unplug" the HDD. Cron will take care of the schedule.

Basically, in pseudo code:

Code:

1-Verify if sdd is connected (powered)... If so, procee to #2, else display message box "The HDD is not connected"

2-Mount the partition sdd5 to the local folder

3-Backup

4-Unmount the partition from the local folder

5-Display a message box "The backup was completed with success.  You can now disconnect the HDD"

Now to the question, how can I automate the detection of the hdd? I would like the script to poll or probe the HDD to see if it was properly turned on, and if so, proceed with the rest... I tried all kind of stuff but no luck... For example, fdisk -l will list the drive & its partitions if it is connected (powered) and wont if the drive is not connected. I cant use the output of fdisk to efficiently accomplish what I am trying to do...

The second question is more complicated. I want the script to display the messages boxes (Please plug the HDD & You can now unplug the HDD) to a different display (i.e. my laptop). The reason is because the script will run on my server that has no monitor...

any idea? Thanks!!

bump!!! nobody got an idea on this one?