bash script HELP

GuitsBoy · 01-27-2004, 04:10 PM

Hi all -

Hopefully some kind soul can point me in the right direction. I have two mirrored webservers, one running FTP. I currently have a directory that is being monitored by dnotify. When something in that directory changes, dnotify runs a script, which invokes rsync to move the changes to the other server.

My problem is that dnotify will run the script hundreds of times while I am FTPing to this directory. Ofcourse, rsync really only needs to run once - at the end of the last file.

How can I make my script buffer when it runs rsync. I am thinking I would need to start a counter (maybe 2 seconds) and every time dnotify executes the scripts, have the counter reset (back to two seconds). When the timer finally runs out, the scripts executes rsync. Trouble is, I have no idea how to do this, or if this is even possible. I have very little shell scripting knowledge, so please be gentle.

Thanks so much,
-Tony

artur · 01-27-2004, 04:42 PM

I guess you could touch a file in temporary directory, and upon each subsequent execution of the script check how long ago the flag was touched. If it is more than two seconds ago, execure rsync and remove the flag file.

Something like:

Code:

#!/bin/bash

timeout=2
now=`date +%s`
last=`date -r /tmp/flag +%s 2> /dev/null`
count=$(($now - $last))
if [ $((count)) -gt $((timeout)) ]; then
   rsync 
   rm -f /tmp/flag
else
   touch /tmp/flag
fi

That's just a rough sketch, may have errorrs in it...

Or you could use find command and look for any files that have been modified less than 2 seconds ago. If there are any, wat. If there aren't then run rsync. I'll let you figure out the script

Hko · 01-27-2004, 05:00 PM

When you put "sleep 4" in your script, it will wait (sleep) for 4 seconds before it continues. If you expect that update the master-FTP server will be finished in 1 minute, then you could put for example "sleep 120" before the rsync command.

Could that enough for your purpose? Or do you really need some mechanism to detect when all changes are done?

GuitsBoy · 01-27-2004, 05:17 PM

Artur - Ive been playing with your snippet of code and Ive got it working and Im debugging some stuff, but theres still one issue. Wouldnt I need to loop that and put a sleep of greater than timeout? Otherwise rsync wont happen till the next time someone FTP's. (assuming the last file takes under 2 seconds to transfer)

Hko - The webservers are pseudo load balanced via a DNS round robin. While the data is out of sync, portions of a web page could be from the newer content while other portions could be the old content. I would much rather everything was transferred in real time +/- a few seconds.

Thank you both for the help. Please let me know if you think of anything else.
-Tony

artur · 01-27-2004, 09:44 PM

ah, those little details...

here's improved code in case you haven't worked it out yet yourself:

Code:

#!/bin/bash
# this script gets executed by dnotify

# still receiving
touch /tmp/flag

# is rsync script started?
if [ ! -e /tmp/my_rsync ]; then 
    (# if not, then execute in separate subshell
    while (touch /tmp/my_rsync); do
      # show that script is running and hasn't choked
      now=`date +%s`
      last=`date -r /tmp/flag +%s 2> /dev/null`
      count=$(($now - $last))
      timeout=2
      # check if still receiving
      if [ $((count)) -gt $((timeout)) ]; then
        # if not, then do rsync
        rsync
        # and remove flags
        rm -f /tmp/flag
        rm -f /tmp/my_rsync
        # and exit subprocess
        exit(0)
      fi
      # loop every 4 seconds
      sleep 4
    done)&
    # all the above gets executed in background in a sub-shell
fi # end of the main script

The touch /tmp/my_rsync could be done outside of while loop, but then you wouldn't know if the background monitoring process hasn't choked. The way I do it, I could check for when my_rsync was last touched and do stuff.

I don't have dnotify to test this with, so I simulated it running by launching

Code:

while (sleep 2); do touch /tmp/flag; done &

to continuously poke the flag. The rsync portion of the script executed seconds after I interrupted that line above

GuitsBoy · 01-28-2004, 09:42 AM

Artur -

You are the MAN! (assuming your male, of course)

Thank you for the code snip. Youve definately got me on my way.

Thanks,
-T