LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 01-27-2004, 05:10 PM   #1
GuitsBoy
LQ Newbie
 
Registered: Oct 2003
Posts: 26

Rep: Reputation: 15
bash script HELP - dnotify > rsync


Hi all -

Hopefully some kind soul can point me in the right direction. I have two mirrored webservers, one running FTP. I currently have a directory that is being monitored by dnotify. When something in that directory changes, dnotify runs a script, which invokes rsync to move the changes to the other server.

My problem is that dnotify will run the script hundreds of times while I am FTPing to this directory. Ofcourse, rsync really only needs to run once - at the end of the last file.

How can I make my script buffer when it runs rsync. I am thinking I would need to start a counter (maybe 2 seconds) and every time dnotify executes the scripts, have the counter reset (back to two seconds). When the timer finally runs out, the scripts executes rsync. Trouble is, I have no idea how to do this, or if this is even possible. I have very little shell scripting knowledge, so please be gentle.

Thanks so much,
-Tony
 
Old 01-27-2004, 05:42 PM   #2
artur
Member
 
Registered: Apr 2002
Location: Illinois, US
Distribution: Red Hat, Fedora, Yellow Dog, Debian, FreeBSD, Embedix
Posts: 106

Rep: Reputation: 15
I guess you could touch a file in temporary directory, and upon each subsequent execution of the script check how long ago the flag was touched. If it is more than two seconds ago, execure rsync and remove the flag file.

Something like:
Code:
#!/bin/bash

timeout=2
now=`date +%s`
last=`date -r /tmp/flag +%s 2> /dev/null`
count=$(($now - $last))
if [ $((count)) -gt $((timeout)) ]; then
   rsync 
   rm -f /tmp/flag
else
   touch /tmp/flag
fi
That's just a rough sketch, may have errorrs in it...

Or you could use find command and look for any files that have been modified less than 2 seconds ago. If there are any, wat. If there aren't then run rsync. I'll let you figure out the script

Last edited by artur; 01-27-2004 at 06:18 PM.
 
Old 01-27-2004, 06:00 PM   #3
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
When you put "sleep 4" in your script, it will wait (sleep) for 4 seconds before it continues. If you expect that update the master-FTP server will be finished in 1 minute, then you could put for example "sleep 120" before the rsync command.

Could that enough for your purpose? Or do you really need some mechanism to detect when all changes are done?
 
Old 01-27-2004, 06:17 PM   #4
GuitsBoy
LQ Newbie
 
Registered: Oct 2003
Posts: 26

Original Poster
Rep: Reputation: 15
Artur - Ive been playing with your snippet of code and Ive got it working and Im debugging some stuff, but theres still one issue. Wouldnt I need to loop that and put a sleep of greater than timeout? Otherwise rsync wont happen till the next time someone FTP's. (assuming the last file takes under 2 seconds to transfer)

Hko - The webservers are pseudo load balanced via a DNS round robin. While the data is out of sync, portions of a web page could be from the newer content while other portions could be the old content. I would much rather everything was transferred in real time +/- a few seconds.

Thank you both for the help. Please let me know if you think of anything else.
-Tony
 
Old 01-27-2004, 10:44 PM   #5
artur
Member
 
Registered: Apr 2002
Location: Illinois, US
Distribution: Red Hat, Fedora, Yellow Dog, Debian, FreeBSD, Embedix
Posts: 106

Rep: Reputation: 15
ah, those little details...

here's improved code in case you haven't worked it out yet yourself:

Code:
#!/bin/bash
# this script gets executed by dnotify

# still receiving
touch /tmp/flag

# is rsync script started?
if [ ! -e /tmp/my_rsync ]; then 
    (# if not, then execute in separate subshell
    while (touch /tmp/my_rsync); do
      # show that script is running and hasn't choked
      now=`date +%s`
      last=`date -r /tmp/flag +%s 2> /dev/null`
      count=$(($now - $last))
      timeout=2
      # check if still receiving
      if [ $((count)) -gt $((timeout)) ]; then
        # if not, then do rsync
        rsync
        # and remove flags
        rm -f /tmp/flag
        rm -f /tmp/my_rsync
        # and exit subprocess
        exit(0)
      fi
      # loop every 4 seconds
      sleep 4
    done)&
    # all the above gets executed in background in a sub-shell
fi # end of the main script
The touch /tmp/my_rsync could be done outside of while loop, but then you wouldn't know if the background monitoring process hasn't choked. The way I do it, I could check for when my_rsync was last touched and do stuff.

I don't have dnotify to test this with, so I simulated it running by launching
Code:
while (sleep 2); do touch /tmp/flag; done &
to continuously poke the flag. The rsync portion of the script executed seconds after I interrupted that line above
 
Old 01-28-2004, 10:42 AM   #6
GuitsBoy
LQ Newbie
 
Registered: Oct 2003
Posts: 26

Original Poster
Rep: Reputation: 15
Artur -

You are the MAN! (assuming your male, of course)

Thank you for the code snip. Youve definately got me on my way.

Thanks,
-T
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
local rsync shell script eradrix Programming 13 07-09-2005 01:57 AM
rsync script twantrd Programming 2 01-06-2005 06:46 AM
Rsync Bandwith saver script pixie Linux - General 0 06-03-2004 04:19 AM
send automatic input to a script called by another script in bash programming jorgecab Programming 2 04-01-2004 01:20 AM
www mirror: is dnotify / rsync my best bet? GuitsBoy Linux - Software 2 01-27-2004 08:34 PM


All times are GMT -5. The time now is 07:36 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration