Shell script to monitor files created on a folder

cmadiam82 · 08-27-2009, 03:33 AM

Hi All Linux Gurus!!!

Can someone please help me on how can i create a script that will monitor file creation on a single folder and sending the newly created file on a separate folder? Only the new created file must be transffered or copied to the other folder. The old ones remains.

I urgently need this for production deployment.

Thanks in advance to all.

You kind help will be much appreciated.

Regards,
cmadiam

JulianTosh · 08-27-2009, 07:55 AM

Check out iNotify..

http://en.wikipedia.org/wiki/Inotify

Down towards the bottom are some programming APIs for iNotify... I like incron and Pyinotify.

anomie · 08-27-2009, 11:23 AM

Quote:

Originally Posted by cmadiam82

Can someone please help me on how can i create a script that will monitor file creation on a single folder and sending the newly created file on a separate folder? Only the new created file must be transffered or copied to the other folder. The old ones remains.

If I'm understanding, this may also be a good candidate for rsync(1).

bigearsbilly · 08-27-2009, 01:37 PM

Code:

while sleep 10;do
  cd source
  find . | cpio -pd destination
done

forrestt · 08-27-2009, 03:26 PM

OK, bigearsbilly's method will be way I/O intensive. Try this instead:

If they are on the same partition:

Code:

#!/bin/sh
SOURCEDIR='/path/to/source/dir'
STORDIR='/path/to/storeage/dir'

while sleep 10 ; do
        for FILE in `ls $SOURCEDIR/* | awk -F/ '{print $NF}'` ; do
                if [ ! -e $STORDIR/$FILE ] ; then
                        ln /tmp/$FILE $STORDIR/$FILE
                fi
        done
done

If they are on different partitions then run that and then the following as often as you need to to not fill up your drive (in that case, /path/to/storage/dir is a temporary storage location):

Code:

rsync -a /path/to/storeage/dir/ /path/to/real/storage ; rm /path/to/storage/dir/*

HTH

Forrest

bigearsbilly · 08-27-2009, 04:22 PM

hmm!
not sure if that's true.
cpio -pd
only copies newer files.

it wasn't a serious offer, just a pointer.

how about a cron job?

forrestt · 08-27-2009, 04:27 PM

OK, not from my interpretation of the man page. However, ln doesn't copy anything, it just makes a new inode that points to the already existing file.

Forrest

p.s. I already run that script from a cron job (without the sleep 10) every minute so I can monitor my son's activities on the net.

p.p.s. Wow, I'm a senior member now!!! Woot!!!

p.p.p.s. I guess I need to change my sig.

cmadiam82 · 08-28-2009, 12:26 AM

Sir, how about if the destination is on the other server? this scenario will apply on our servers.

Quote:

Originally Posted by forrestt

OK, bigearsbilly's method will be way I/O intensive. Try this instead:

If they are on the same partition:

Code:

#!/bin/sh
SOURCEDIR='/path/to/source/dir'
STORDIR='/path/to/storeage/dir'

while sleep 10 ; do
        for FILE in `ls $SOURCEDIR/* | awk -F/ '{print $NF}'` ; do
                if [ ! -e $STORDIR/$FILE ] ; then
                        ln /tmp/$FILE $STORDIR/$FILE
                fi
        done
done

If they are on different partitions then run that and then the following as often as you need to to not fill up your drive (in that case, /path/to/storage/dir is a temporary storage location):

Code:

rsync -a /path/to/storeage/dir/ /path/to/real/storage ; rm /path/to/storage/dir/*

HTH

Forrest

forrestt · 08-28-2009, 08:40 AM

Change the rsync command to:

Code:

rsync -ave ssh monitoredsystem:/path/to/storeage/dir/ /path/to/real/storage
ssh someuser@monitoredsystem rm /path/to/storage/dir/\*

That should do it, but you will still want to test it since this is a production system.

HTH

Forrest

rnldpj · 05-11-2013, 12:59 AM

Check out the script mentioned in the url pasted below. It uses inotifywait to recursively monitor and record file creation events.

http://jackal777.wordpress.com/2013/...-documentroot/