set up a watch folder or rsync

TenTenths · 08-25-2017, 04:01 AM

Right, I'm back in the office

Grab yourself a coffee, I did, and let's get started!

For this example I'm using /home/incron/source as the source (clue is in the name!

) and /home/incron/dest as the destination (clue is also in the name!

)

Code:

# incrontab -l
/home/incron/source IN_DELETE,IN_CLOSE_WRITE /home/incron/action.sh $@ $# $% /home/incron/dest

action.sh:

Code:

#!/bin/bash

# Author : Grant MacDonald
# Purpose: Action handler for files in folders
#        : https://www.linuxquestions.org/questions/centos-111/set-up-a-watch-folder-or-rsync-4175612499/
# Version: 1.0
# Date   : 25/08/2017

# Version History
# 1.0 Initial version

# Needs the following incrontab -e entry
# /home/incron/source IN_CLOSE_WRITE,IN_DELETE /home/incron/action.sh $@ $# $% /home/incron/dest

# incrontab uses the following:

# $$   dollar sign
# $@   watched filesystem path (see above)
# $#   event-related file name
# $%   event flags (textually)
# $&   event flags (numerically)

# Define some variables based on the command line
THEFOLDER=$1
THEFILE=$2
THEACTION=$3
THEDEST=$4
THELOG=/home/incron/action.log

echo "File:   ${THEFILE}" >> ${THELOG}
echo "Source: ${THEFOLDER}" >> ${THELOG}
echo "Action: ${THEACTION}" >> ${THELOG}
echo "Dest:   ${THEDEST}" >> ${THELOG}

case ${THEACTION} in

  IN_CLOSE_WRITE) # A file has been closed after change
                  echo "cp -f ${THEFOLDER}/${THEFILE} ${THEDEST}/${THEFILE}" >> ${THELOG}
                  cp -f ${THEFOLDER}/${THEFILE} ${THEDEST}/${THEFILE}
  ;;

  IN_DELETE) # A file has been deleted
             echo "rm -f ${THEDEST}/${THEFILE}" >> ${THELOG}
             rm -f ${THEDEST}/${THEFILE}
  ;;

esac

# Elvis Has Left The Server

Now we try it:
Our folders are pretty much empty:

Code:

[someuser@somehost incron]# ls -lath
total 4.0K
drwxr-xr-x  4 someuser someuser   46 Aug 25 09:41 .
drwxr-xr-x  2 someuser someuser    6 Aug 25 09:40 dest
drwxr-xr-x  2 someuser someuser    6 Aug 25 09:40 source
-rwxr-xr-x  1 someuser someuser 1.1K Aug 25 09:40 action.sh
drwxr-xr-x. 5 someuser someuser   61 Aug 25 09:32 ..

[someuser@somehost incron]# ls -lath source
total 0
drwxr-xr-x 4 someuser someuser 46 Aug 25 09:41 ..
drwxr-xr-x 2 someuser someuser  6 Aug 25 09:40 .

[someuser@somehost incron]# ls -lath dest
total 0
drwxr-xr-x 4 someuser someuser 46 Aug 25 09:41 ..
drwxr-xr-x 2 someuser someuser  6 Aug 25 09:40 .

Now we create a file in the source folder

Code:

[someuser@somehost incron]# echo "Test File 1" > source/testfile1

Now we see the file created in source, in dest and we have a log file!

Code:

[someuser@somehost incron]# ls -lath source
total 4.0K
drwxr-xr-x 4 someuser someuser 63 Aug 25 09:41 ..
drwxr-xr-x 2 someuser someuser 22 Aug 25 09:41 .
-rw-r--r-- 1 someuser someuser 12 Aug 25 09:41 testfile1

[someuser@somehost incron]# ls -lath dest
total 4.0K
drwxr-xr-x 2 someuser someuser 22 Aug 25 09:41 .
-rw-r--r-- 1 someuser someuser 12 Aug 25 09:41 testfile1
drwxr-xr-x 4 someuser someuser 63 Aug 25 09:41 ..

[someuser@somehost incron]# ls -lath
total 8.0K
drwxr-xr-x  2 someuser someuser   22 Aug 25 09:41 dest
-rw-r--r--  1 someuser someuser  159 Aug 25 09:41 action.log
drwxr-xr-x  4 someuser someuser   63 Aug 25 09:41 .
drwxr-xr-x  2 someuser someuser   22 Aug 25 09:41 source
-rwxr-xr-x  1 someuser someuser 1.1K Aug 25 09:40 action.sh
drwxr-xr-x. 5 someuser someuser   61 Aug 25 09:32 ..

[someuser@somehost incron]# cat action.log
File:   testfile1
Source: /home/incron/source
Action: IN_CLOSE_WRITE
Dest:   /home/incron/dest
cp -f /home/incron/source/testfile1 /home/incron/dest/testfile1

But we also want to delete files:

Code:

[someuser@somehost incron]# rm source/testfile1
rm: remove regular file ‘source/testfile1’? y

And now we look in the folders again and they are empty and we have more in the log

Code:

[someuser@somehost incron]# ls -lath source
total 0
drwxr-xr-x 2 someuser someuser  6 Aug 25 09:41 .
drwxr-xr-x 4 someuser someuser 63 Aug 25 09:41 ..

[someuser@somehost incron]# ls -lath dest
total 0
drwxr-xr-x 2 someuser someuser  6 Aug 25 09:41 .
drwxr-xr-x 4 someuser someuser 63 Aug 25 09:41 ..

[someuser@somehost incron]# cat action.log
File:   testfile1
Source: /home/incron/source
Action: IN_CLOSE_WRITE
Dest:   /home/incron/dest
cp -f /home/incron/source/testfile1 /home/incron/dest/testfile1
File:   testfile1
Source: /home/incron/source
Action: IN_DELETE
Dest:   /home/incron/dest
rm -f /home/incron/dest/testfile1
[someuser@somehost incron]#

Tested on CentOS Linux release 7.3.1611 (Core) and CentOS release 6.8 (Final)

robertkwild · 08-25-2017, 04:48 AM

wow thanks tentenths

but why shouldnt my 2 commands works just by the commands in the incrontab, why do i need to write a script for the two commands but if i write one command in the incrontab it works?

is there another program i could use that does this

is there anything that can do folders aswell as files

TenTenths · 08-25-2017, 05:10 AM

Quote:

Originally Posted by robertkwild

wow thanks tentenths

but why shouldnt my 2 commands works just by the commands in the incrontab, why do i need to write a script for the two commands but if i write one command in the incrontab it works?

is there another program i could use that does this

is there anything that can do folders aswell as files

Not really looked in to why the two entry version didnt work, may have that as a coffee break problem.

As for sub-folders, I've never been able to easily make that work.

robertkwild · 08-25-2017, 05:22 AM

i know inotifywait, only does files aswell, but i may just do a rsync command or copy command to do folders files aswell using the * to transfer all files/folders in the source, what do you think

TenTenths · 08-25-2017, 05:46 AM

You could then set up your incrontab to look for a few different events based on your environment and rsync sure, it's your system, you'd have to see if rsync caused excessive overhead, I think you've probably learned enough to have a few ideas on things to try.

michaelk · 08-25-2017, 06:35 AM

inotifywait can be recursive i.e. watch subdirectories but as far as I know incron can not.

Not sure what you mean by only does files.

Don't know exactly how incron works but there might be some recursion happening because you have created two watches for the same directory.

robertkwild · 08-25-2017, 08:17 AM

oh, i see micheal where your getting at, as i have created two watches for the same folder so it doesnt know what one to do first

i have read the man page and theres this -

Additionaly, there is a symbol which doesn't appear in the inotify symbol set. It it IN_NO_LOOP. This symbol disables monitoring events until the current one is completely handled (until its child process exits).

is this what i need?

michaelk · 08-25-2017, 08:40 AM

I don't know and never have tried it. I have always used scripts similar to what TenTenths posted to process events.

TenTenths · 08-25-2017, 09:04 AM

Been trying this just for a laugh and it does seem that the first action defined in incrontab for a folder is the only one that will "fire"

robertkwild · 08-25-2017, 09:40 AM

thanks tentenths for trying mate, appreciate it big time

as michael said probably its because there watching the same folder, i bet if they were different watch folders ie line one was source a and dest b and line 2 was source c dest d it would work

wouldnt the "IN_NO_LOOP" work?

TenTenths · 08-25-2017, 10:02 AM

IN_NO_LOOP seems to break my Centos 6

robertkwild · 08-25-2017, 10:08 AM

i dont know where to put that command, i will test tonight on my test rig at home

/mnt/invoicetest IN_CLOSE_WRITE cp -r $@/$# /mnt/invoicetest_1/$#
IN_NO_LOOP
/mnt/invoicetest IN_DELETE rm -f $@/$# /mnt/invoicetest_1/$#

or

/mnt/invoicetest IN_CLOSE_WRITE,IN_NO_LOOP cp -r $@/$# /mnt/invoicetest_1/$#
/mnt/invoicetest IN_DELETE,IN_NO_LOOP rm -f $@/$# /mnt/invoicetest_1/$#

TenTenths · 08-25-2017, 10:13 AM

Quote:

Originally Posted by robertkwild

/mnt/invoicetest IN_CLOSE_WRITE,IN_NO_LOOP cp -r $@/$# /mnt/invoicetest_1/$#
/mnt/invoicetest IN_DELETE,IN_NO_LOOP rm -f $@/$# /mnt/invoicetest_1/$#

Tried this on Centos 6 and Centos 7 and it doesn't have the desired result. To be honest you're better off with the scripted version.

robertkwild · 08-25-2017, 10:47 AM

THEFOLDER=$1

cp -f ${THEFOLDER}/${THEFILE} ${THEDEST}/${THEFILE}

i just want to understand your script abit more -

cp -f ${THEFOLDER}/${THEFILE} ${THEDEST}/${THEFILE}

but the variable THEFOLDER=$1, what does $1 do?

as normal scripts i have done in the past

source=/mnt/invoicetest

so if you put in your script $source its really a shortcut to /mnt/invoicetest

thanks,

rob

TenTenths · 08-25-2017, 12:13 PM

$1 is the first variable passed on the command line. In this case the incrontab variable for the source folder is passed.
$2 is the filename again from the incrontab variable
$3 is the action in word form
$4 I pass the destination folder

Why don't I code source and dest in the script? Because I write portable scripts. You could have another line in incrontab for _source2 and a different destination folder as the 4th parameter and you dont have to edit another copy of the script.