Check if folder have new files?

newbie456 · 08-23-2012, 08:01 AM

Hi,
I got a problem regarding to making a script to monitor a folder for any new files. Im using Ubuntu 12.04 LTS

My requirement is that:
The script had to be constantly checking the folder for any new files. If detected i will then run a script to edit the new file. After editing the new file, the script had to go back to checking the folder again for any new file.

I got a link that has what i wanted but when i tried it out, it cannot work.
http://unix.stackexchange.com/questi...-for-new-files

This is the code:

Code:

inotifywait -m /path 2>&- | awk '$2 == "CREATE" { print $3; fflush() }' |
    while read file; do
        echo "$file"
        # do something with the file
    done

As my folder name in called 'output', i edit the script to my needs:

Code:

inotifywait -m /home/user/output 2>&- | awk '$2 == "CREATE" { print $3; fflush() }' |
    while read file; do
        echo "$file"
        # do something with the file
    done

However, when i copy a file into /home/user/output the script did not display "file". Why is it not working ?
Or is there any other solution?

newbie456 · 08-23-2012, 09:37 AM

Hi,
I have a problem regarding checking if a folder has new files?

My problem is that the script has to keep checking a folder for any new files and if detected new files, i will then run a script to edit the new file.

I found a similar script that suit my needs but it does not seem to be working.

The script that i found is:

Code:

inotifywait -m /path 2>&- | awk '$2 == "CREATE" { print $3; fflush() }' |
    while read file; do
        echo "$file"
        # do something with the file
    done

I edited the script to my path where my folder is located and it does not seems to work.

Code:

inotifywait -m /home/user/output 2>&- | awk '$2 == "CREATE" { print $3; fflush() }' |
    while read file; do
        echo "$file"
        # do something with the file
    done

Why is this not working?
Or is there any other way?

MensaWater · 08-23-2012, 09:56 AM

Do you get any errors when you run the script?

Have you verified the command, inotifywait, exists and is in your PATH?

newbie456 · 08-23-2012, 10:01 AM

Quote:

Originally Posted by MensaWater

Do you get any errors when you run the script?

Have you verified the command, inotifywait, exists and is in your PATH?

When i run the script in terminal, the terminal just open up and close.

I had already installed inotify and what do you mean by verifying the command, inotifywait, exists and is in your PATH?

michaelk · 08-23-2012, 11:02 AM

Welcome to LinuxQuestions.
What distribution/version are you running? As stated inotifywait may not be in your environment path. The environment path is a list of directories that is searched to locate an application for example when it is executed. I would assume it is /usr/bin. I would specify the event so all you need to do is have awk print out the filename.

/usr/bin/inotifywait -e create -mrq /home/user/output | awk '{print $3; fflush() }' | while...

Start the program in one terminal. Open up a second and run the command touch /home/user/output/test.me The file will be created and you should see echoed in the first terminal.

newbie456 · 08-23-2012, 11:05 AM

Quote:

Originally Posted by michaelk

Welcome to LinuxQuestions.
What distribution/version are you running? As stated inotifywait may not be in your environment path. The environment path is a list of directories that is searched to locate an application for example when it is executed. I would assume it is /usr/bin. I would specify the event so all you need to do is have awk print out the filename.

/usr/bin/inotifywait -e create -mrq /home/user/output | awk '{print $3; fflush() }' | while...

Start the program in one terminal. Open up a second and run the command touch /home/user/output/test.me The file will be created and you should see echoed in the first terminal.

Im using Ubuntu 12.04 LTS.
By using synaptic, i installed inotifywait.

MensaWater · 08-23-2012, 11:07 AM

Type "which inotifywait" to see if it shows you the directory the command is in. If it does then it exists and is in your PATH. If it doesn't then it may exist but not be in your PATH.

The PATH variable determines where commands are located when you run them. You can see your PATH by typing "echo $PATH". The output will be a colon delimited list of directories such as:
/bin:/usr/bin:/usr/local/bin:/opt/someapp

When you type a command without putting in the full path then it searches through the PATH variable directories in order until it finds it. If your PATH had what I showed above but the intotifywait command was actually in something like /usr/share/bin it wouldn't find it when you typed it because that directory is not in your PATH. Every time you ran the command you'd have to type /usr/share/bin/inotifywait if it wasn't in your PATH.

Since you say you installed it the command presumably exists - you just need to know where and be sure it is in PATH.

newbie456 · 08-23-2012, 07:57 PM

Quote:

Originally Posted by MensaWater

Type "which inotifywait" to see if it shows you the directory the command is in. If it does then it exists and is in your PATH. If it doesn't then it may exist but not be in your PATH.

The PATH variable determines where commands are located when you run them. You can see your PATH by typing "echo $PATH". The output will be a colon delimited list of directories such as:
/bin:/usr/bin:/usr/local/bin:/opt/someapp

When you type a command without putting in the full path then it searches through the PATH variable directories in order until it finds it. If your PATH had what I showed above but the intotifywait command was actually in something like /usr/share/bin it wouldn't find it when you typed it because that directory is not in your PATH. Every time you ran the command you'd have to type /usr/share/bin/inotifywait if it wasn't in your PATH.

Since you say you installed it the command presumably exists - you just need to know where and be sure it is in PATH.

When i type "which inotifywait", the result is:

Code:

/usr/bin/inotifywait

My PATH is:

Code:

/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

newbie456 · 08-23-2012, 10:06 PM

Well, i think i found another method to check if the folder have new files and do something about it.

First, i use watch to check a script every second.
Next, the script will check for the folder if it has files in it. Because, at first the folder is empty.
If, there files detected, the script will do something about it.

Code:

#!/bin/sh

watch -n1 ./test.sh

test.sh:

Code:

if [ $(ls -1A output | wc -l) -gt 0 ] ;then "do_something"; fi

chrism01 · 08-23-2012, 11:31 PM

Do you really need to fire off a script every second; that's a lot of overhead on your system.
Why not just create simple daemon

Code:

while 1
do
    do_whatever
done

& have do_whatever attempt to process any files it can find; in fact just put the infinite loop inside the main prog (if you have the src) then

Code:

nohup do_whatever &

You can even add a sleep at the bottom of the loop if you want to stop it spinning too much.
How soon/how often does it expect to find/need to process a file.
If its going to loop quickly, use the nice cmd, so it doesn't dominate.

allend · 08-24-2012, 09:35 AM

Try adding the '-e close_write' option to inotifywait so that no action is taken until the file is actually written and closed.
I would also try the -q option to inotifywait rather than the error redirection.
I think the awk stuff is redundant.

Code:

inotifywait -mq -e close_write /home/user/output | \
    while read file; do
        echo "$file"
        # do something with the file
    done