LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-23-2012, 08:01 AM   #1
newbie456
LQ Newbie
 
Registered: Aug 2012
Posts: 6

Rep: Reputation: Disabled
Script to monitor folder for new files?


Hi,
I got a problem regarding to making a script to monitor a folder for any new files. Im using Ubuntu 12.04 LTS

My requirement is that:
The script had to be constantly checking the folder for any new files. If detected i will then run a script to edit the new file. After editing the new file, the script had to go back to checking the folder again for any new file.

I got a link that has what i wanted but when i tried it out, it cannot work.
http://unix.stackexchange.com/questi...-for-new-files

This is the code:
Code:
inotifywait -m /path 2>&- | awk '$2 == "CREATE" { print $3; fflush() }' |
    while read file; do
        echo "$file"
        # do something with the file
    done
As my folder name in called 'output', i edit the script to my needs:
Code:
inotifywait -m /home/user/output 2>&- | awk '$2 == "CREATE" { print $3; fflush() }' |
    while read file; do
        echo "$file"
        # do something with the file
    done
However, when i copy a file into /home/user/output the script did not display "file". Why is it not working ?
Or is there any other solution?
 
Old 08-23-2012, 09:37 AM   #2
newbie456
LQ Newbie
 
Registered: Aug 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Talking Check if folder have new files?

Hi,
I have a problem regarding checking if a folder has new files?

My problem is that the script has to keep checking a folder for any new files and if detected new files, i will then run a script to edit the new file.

I found a similar script that suit my needs but it does not seem to be working.

The script that i found is:
Code:
inotifywait -m /path 2>&- | awk '$2 == "CREATE" { print $3; fflush() }' |
    while read file; do
        echo "$file"
        # do something with the file
    done
I edited the script to my path where my folder is located and it does not seems to work.
Code:
inotifywait -m /home/user/output 2>&- | awk '$2 == "CREATE" { print $3; fflush() }' |
    while read file; do
        echo "$file"
        # do something with the file
    done
Why is this not working?
Or is there any other way?
 
Old 08-23-2012, 09:56 AM   #3
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,911
Blog Entries: 14

Rep: Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142
Do you get any errors when you run the script?

Have you verified the command, inotifywait, exists and is in your PATH?
 
Old 08-23-2012, 10:01 AM   #4
newbie456
LQ Newbie
 
Registered: Aug 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by MensaWater View Post
Do you get any errors when you run the script?

Have you verified the command, inotifywait, exists and is in your PATH?
When i run the script in terminal, the terminal just open up and close.

I had already installed inotify and what do you mean by verifying the command, inotifywait, exists and is in your PATH?
 
Old 08-23-2012, 11:02 AM   #5
michaelk
Moderator
 
Registered: Aug 2002
Posts: 15,929

Rep: Reputation: 1824Reputation: 1824Reputation: 1824Reputation: 1824Reputation: 1824Reputation: 1824Reputation: 1824Reputation: 1824Reputation: 1824Reputation: 1824Reputation: 1824
Welcome to LinuxQuestions.
What distribution/version are you running? As stated inotifywait may not be in your environment path. The environment path is a list of directories that is searched to locate an application for example when it is executed. I would assume it is /usr/bin. I would specify the event so all you need to do is have awk print out the filename.

/usr/bin/inotifywait -e create -mrq /home/user/output | awk '{print $3; fflush() }' | while...

Start the program in one terminal. Open up a second and run the command touch /home/user/output/test.me The file will be created and you should see echoed in the first terminal.
 
Old 08-23-2012, 11:05 AM   #6
newbie456
LQ Newbie
 
Registered: Aug 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by michaelk View Post
Welcome to LinuxQuestions.
What distribution/version are you running? As stated inotifywait may not be in your environment path. The environment path is a list of directories that is searched to locate an application for example when it is executed. I would assume it is /usr/bin. I would specify the event so all you need to do is have awk print out the filename.

/usr/bin/inotifywait -e create -mrq /home/user/output | awk '{print $3; fflush() }' | while...

Start the program in one terminal. Open up a second and run the command touch /home/user/output/test.me The file will be created and you should see echoed in the first terminal.
Im using Ubuntu 12.04 LTS.
By using synaptic, i installed inotifywait.
 
Old 08-23-2012, 11:07 AM   #7
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,911
Blog Entries: 14

Rep: Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142Reputation: 1142
Type "which inotifywait" to see if it shows you the directory the command is in. If it does then it exists and is in your PATH. If it doesn't then it may exist but not be in your PATH.

The PATH variable determines where commands are located when you run them. You can see your PATH by typing "echo $PATH". The output will be a colon delimited list of directories such as:
/bin:/usr/bin:/usr/local/bin:/opt/someapp

When you type a command without putting in the full path then it searches through the PATH variable directories in order until it finds it. If your PATH had what I showed above but the intotifywait command was actually in something like /usr/share/bin it wouldn't find it when you typed it because that directory is not in your PATH. Every time you ran the command you'd have to type /usr/share/bin/inotifywait if it wasn't in your PATH.

Since you say you installed it the command presumably exists - you just need to know where and be sure it is in PATH.
 
Old 08-23-2012, 07:57 PM   #8
newbie456
LQ Newbie
 
Registered: Aug 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by MensaWater View Post
Type "which inotifywait" to see if it shows you the directory the command is in. If it does then it exists and is in your PATH. If it doesn't then it may exist but not be in your PATH.

The PATH variable determines where commands are located when you run them. You can see your PATH by typing "echo $PATH". The output will be a colon delimited list of directories such as:
/bin:/usr/bin:/usr/local/bin:/opt/someapp

When you type a command without putting in the full path then it searches through the PATH variable directories in order until it finds it. If your PATH had what I showed above but the intotifywait command was actually in something like /usr/share/bin it wouldn't find it when you typed it because that directory is not in your PATH. Every time you ran the command you'd have to type /usr/share/bin/inotifywait if it wasn't in your PATH.

Since you say you installed it the command presumably exists - you just need to know where and be sure it is in PATH.
When i type "which inotifywait", the result is:
Code:
/usr/bin/inotifywait
My PATH is:
Code:
/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
 
Old 08-23-2012, 10:06 PM   #9
newbie456
LQ Newbie
 
Registered: Aug 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Well, i think i found another method to check if the folder have new files and do something about it.

First, i use watch to check a script every second.
Next, the script will check for the folder if it has files in it. Because, at first the folder is empty.
If, there files detected, the script will do something about it.

Code:
#!/bin/sh

watch -n1 ./test.sh
test.sh:
Code:
if [ $(ls -1A output | wc -l) -gt 0 ] ;then "do_something"; fi
 
Old 08-23-2012, 11:31 PM   #10
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,374

Rep: Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383Reputation: 2383
Do you really need to fire off a script every second; that's a lot of overhead on your system.
Why not just create simple daemon
Code:
while 1
do
    do_whatever
done
& have do_whatever attempt to process any files it can find; in fact just put the infinite loop inside the main prog (if you have the src) then
Code:
nohup do_whatever &
You can even add a sleep at the bottom of the loop if you want to stop it spinning too much.
How soon/how often does it expect to find/need to process a file.
If its going to loop quickly, use the nice cmd, so it doesn't dominate.

Last edited by chrism01; 08-27-2012 at 04:31 AM.
 
Old 08-24-2012, 09:35 AM   #11
allend
Senior Member
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 4,573

Rep: Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489Reputation: 1489
Try adding the '-e close_write' option to inotifywait so that no action is taken until the file is actually written and closed.
I would also try the -q option to inotifywait rather than the error redirection.
I think the awk stuff is redundant.
Code:
inotifywait -mq -e close_write /home/user/output | \
    while read file; do
        echo "$file"
        # do something with the file
    done

Last edited by allend; 08-24-2012 at 09:39 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] create sub-folder from folder name and transfer files on the fly BoydRice Linux - General 5 03-07-2012 07:44 PM
[SOLVED] Using terminal command -Find files in a folder and copy them to a different folder j-jock Linux - General 4 11-28-2011 02:20 AM
[SOLVED] How to list folder size, and number of files and folders in folder steven.c.banks Linux - General 3 11-24-2010 06:24 AM
[SOLVED] Check folder for specific files than move script BlackCrowe Programming 3 11-23-2010 05:59 AM
how to check folder if it has files in it using Bash Shell script vincent.dang Linux - Newbie 7 08-05-2010 07:58 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 03:13 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration