LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-18-2013, 12:55 PM   #1
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,005

Rep: Reputation: 67
is there a command that will permit me to monitor changes to a directory?


I would like to know which files and subdirectories in a particular directory are being modified. Is there some command or series of commands like top or tail -f that will allow me to know immediately when a file is changed?

I know that the find command has a -newer flag which means I can do this:
Code:
touch ./mark.txt
find /path/to/directory -newer ./mark.txt
Just wondering if there's some way to have such a command continually report file changes in a directory.
 
Old 01-18-2013, 01:11 PM   #2
thesnow
Member
 
Registered: Nov 2010
Location: Minneapolis, MN
Distribution: Ubuntu, Red Hat, Mint
Posts: 170

Rep: Reputation: 56
Check this thread https://www.linuxquestions.org/quest...changes-54998/ for info on fam. If you just want something quick I sometimes use
Code:
watch -d -n 1 'ls -altr /var/log|tail -40'
 
1 members found this post helpful.
Old 01-18-2013, 01:18 PM   #3
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,335
Blog Entries: 55

Rep: Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535
Quote:
Originally Posted by thesnow View Post
Check this thread https://www.linuxquestions.org/quest...changes-54998/ for info on fam. If you just want something quick I sometimes use
Code:
watch -d -n 1 'ls -altr /var/log|tail -40'
FAM was a kernel 2.4 dnotify thing (notice the age of that thread). Nowadays you'll prolly want to use something inotify-based. Here's a good write-up of the differences: http://www.noah.org/wiki/Inotify,_FAM,_Gamin
Example:
Code:
inotifywait -m /some/dir/ -e create --timefmt "%Y%m%d%H%M%S" --format "%Tw:%wf:%fe:%e" 2>&-

Last edited by unSpawn; 01-18-2013 at 01:25 PM. Reason: //Add reference, example
 
1 members found this post helpful.
Old 01-18-2013, 01:50 PM   #4
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,005

Original Poster
Rep: Reputation: 67
Thank you both...

Quote:
Originally Posted by unSpawn View Post
Nowadays you'll prolly want to use something inotify-based. Here's a good write-up of the differences: http://www.noah.org/wiki/Inotify,_FAM,_Gamin
unSpawn, I've read that writeup and am still wondering one thing in particular. Suppose someone were uploading a massive FTP file to the directory I'd like to monitor. Would inotify fire off an event each time the file changed or only when the file was complete? I had been looking at lysncd for another issue and this question occurred to me. The vague docs on lsyncd say:
Quote:
Lsyncd watches a local directory trees event monitor interface (inotify or fsevents). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes.
I'm hoping to get a better idea of what the notifications really correspond to.
 
Old 01-18-2013, 02:07 PM   #5
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,335
Blog Entries: 55

Rep: Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535
Quote:
Originally Posted by sneakyimp View Post
Suppose someone were uploading a massive FTP file to the directory I'd like to monitor. Would inotify fire off an event each time the file changed or only when the file was complete?
It's a syscall thing ('man 2 syscalls'). If you for example would run 'strace -v -o /tmp/strace.log nano ~/newfile' then write something, save and close the file, then in "/tmp/strace.log" you can see which syscalls were used. Put simplified the FTP daemon would use "create" once to create the item, then a lot of writes due to chunks arriving over the network and then a close. The reason lysncd aggregates and combines events is more an trade-off or efficiency thing I think: while burst-like behavior has its drawbacks you don't want to rsync for example a file that's still being written to.
 
Old 01-18-2013, 04:58 PM   #6
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,005

Original Poster
Rep: Reputation: 67
This inotifywait command looks extremely useful.

I had to install inotify-tools on my ubuntu desktop to acquire the inotifywait command. There is no inotify-related package in the amazon linux repositories:
Code:
$ yum search inotify
Loaded plugins: priorities, security, update-motd, upgrade-helper
amzn-main                                                                                                                                                                                     | 2.1 kB     00:00     
amzn-updates                                                                                                                                                                                  | 2.3 kB     00:00     
Warning: No matches found for: inotify
No Matches found
However, man inotify returns a lot of information but does not mention inotifywait. I wonder of lsyncd need inotify-tools...

I noticed from the docs on my ubuntu workstation the following section detailing events:
Quote:
access A watched file or a file within a watched directory was read from.

modify A watched file or a file within a watched directory was written to.

attrib The metadata of a watched file or a file within a watched directory was modified. This includes timestamps, file permissions, extended attributes etc.

close_write A watched file or a file within a watched directory was closed, after being opened in writeable mode. This does not necessarily imply the file was written to.

close_nowrite A watched file or a file within a watched directory was closed, after being opened in read-only mode.

close A watched file or a file within a watched directory was closed, regardless of how it was opened. Note that this is actually implemented simply by listening for both close_write and
close_nowrite, hence all close events received will be output as one of these, not CLOSE.

open A watched file or a file within a watched directory was opened.

moved_to
A file or directory was moved into a watched directory. This event occurs even if the file is simply moved from and to the same directory.

moved_from
A file or directory was moved from a watched directory. This event occurs even if the file is simply moved from and to the same directory.

move A file or directory was moved from or to a watched directory. Note that this is actually implemented simply by listening for both moved_to and moved_from, hence all close events received will
be output as one or both of these, not MOVE.

move_self
A watched file or directory was moved. After this event, the file or directory is no longer being watched.

create A file or directory was created within a watched directory.

delete A file or directory within a watched directory was deleted.

delete_self
A watched file or directory was deleted. After this event the file or directory is no longer being watched. Note that this event can occur even if it is not explicitly being listened for.

unmount
The filesystem on which a watched file or directory resides was unmounted. After this event the file or directory is no longer being watched. Note that this event can occur even if it is not
explicitly being listened to.

 
Old 01-18-2013, 07:09 PM   #7
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,335
Blog Entries: 55

Rep: Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535
Quote:
Originally Posted by sneakyimp View Post
There is no inotify-related package in the amazon linux repositories
Try a package already in a compatible distribution or compile from scratch?


Quote:
Originally Posted by sneakyimp View Post
man inotify returns a lot of information but does not mention inotifywait.
Try 'man inotifywait'?


Quote:
Originally Posted by sneakyimp View Post
I wonder of lsyncd need inotify-tools...
AFAIK none, it provides its own interface: https://github.com/axkibe/lsyncd/blob/master/inotify.c


Quote:
Originally Posted by sneakyimp View Post
I noticed from the docs on my ubuntu workstation the following section detailing events
So wrt your FTP example using "-e create -e close_write" would log the start and end of a file transfer.
 
Old 01-18-2013, 09:11 PM   #8
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,005

Original Poster
Rep: Reputation: 67
This inotify wait command is really coool.

Quote:
Originally Posted by unSpawn View Post
Try a package already in a compatible distribution or compile from scratch?
I don't know if I'll need lsyncd just yet. I'll cross this bridge when I come to it.

Quote:
Originally Posted by unSpawn View Post
Try 'man inotifywait'?
Code:
$ man inotifywait
No manual entry for inotifywait


I have been using this command to monitor access (among other things) to an apache-hosted web root on my workstation with this command:
Code:
inotifywait -mr /var/www/domain/html -e access -e create -e close_write -e modify -e move -e delete --timefmt "%Y%m%d%H%M%S" --format "%Tw:%wf:%f e:%e" 2>&-
What is totally baffling me is why I can access dozens of pages hosted from this web root and only the .htaccess file ever gets accessed -- no images or anything. I have restarted Apache, I have cleared my browser cache, and not one of the php, css, js, or image files ever triggers an access event. I guess I must misunderstand what access means? And why does Apache keep accessing htaccess? So confused.

E.g., this is what results from the *very first request* to a particular url after rebooting Apache:
Code:
$ inotifywait -mr /var/www/domain/html -e access -e create -e close_write -e modify -e move -e delete --timefmt "%Y%m%d%H%M%S" --format "%Tw:%wf:%f e:%e" 2>&-

20130118185821w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
20130118185823w:/var/www/domain/html/f:.htaccess e:ACCESS
 
Old 01-20-2013, 03:27 PM   #9
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,005

Original Poster
Rep: Reputation: 67
Quote:
Originally Posted by sneakyimp View Post
What is totally baffling me is why I can access dozens of pages hosted from this web root and only the .htaccess file ever gets accessed -- no images or anything. I have restarted Apache, I have cleared my browser cache, and not one of the php, css, js, or image files ever triggers an access event. I guess I must misunderstand what access means? And why does Apache keep accessing htaccess? So confused.
OK so having thought about it, Apache is obviously caching things. I have searched around and see this in my apache conf:
Code:
CacheRoot /var/cache/apache2/mod_disk_cache
and yet I cannot seem to detect any evidence that apache ever touches this directory. That's a different topic though.


So how does checking a file's modification date differ from file access? I'm guessing the former is a lot more efficient. E.g., checking a few bytes in a file system data structure. Is there some way to monitor this mod-date-checking activity?
 
Old 01-20-2013, 07:44 PM   #10
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,335
Blog Entries: 55

Rep: Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535
Quote:
Originally Posted by sneakyimp View Post
This inotify wait command is really cool.
It is. Did you know BTW Samhain can watch files using inotify too?


Quote:
Originally Posted by sneakyimp View Post
Code:
$ man inotifywait
No manual entry for inotifywait
Odd. I'm pretty sure the package included a manual page for it, else see http://manpages.ubuntu.com/manpages/...ifywait.1.html


Quote:
Originally Posted by sneakyimp View Post
What is totally baffling me is why I can access dozens of pages hosted from this web root and only the .htaccess file ever gets accessed -- no images or anything. I have restarted Apache, I have cleared my browser cache, and not one of the php, css, js, or image files ever triggers an access event. I guess I must misunderstand what access means?
Inotify is based on system calls. To find out try this on your workstation not on your production host!: copy your httpd.conf somewhere else and edit it so it's got no spare workers and a maximum of say 2 active children max. Fire up Apache with that ("-f") config running it from strace something like "sudo /usr/bin/strace -f -ff -q -v -o /tmp/httpd /usr/sbin/httpd -f /path/to/httpd.conf". Access a few files, stop strace then check the /tmp/httpd* files for the system calls used.


Quote:
Originally Posted by sneakyimp View Post
And why does Apache keep accessing htaccess?
"There are two main reasons to avoid the use of .htaccess files. The first of these is performance.": http://httpd.apache.org/docs/2.2/howto/htaccess.html


Quote:
Originally Posted by sneakyimp View Post
I cannot seem to detect any evidence that apache ever touches this directory. That's a different topic though.
Looking up some web server benchmark reports could get you a general overview of what's involved.


Quote:
Originally Posted by sneakyimp View Post
So how does checking a file's modification date differ from file access?
'man 2 stat' vs 'man 2 read' / 'man 2 readv'?
 
1 members found this post helpful.
Old 01-20-2013, 11:33 PM   #11
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,005

Original Poster
Rep: Reputation: 67
Quote:
Originally Posted by unSpawn View Post
It is. Did you know BTW Samhain can watch files using inotify too?
I have set up Samhain on a couple of servers now. I wasn't aware that it used inotify.

Quote:
Originally Posted by unSpawn View Post
"There are two main reasons to avoid the use of .htaccess files. The first of these is performance.": http://httpd.apache.org/docs/2.2/howto/htaccess.html
I just find it bizarre that Apache thinks it needs to access the .htaccess file every time it requests a page. I.e., why doesn't it bother caching it and checking modification dates.

Quote:
Originally Posted by unSpawn View Post
'man 2 stat' vs 'man 2 read' / 'man 2 readv'?
OK that helps explain a bit about it. A struct sounds like possibly a lot less data than a buffer.

I think this thread has been resolved.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] cu does not permit connection to a modem vsandilya Linux - Newbie 3 09-13-2012 10:05 AM
Find directory older than x and mv directory with sub files command ajhart Linux - Newbie 4 07-15-2011 05:24 AM
Permit yahoo messanger in squid MEETKAMBOJ Mandriva 3 01-10-2011 12:30 PM
Permit /opt/ in ubuntu ajd344 Linux - Newbie 2 08-21-2006 08:36 PM
How to Monitor a Directory Daddyfix Linux - Newbie 1 11-07-2005 07:02 AM


All times are GMT -5. The time now is 08:16 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration