[SOLVED] Most efficient method to `monitor' for additions to a file ... in real-time
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Most efficient method to `monitor' for additions to a file ... in real-time
The environment is Linux; language is C/C++; solar powered system so power is at a premium...
I have a file that is added to (as in, something is appended to the tail end) every 1 second. I need to identify, in a separate process, as soon data is added to the end of the file so that I can take action on it. The entire purpose of separate process is to monitor this file so the solution can be blocking.
What is the right way to do this?
The `dumb' ways I can think of involve infinite loops, sleep commands, and repeatedly trying to read ... but if I'm out of sync on my loop it can be nearly another second before I identify that data is available ... and the whole method just seems inefficient.
Is there a graceful way to identify exactly when new data is added to a file .. when that data is added once every ~1 second ... and I'm going to be monitoring this file 24/7?
Write a program, in which you open the file and read from it. Don't stop at eof, sleep a little then try reading again. Repeat infinitely.
I would suggest combining both of these approaches: instead of "busy waiting" to see if the file has been extended (which, actually, can be problematic if the file is "remote"), use some appropriate notification mechanism to allow the program to efficiently "sleep" until there might be more data available in the file.
Obviously, the cleanest approach would be to write the data to a pipe that is being read by a process which eventually writes the data to the permanent destination.
I think the method I have settled on is using inotify if the input file is a regular file, using O_ASYNC and sigaction() to set up SIGIO for the inotify file-descriptor, and pause() while I wait for signals. If the input file is actually a pipe, I am simply setting up SIGIO to trigger ``I/O possible'' for the pipe, and again, using pause() while I wait.
I realize I could open the file in blocking mode and have read() block while I wait for input ... but there is also a network socket I am listening to ... so I don't want either the file or the socket blocking.
I saw some post somewhere where an `expert' said, ``if signals are your answer then you're asking the wrong question''... but I don't see how anything would be more appropriate for my application than pause() and SIGIO.
I realize I could open the file in blocking mode and have read() block while I wait for input ... but there is also a network socket I am listening to ... so I don't want either the file or the socket blocking.
Sounds like a job for select(3) (or poll(3)).
EDIT: nevermind, won't work on files, as NevemTeve points out.
Although if you already have something working maybe "if it ain't broken don't fix it" applies...
Last edited by ntubski; 01-20-2016 at 09:51 AM.
Reason: oops
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.