[SOLVED] Perform some tasks when a new file is added in the folder
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Perform some tasks when a new file is added in the folder
Hi Guys,
I am really new in Linux but to be honest so far really impressed and same time stunned to see its powers
Well I have a question for you here;
I have a folder named "logs" which contains log files. I would like to have a bash script or a cron job or something like this which will check the folder continuousy and perform some tasks (for example echo "New file is created") every time a new file with pattern like "Screen.log.***" is created in the folder "logs"
Is it possible in Linux?
Almost, if not completely, everything is possible in Linux one way or another. You can for example just write your script plain and simple so it does what you need and then put it in a cronjob that gets executed every time instance (1 min, 5 min, ...) you want. Or you can write your script to include a timer (with sleep for example) so that it continuously goes through a loop (check the directory for new files) and performs action when needed. If you show us what you've already written as a script we'll take it from there. You'll understand that we will not do all the work for you but will gladly help you solve any problems or clear any doubts you might have. If you're new to scripting check out these sites: Bash Guide for Beginners Advanced Bash Scripting Guide
I am really new in Linux but to be honest so far really impressed and same time stunned to see its powers
Well I have a question for you here;
I have a folder named "logs" which contains log files. I would like to have a bash script or a cron job or something like this which will check the folder continuousy and perform some tasks (for example echo "New file is created") every time a new file with pattern like "Screen.log.***" is created in the folder "logs"
Is it possible in Linux?
I hope i explained good enough.
Regards,
Malik
For the task at hand I'd use inotifywait in conjunction w/ some
filtering via grep.
I googled and tried to find some example to use inotify in bash scripting but unfortunately could not find anything. I have the simplest bash script file (.sh) which only has one line;
Code:
#!/bin/bash
declare -i fd= inotify_init();
but its giving me following error;
Code:
./test-inotify.sh: line 2: syntax error near unexpected token `('
./test-inotify.sh: line 2: `declare -i fd= inotify_init();'
Looks like we can not declare/use class variable in shall script. I even tried to include inotify file by following but did not help either;
Code:
source /usr/include/sys/inotify.h
The only thing i want do is to declare/use a "inotify_add_watch()" job in shell scripting (.sh) and then set it to cron job as EricTRA mentioned.
That could be one option, haven't heard about incrond yet, so thanks for pointing that out. The other way if you don't want to install anything as you pointed out is to use the tools available to you in a script and write it as a daemon, use a timer or put it in a cronjob as pointed to in my first post (#2). Basically upon start you can get the basic information of what you want to monitor. If you want to get notified on new files, you could get the number of files in a specific directory and make your script 'sleep' for let's say 5 minutes, then check again. If the number of files has changed then get the name of the latest modified/created file and echo a message or mail or whatever action you want. You can get far with the basic commands in Linux.
For bash, look at the fam and fam-server package. "File Alteration Monitor". It includes a daemon that monitors files, and a program "fileschanged" that prints the changed files. There are several option on which events to monitor for.
Look in your repository. There should also be an inotify monitoring package for Perl and one for python.
The way I see it he got the error because he didn't have
inotify-tools installed ..
Cheers,
Tink
Hi,
I probably expressed myself in a poor way, not being native English, sorry for that. I only referred to the way he's calling inotify from within the script he was trying to write. For as far as I know you need indeed the inotify-tools package, as you pointed out, that provides a wrapper for inotify. I'm probably mistaking but I thought inotify itself cannot be used directly from within a bash script. That's what I was trying to point out. Sorry for the confusion.
EricTRA, i checked the both links you provided but unfortunately they do not fit to my requirement
Logic in those examples monitors directory changes and echo/alert me even if a file is modified, added or deleted while i only want to see the echo/alert when a new file is added in the directory.
Here is a complete description of my problem once again;
I am using services/tool from a third party let's say company named "ABC". "ABC" created a "log" directory at one of my linux boxes.
Now what happens, they create log files in the format (Screen.log.1, Screen.log.2, Screen.log.3 etc.....) in "log" directory. A Log file is updated after every second. When size of the log file is reached at some specific size for example 20.1, 21.5, 21.8 or 22.2 MB (NOT FIXED) a new log file is created (for example Screen.log.4)
After a while when number of log files (ALSO NOT FIXED) in the "log" directory reached at a specific number they also delete the old log files.
I want to process the log file and save the results into my internal DB. I tried to use crontab but it was causing duplicate values in the DB because crontab runs after a certain amount of time and i do not know the time when a new log file will be created. So its repeating the records.
To conclude, i think the best thing would be to perform the DB insertion steps only when a new log file is created.
The right way, as said above, is to use inotify, which does exactly what he wants.
The rest of options are in some point between outdated and sub-optimal, being a mix of both.
Just install whatever package contains inotifywait in your distro, then do this:
Code:
$ inotifywait -m -r /tmp/
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
/tmp/ CREATE foo
/tmp/ OPEN foo
/tmp/ ATTRIB foo
/tmp/ CLOSE_WRITE,CLOSE foo
That is the output you'll get if you do this in another terminal:
Code:
$ touch /tmp/foo
The rest of the project is learning bash script and reading the inotifywait man page.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.