LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-21-2011, 09:49 AM   #1
abubakar-malik
LQ Newbie
 
Registered: Feb 2011
Posts: 10

Rep: Reputation: 3
Perform some tasks when a new file is added in the folder


Hi Guys,

I am really new in Linux but to be honest so far really impressed and same time stunned to see its powers
Well I have a question for you here;

I have a folder named "logs" which contains log files. I would like to have a bash script or a cron job or something like this which will check the folder continuousy and perform some tasks (for example echo "New file is created") every time a new file with pattern like "Screen.log.***" is created in the folder "logs"
Is it possible in Linux?

I hope i explained good enough.

Regards,
Malik
 
Old 02-21-2011, 09:55 AM   #2
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295
Hello,

Almost, if not completely, everything is possible in Linux one way or another. You can for example just write your script plain and simple so it does what you need and then put it in a cronjob that gets executed every time instance (1 min, 5 min, ...) you want. Or you can write your script to include a timer (with sleep for example) so that it continuously goes through a loop (check the directory for new files) and performs action when needed. If you show us what you've already written as a script we'll take it from there. You'll understand that we will not do all the work for you but will gladly help you solve any problems or clear any doubts you might have. If you're new to scripting check out these sites:
Bash Guide for Beginners
Advanced Bash Scripting Guide

Kind regards,

Eric
 
Old 02-21-2011, 09:58 AM   #3
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Quote:
Originally Posted by abubakar-malik View Post
Hi Guys,

I am really new in Linux but to be honest so far really impressed and same time stunned to see its powers
Well I have a question for you here;

I have a folder named "logs" which contains log files. I would like to have a bash script or a cron job or something like this which will check the folder continuousy and perform some tasks (for example echo "New file is created") every time a new file with pattern like "Screen.log.***" is created in the folder "logs"
Is it possible in Linux?

I hope i explained good enough.

Regards,
Malik

For the task at hand I'd use inotifywait in conjunction w/ some
filtering via grep.


Cheers,
Tink
 
Old 02-21-2011, 09:58 AM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 15,883

Rep: Reputation: 2191Reputation: 2191Reputation: 2191Reputation: 2191Reputation: 2191Reputation: 2191Reputation: 2191Reputation: 2191Reputation: 2191Reputation: 2191Reputation: 2191
This what inotify is for. Check out inotify-tools - page down to the explanation for inotifywait.
 
Old 02-22-2011, 04:21 AM   #5
abubakar-malik
LQ Newbie
 
Registered: Feb 2011
Posts: 10

Original Poster
Rep: Reputation: 3
Hi All,

I googled and tried to find some example to use inotify in bash scripting but unfortunately could not find anything. I have the simplest bash script file (.sh) which only has one line;

Code:
#!/bin/bash
declare -i fd= inotify_init();
but its giving me following error;
Code:
./test-inotify.sh: line 2: syntax error near unexpected token `('
./test-inotify.sh: line 2: `declare -i fd= inotify_init();'
Looks like we can not declare/use class variable in shall script. I even tried to include inotify file by following but did not help either;

Code:
source /usr/include/sys/inotify.h
The only thing i want do is to declare/use a "inotify_add_watch()" job in shell scripting (.sh) and then set it to cron job as EricTRA mentioned.

Ideas????
 
Old 02-22-2011, 04:39 AM   #6
abubakar-malik
LQ Newbie
 
Registered: Feb 2011
Posts: 10

Original Poster
Rep: Reputation: 3
FYI, I also tried to use "inotifywait" as below;

Code:
inotifywait -m /path/to/dir
but did not help either. getting the following
Code:
-bash: inotifywait: command not found
I do not want to install any new package and currently I am using Linux 2.6.15
 
Old 02-22-2011, 05:00 AM   #7
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295
Hello,

You get the error because inotify is not bash adapted. Have a look at this page on how to use inotify.

Kind regards,

Eric
 
Old 02-22-2011, 07:18 AM   #8
abubakar-malik
LQ Newbie
 
Registered: Feb 2011
Posts: 10

Original Poster
Rep: Reputation: 3
Yes EricTRA. You are right.

So is there any other way to implement my requirement in bash scripting?

I guess installing "incrond" utility as mentioned in the following link or any other better approach?

http://www.cyberciti.biz/faq/linux-i...e-directories/


I do not want to use any programming library like C, C++ except bash programming

Last edited by abubakar-malik; 02-22-2011 at 07:19 AM.
 
Old 02-22-2011, 07:27 AM   #9
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295
Hello,

That could be one option, haven't heard about incrond yet, so thanks for pointing that out. The other way if you don't want to install anything as you pointed out is to use the tools available to you in a script and write it as a daemon, use a timer or put it in a cronjob as pointed to in my first post (#2). Basically upon start you can get the basic information of what you want to monitor. If you want to get notified on new files, you could get the number of files in a specific directory and make your script 'sleep' for let's say 5 minutes, then check again. If the number of files has changed then get the name of the latest modified/created file and echo a message or mail or whatever action you want. You can get far with the basic commands in Linux.

Kind regards,

Eric
 
Old 02-22-2011, 07:31 AM   #10
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295
Hi,

Here are some 'ideas' of ways you can obtain what you want.

http://snippets.dzone.com/posts/show/7384
http://aplawrence.com/Unixart/watchdir.html

Kind regards,

Eric
 
Old 02-22-2011, 07:53 AM   #11
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
For bash, look at the fam and fam-server package. "File Alteration Monitor". It includes a daemon that monitors files, and a program "fileschanged" that prints the changed files. There are several option on which events to monitor for.

Look in your repository. There should also be an inotify monitoring package for Perl and one for python.
 
Old 02-22-2011, 10:17 AM   #12
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Quote:
Originally Posted by EricTRA View Post
Hello,

You get the error because inotify is not bash adapted.

Kind regards,

Eric
What is that supposed to mean?

The way I see it he got the error because he didn't have
inotify-tools installed ..


Cheers,
Tink
 
Old 02-23-2011, 01:25 AM   #13
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295Reputation: 1295
Quote:
Originally Posted by Tinkster View Post
What is that supposed to mean?

The way I see it he got the error because he didn't have
inotify-tools installed ..


Cheers,
Tink
Hi,

I probably expressed myself in a poor way, not being native English, sorry for that. I only referred to the way he's calling inotify from within the script he was trying to write. For as far as I know you need indeed the inotify-tools package, as you pointed out, that provides a wrapper for inotify. I'm probably mistaking but I thought inotify itself cannot be used directly from within a bash script. That's what I was trying to point out. Sorry for the confusion.

Kind regards,

Eric
 
Old 02-23-2011, 04:46 AM   #14
abubakar-malik
LQ Newbie
 
Registered: Feb 2011
Posts: 10

Original Poster
Rep: Reputation: 3
EricTRA, i checked the both links you provided but unfortunately they do not fit to my requirement
Logic in those examples monitors directory changes and echo/alert me even if a file is modified, added or deleted while i only want to see the echo/alert when a new file is added in the directory.

Here is a complete description of my problem once again;

I am using services/tool from a third party let's say company named "ABC". "ABC" created a "log" directory at one of my linux boxes.
Now what happens, they create log files in the format (Screen.log.1, Screen.log.2, Screen.log.3 etc.....) in "log" directory. A Log file is updated after every second. When size of the log file is reached at some specific size for example 20.1, 21.5, 21.8 or 22.2 MB (NOT FIXED) a new log file is created (for example Screen.log.4)
After a while when number of log files (ALSO NOT FIXED) in the "log" directory reached at a specific number they also delete the old log files.

I want to process the log file and save the results into my internal DB. I tried to use crontab but it was causing duplicate values in the DB because crontab runs after a certain amount of time and i do not know the time when a new log file will be created. So its repeating the records.

To conclude, i think the best thing would be to perform the DB insertion steps only when a new log file is created.

I hope it will give you a better picture.

Ideas???
 
Old 02-23-2011, 04:53 AM   #15
i92guboj
Gentoo support team
 
Registered: May 2008
Location: Lucena, Córdoba (Spain)
Distribution: Gentoo
Posts: 4,074

Rep: Reputation: 386Reputation: 386Reputation: 386Reputation: 386
The right way, as said above, is to use inotify, which does exactly what he wants.

The rest of options are in some point between outdated and sub-optimal, being a mix of both.

Just install whatever package contains inotifywait in your distro, then do this:

Code:
$ inotifywait -m -r /tmp/
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/tmp/ CREATE foo
/tmp/ OPEN foo
/tmp/ ATTRIB foo
/tmp/ CLOSE_WRITE,CLOSE foo
That is the output you'll get if you do this in another terminal:

Code:
$ touch /tmp/foo
The rest of the project is learning bash script and reading the inotifywait man page.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to perform two tasks at a time in kernel... raulapati Linux - Embedded & Single-board computer 2 01-18-2011 04:46 AM
[SOLVED] How to perform root level tasks while in GUI? jlavik Linux - Newbie 8 10-06-2010 10:39 AM
Can ks.cfg be added to a linux install boot DVD to perform kickstart installation? Latitude Linux - Software 2 06-05-2009 08:39 AM
Tasks to perform when host is connected to internet. venkat_k Red Hat 1 01-30-2009 03:52 AM
writing a script to perform a function on each users mail folder jhill Linux - Newbie 2 08-24-2004 07:18 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:19 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration