LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-10-2014, 01:10 PM   #16
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42

Thanks schneidz, will check it out.

UPDATE: Ok I checked it out and it wouldnt work.I dont create nor do I have permissions to create files in those directories.Those directories are update with files @ certain times of the day, and the generic names are listed in inputFileNames.txt .when those files arrive that are in the inputfileNames.txt file, I want to be notified.They subsequently go into different direcotires to be processed,I want to know, and in the end, arrive in the outbox directory , I want to know..etc
Quote:
Originally Posted by schneidz View Post
something like this mite work[untested]:
Code:
touch inbox2.lst processed2.lst outbox2.lst transferred2.lst
while [ 1 ]
do
 sleep 5
 find /path/to/inbox > inbox1.lst
 if [ "`diff inbox1.lst inbox2.lst | grep -f files.txt | grep ^\<`" ]
 then
  echo file "`diff inbox1.lst inbox2.lst | grep -f files.txt | grep ^\<`" received
 fi
 if [ "`diff inbox1.lst inbox2.lst | grep ^\>`" ]
 then
  sleep 5
  find /path/to/processed > processed1.lst
  if [ "`diff processed1.lst processed2.lst | grep -f files.txt | grep ^\<`" ]
  then
   echo file "`diff processed1.lst processed2.lst | grep -f files.txt | grep ^\<`" processed
   sleep 5
   find /path/to/outbox > outbox1.lst
   if [ "`diff outbox1.lst outbox2.lst | grep -f files.txt | grep ^\<`" ]
   then
    echo file "`diff outbox1.lst outbox2.lst | grep -f files.txt | grep ^\<`" ready to be transferred
   fi
   if [ "`diff outbox1.lst outbox2.lst | grep ^\>`" ]
   then
    sleep 5
    find /path/to/transferred > transferred1.lst
    if [ "`diff transferred1.lst transferred2.lst | grep -f files.txt | grep ^\<`" ]
    then
     echo files "`diff transferred1.lst transferred2.lst | grep -f files.txt | grep ^\<`" have been delivered
    fi
   fi
  fi
 fi
 mv inbox1.lst inbox2.lst
 mv processed1.lst processed2.lst
 mv outbox1.lst outbox2.lst
 mv transferred1.lst transferred2.lst
done

Last edited by cbtshare; 02-10-2014 at 01:33 PM.
 
Old 02-11-2014, 12:44 AM   #17
SAbhi
Member
 
Registered: Aug 2009
Location: Bangaluru, India
Distribution: CentOS 6.5, SuSE SLED/ SLES 10.2 SP2 /11.2, Fedora 11/16
Posts: 664

Rep: Reputation: 80
Quote:
Originally Posted by cbtshare View Post
Thank you guys for all your help.

So far I've changed the appraoch for the script so many times lol...this is the approach I'm finally taking, this is just ehe logic for the first folder.

Code:
#!/bin/bash
set -e

IFS=$'\n'
lines=($(cat /tmp/EODFileAudit/inputFileName.txt))
linesout=($(cat /tmp/EODFileAudit/outputFileName.txt))


#Unitil results text file has in it , that the 2 files have been received, continue
#to monitor the directories
unnecesary use of "cat" when you can read a file in while loop.

Code:
while read line; do some operaion on $line; done < $filename
 
Old 02-11-2014, 09:03 AM   #18
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,026

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
Quote:
Originally Posted by SAbhi View Post
unnecesary use of "cat" when you can read a file in while loop.

Code:
while read line; do some operaion on $line; done < $filename
i usually prefer to use cat to supply a stream to a loop because its easier to read left-to-rite (and is a very negligible performance hit).
Code:
cat $filename | while read line; do some operaion on $line; done

Last edited by schneidz; 02-11-2014 at 09:04 AM.
 
Old 02-11-2014, 10:32 AM   #19
SAbhi
Member
 
Registered: Aug 2009
Location: Bangaluru, India
Distribution: CentOS 6.5, SuSE SLED/ SLES 10.2 SP2 /11.2, Fedora 11/16
Posts: 664

Rep: Reputation: 80
Quote:
Originally Posted by schneidz View Post
i usually prefer to use cat to supply a stream to a loop because its easier to read left-to-rite (and is a very negligible performance hit).
Code:
cat $filename | while read line; do some operaion on $line; done
Thats correct too but using this way involves one more pipe and that means one more sub process!! that would be triggered for a bit while supplying it with redirectional operator doesnt.
 
1 members found this post helpful.
Old 02-11-2014, 11:14 AM   #20
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
Quote:
Originally Posted by SAbhi View Post
unnecesary use of "cat" when you can read a file in while loop.
I am putting the content into an array, not reading the file.I know I could use readarray command and others.

Last edited by cbtshare; 02-11-2014 at 11:32 AM.
 
Old 02-11-2014, 12:13 PM   #21
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 3,774
Blog Entries: 1

Rep: Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339
The easiest way to do this is a few oneliners:

# this will give you a md5sum of the directory in question and all its files and timestamps and sizes
ls -lR . | md5sum

# if you compare that result to a new result you will always know when a file has changed

Rather than write a script I looked one up that does this very thing beautifully:

ref: http://stackoverflow.com/questions/2...-autospec-does
Code:
#!/bin/bash

sha=0
previous_sha=0

update_sha()
{
    sha=`ls -lR . | sha1sum`
}

build () {
    ## Build/make commands here
    echo
    echo "--> Monitor: Monitoring filesystem... (Press enter to force a build/update)"
}

changed () {
    echo "--> Monitor: Files changed, Building..."
    build
    previous_sha=$sha
}

compare () {
    update_sha
    if [[ $sha != $previous_sha ]] ; then changed; fi
}

run () {
    while true; do

        compare

        read -s -t 1 && (
            echo "--> Monitor: Forced Update..."
            build
        )

    done
}

echo "--> Monitor: Init..."
echo "--> Monitor: Monitoring filesystem... (Press enter to force a build/update)"
run
 
Old 02-12-2014, 06:00 PM   #22
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
Thank you, but I am not monitoring files.Its the directories I am concerned about and when new files come in , I want to be notified and when they leave and go for processing and get a new name I want to know about it and when it goes to the outbox I want to know, and then when the vendor has taken the file from a specific directory I want to know.The above script doesnt seem to do that
 
Old 02-13-2014, 12:36 PM   #23
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 3,774
Blog Entries: 1

Rep: Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339
Yeah it does that.

List every file/directory/size/modified time. MD5SUM the result.

if there are any changes to files, adds/deletes, new directories, renames, updates etc... the new MD5SUM will be different.

This method monitors what you are describing.

Fyi: Directories are files.

Last edited by szboardstretcher; 02-13-2014 at 12:42 PM.
 
Old 02-19-2014, 02:28 PM   #24
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
hmm, ok I understand your point,but I need to compare the md5 to another file and if it matches its the file I want, and then I get notified that the file has left or arrived.The issue with your code is what will I compare the md5 to? because the file has not come in , and since the file changes everyday, I cant compare the md5 or have a standard md5.

As I have said before, I need to monitor directories(files come in and leave those directories constantly) and when files come in (inbox)that match files in an array or text file I want to be notified when processed (processed directory) I want to be notified etc etc .

I have the following code which works for the just files arriving but if the file has NOT arrived as yet, I just want to be notified once for each file not repeatedly which is what the script is doing.

Code:
#!/bin/bash
#set -e

IFS=$','

for i in ${array[@]}

do

if [ -f /home/qtscfp/sftp/iqplatform/inbox/$i  ]
then


                if [ ! grep -Fxq "$i File has been received" /tmp/EODFileAudit/result.txt   ]
                then

                        echo "$i File has been received" >> /tmp/EODFileAudit/result.txt

                        else

                        sed -i '$i File has been received/$i File has been received/g' /tmp/EODFileAudit/result.txt

                fi



else

(I'm not sure how to get this to only print once I tired [ ! grep -Fxq "$i File NOT received" /tmp/EODFileAudit/result.txt , but it doesnt work)

echo "$i File NOT received" >> /tmp/EODFileAudit/result.txt

#sed -i "$i File NOT received/$i File NOT received/g" /tmp/EODFileAudit/result.txt

#   [ ! grep -Fxq "$i File NOT received" /tmp/EODFileAudit/result.txt   ]

#echo "$i File NOT received" >> /tmp/EODFileAudit/result.txt #&&
 #  sed -i '$i File has been received/$i File has been received/g' /tmp/EODFileAudit/result.txt


fi

done
 
Old 02-21-2014, 10:47 AM   #25
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 3,774
Blog Entries: 1

Rep: Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339Reputation: 1339
Quote:
Originally Posted by cbtshare View Post
hmm, ok I understand your point,but I need to compare the md5 to another file and if it matches its the file I want, and then I get notified that the file has left or arrived.The issue with your code is what will I compare the md5 to? because the file has not come in , and since the file changes everyday, I cant compare the md5 or have a standard md5.

As I have said before, I need to monitor directories(files come in and leave those directories constantly) and when files come in (inbox)that match files in an array or text file I want to be notified when processed (processed directory) I want to be notified etc etc .

I have the following code which works for the just files arriving but if the file has NOT arrived as yet, I just want to be notified once for each file not repeatedly which is what the script is doing.
It creates an INITIAL md5 on your first run, because you ask it to. That is the CURRENT state of the files and directories underneath.

Then it checks the directory and files at an interval you specify and makes a TEMPORARY md5 and compares it to the INITIAL one.

If there is a difference, it ALERTS you that a file, directory, permission, whatever has changed or left, then replaces the INTIAL md5 with the TEMPORARY md5.

Everytime there is a change, it will ALERT you and create a new INITIAL md5 to reference all the TEMPORARY checks against.

Hope that clears things up. For doing what you want to do, this is how I have done it. (without inotify)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
real time sound monitoring niteen Programming 14 04-15-2013 01:56 PM
squid real time monitoring Ammad Linux - Networking 2 02-08-2010 12:46 PM
Squid Real-time Monitoring molybtek Linux - Networking 1 10-10-2008 10:42 PM
Real time Monitoring squid logs jatender Linux - Server 3 10-03-2007 04:08 AM
Keylogger / session monitoring in real time? houler Linux - Security 2 04-06-2005 10:21 PM


All times are GMT -5. The time now is 03:23 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration