LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-05-2014, 07:58 PM   #1
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Rep: Reputation: 42
bash script monitoring directories real time


Hey all,

I am writing a script to monitor 6 different directories for files that come in.(I cant use inotify, because I cant install anything on the server)

The directories are :

/home/revi/sftp/main/inbox (file come in and leave every 5- 10 mins)
/home/directory/outbox/ (file come in and leave every 5- 10 mins)
/home/commo/ftp/pwatch/inbox/(file come in and leave every 5- 10 mins)
/home/commo/ftp/sterl/inbox/(file come in and leave every 5- 10 mins)
/home/revi/sftp/main/processed/ (Files are here permanently)
/home/directory/ndm/directory/outbox/transferred/ (Files are here permanently)

I have 10 filenames in text called inputFileNames, and 10 filenames in a text called outputfileNames. When files come in they go into the /home/revi/sftp/main/inbox , then a script moves and process them,changes the name(to match a name in the outputFileNames text file) and puts a copy of the unchanged name text in the processed directory
/home/revi/sftp/main/processed/ and also in /home/directory/outbox/, then file is moved from outbox to some remote location and moved to
/home/directory/ndm/directory/outbox/transferred/.

So in essence I want to write a script to let me know in a log file when a file that matches the name in inputFileNames arrives in the inbox. Then in the log I want to know when the file is moved to the processed directory, which means it has been processed.I then want to know via log when the processed file (with name change matching the outputFileNames text)is in the outbox directory /home/directory/outbox/ , and then know when it is in the transferred directory.And then this continues for the other 10 files .

I basically tried approaching it from 2 angles ,but I'm stuck , so am asking for help or ideas how to do this(without inotify).

Here is what I have so far, on both occasions

Code:
#!/bin/bash -xv

IFS=$'\n'
lines=($(cat /tmp/EODFileAudit/inputFileName.txt))
linesout=($(cat /tmp/EODFileAudit/outputFileName.txt))


ls  /home/revi/sftp/main/inbox | grep -E '^[^d]' | grep -E ^[^Total] |

while read line
do

#e=$(echo $line | cut -d " " -f 11)
#echo $line

for i in ${linesout[@]}
do

if [[ $line = $i ]]
then

echo "This file  has been recieved....YEAAA"


fi




done







done
I was also trying another method below, but I just confused myself , or forgot what I wanted to do .,...

Code:
old_IFS=$IFS
IFS=$'\n'
lines=($(cat /tmp/EODFileAudit/inputFileName.txt))
linesout=($(cat /tmp/EODFileAudit/outputFileName.txt))
i=0
#IFS=$old_IFS
#$(sed -n '/${lines[0]} /p'
#printf "%s\n" "${lines[0]}"
#printf "%s\n" "${lines[1]}"

^[^Total]
#echo ${lines[0]}


until [ sed -n ' Broker has taken the file ${lines[$i]} /p ' /tf/EOfdrt/result.txt ] && [  sed -n 'Broker has taken the file ${lines[$i]}/p'  /tf/EOfdrt/result.txt ]
do
                  if [[ -f /home/daion/sftp/register/inbox/${lines[$i]} ]]
echo ${lines[$i]}
                        then
                        echo "++++++++++++++++++++++++++++++++++++ " >> /tmp/EODFileAudit/result.txt

                        printf "%s\n" " ${lines[$i]} file has been received" >> /tmp/EODFileAudit/result.txt



                        elif [[ -f /home/revi/sftp/insta/processed/${lines[$i]} ]]

                                then

                                sed -i 's/${lines[$i]} file has been received/${lines[$i]} file has been processed/g' /tmp/EODFileAudit/result.txt


                        elif [[ -f /home/main/outbox/${linesout[$i]} ]]
                                then
                                sed -i 's/${lines[$i]} file has been processed/${lines[$i]} awaiting transfer/g' /tmp/EODFileAudit/result.txt


                        elif [[ ${linesout[$i]:0:4} = "TIFI" ]] && [[ -f /home/qtscfp/ndm/broadridge/outbox/transferred/${linesout[$i]} ]]
                                then

                                sed -i 's/${lines[$i]} awaiting transfer/Broker has taken the file ${lines[$i]} /g' /tmp/EODFileAudit/result.txt

                        elif [[ ${linesout[$i]:0:4} = "INTR"]] && [[ ! -f /home/qtscfp/outbox ]]

                        then

                                sed -i 's/${lines[$i]} awaiting transfer/Broker has taken the file ${lines[$i]} /g' /tmp/EODFileAudit/result.txt
                        echo "Nothing Happening" > /tmp/EODFileAudit/result.txt



                        #echo "++++++++++++++++++++++++++++++++++++ " >> /tmp/EODFileAudit/result.txt
                        #printf "%s\n" "INTR_APEXQSTRPSD1.DAT not yet received " >> /tmp/EODFileAudit/result.txt

                  fi
sleep 5
$i=(($i+1))
done
 
Old 02-06-2014, 12:17 PM   #2
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
Here is a simplified explanation.

(Do) Monitor Directory -Inbox
(test) ->If file(s) come in that match name in a textFile
(Action) ->Report that file(s) has been received .

<If files are gone, from INBOX,start monitoring PROCESSED directory>

(Do) Monitor Directory - PROCESSED
(test) -> If file(s) come in that match names on a textFile
(Action) -> Report that file(s) has been processed
<If files appear in PROCESSED start, monitoring OUTBOX> directory

(Do) Monitor Directory - OUTBOX
(test) ->If file(s) come in that matches name on textFile
(Action) ->Report that file(s) are ready to be transferred
<If file(s) are gone from OUTBOX start monitoring TRANSFERRED directory


(Do) Monitor Directory - TRANSFERRED
(test) -> If file(s) come in that matches name on a textFile
(Action) -> Report that file(s) have been delivered

^
|-Keep monitoring for more files
 
Old 02-06-2014, 12:45 PM   #3
Prince Imhotep
Member
 
Registered: Jan 2014
Posts: 45

Rep: Reputation: Disabled
You can use the watch command to chronically execute another command for you.

Example:
Code:
watch -n (x seconds) sudo ls -l /.../.../.../(any directory)
You can use this command in your script too.
 
Old 02-06-2014, 03:18 PM   #4
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
Yea lol, I have been using watch, but I have to browse through a long list of files to match what files I want to see are there.so I need a script that does it.
I dont know if I am starting on the right track, or the logic is correct.The tricky part is to continue to watch realtime and update the logs with the notification once in a log file

Code:
#!/bin/bash -xv
set -e

IFS=$'\n'
lines=($(cat /tmp/EODFileAudit/inputFileName.txt))
linesout=($(cat /tmp/EODFileAudit/outputFileName.txt))


#Unitil results text file has in it , that the 2 files have been received, continue
#to monitor the directories

until [ sed -n ' ${lines[0]} File has been received /p '  /tmp/EODFileAudit/result.txt ] && [  sed -n '${lines[0]} File has been received/p'  /tmp/EODFileAudit/result.txt ]
do
ls  /home/qtscfp/inbox/ | grep -E '^[^d]' | grep -E ^[^Total] |

if [[ -f /home/qtscfp/inbox/${lines[0]} && $(sed -n '${lines[0]} File has been received/p ' /tmp/EODFileAudit/result.txt)| $? -ne "0" ]]
then

echo "${lines[0]} File has been received" > /tmp/EODFileAudit/result.txt



fi

if [[ -f  /home/qtscfp/inbox/${lines[1]}  && $(sed -n '${lines[1]} File has been received/p ' /tmp/EODFileAudit/result.txt)| $? -ne "0" ]]
then

echo "${lines[1]} File has been received" > /tmp/EODFileAudit/result.txt


fi
done


##Monitoring the processed directory, once the

if [[ !-f /home/qtscfp/inbox/${lines[0]} && /home/qtscfp/inbox/${lines[1]} ]]

then

ls -l /home/qtscfp/sftp/iqplatform/processed/

fi
 
Old 02-06-2014, 03:31 PM   #5
Prince Imhotep
Member
 
Registered: Jan 2014
Posts: 45

Rep: Reputation: Disabled
So what you need exactly is a script that watches and logs the status of each file in a lot of files in a single directory?

Is that right?
 
Old 02-07-2014, 11:37 AM   #6
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
No, not in a single directory, multiple directories.I have started something,can I get some help to look it over please?

##I'm not matching against a list of filenames in inputFileName.txt and
##outputFileName.txt as yet, just want to get the basic down. have however put the list
##of files into an array and check that the files are there

Code:
#!/bin/bash -xv
set -e

IFS=$'\n'
lines=($(cat /tmp/EODFileAudit/inputFileName.txt))
linesout=($(cat /tmp/EODFileAudit/outputFileName.txt))


#Unitil results text file has in it , that the 2 files have been received, continue
#to monitor the directories

until [ sed -n " ${lines[0]} File has been received /p "  /tmp/EODFileAudit/result.txt ] && [  sed -n "${lines[1]} File has been received/p"  /tmp/EODFileAudit/result.txt ]
do
ls  /home/qtscfp/sftp/iqplatform/inbox | grep -E '^[^d]' | grep -E ^[^Total] |

if [[ -f /home/qtscfp/sftp/iqplatform/inbox/${lines[0]} && $(sed -n "${lines[0]} File has been received/p " /tmp/EODFileAudit/result.txt)| $? -ne "0" ]]
then

printf "%s\n" "${lines[0]} File has been received" > /tmp/EODFileAudit/result.txt



fi

if [[ -f  /home/qtscfp/sftp/iqplatform/inbox/${lines[1]}  && $(sed -n "${lines[1]} File has been received/p " /tmp/EODFileAudit/result.txt)| $? -ne "0" ]]
then

printf "%s\n" "${lines[1]} File has been received" > /tmp/EODFileAudit/result.txt


fi
done


##Monitoring the processed directory, once the files have left the inbox directory

if [[ !-f /home/qtscfp/sftp/iqplatform/inbox/${lines[0]} && /home/qtscfp/sftp/iqplatform/inbox/${lines[1]} ]]

then

until [ sed -n " ${lines[0]} File has been processed /p "  /tmp/EODFileAudit/result.txt ] && [  sed -n "${lines[1]} File has been processed/p"  /tmp/EODFileAudit/result.txt ]
do
ls  /home/qtscfp/sftp/iqplatform/processed/ | grep -E '^[^d]' | grep -E ^[^Total] |

if [[ -f /home/qtscfp/sftp/iqplatform/processed/${lines[0]} && $(sed -n "${lines[0]} File has been processed/p " /tmp/EODFileAudit/result.txt) | $? -ne "0" ]]
then


printf "%s\n" "${lines[0]} File has been processed" >> /tmp/EODFileAudit/result.txt



fi

if [[ -f /home/qtscfp/sftp/iqplatform/processed/${lines[1]}  && $(sed -n "${lines[1]} File has been received/p " /tmp/EODFileAudit/result.txt) | $? -ne "0" ]]
then

printf "%s\n" "${lines[1]} File has been processed" >> /tmp/EODFileAudit/result.txt


fi
done


fi

###Monitors the outbox directory to see that files are awaiting collection and submission

until [ sed -n " ${linesout[0]} File awaiting submission /p "  /tmp/EODFileAudit/result.txt ] && [  sed -n "${linesout[1]} File has awaiting to be taken /p"  /tmp/EODFileAudit/result.txt ]
do
ls  /home/qtscfp/outbox/ | grep -E '^[^d]' | grep -E ^[^Total] |

if [[ -f /home/qtscfp/outbox/${linesout[0]} && $(sed -n "${linesout[0]} File awaiting submission/p " /tmp/EODFileAudit/result.txt)| $? -ne "0" ]]
then

printf "%s\n" "${linesout[0]} File awaiting submission" >> /tmp/EODFileAudit/result.txt



fi

if [[ -f   /home/qtscfp/outbox/${linesout[1]}  && $(sed -n "${linesout[1]} File awaiting submission/p" /tmp/EODFileAudit/result.txt)| $? -ne "0" ]]
then

printf "%s\n" "${linesout[1]} File awaiting submission" >> /tmp/EODFileAudit/result.txt


fi
done

###Final step is files are no longer in outbox, they should be in transferred directory of if they
##are no longer in outbox and begin with INTR, they have been taken,so in both cases, output to log as Completed!

if [[ !-f /home/qtscfp/outbox/${linesout[0]} ]]
then

until [ sed -n " ${linesout[0]} vendor has took the File /p "  /tmp/EODFileAudit/result.txt ]
do

printf "%s\n" "${linesout[0]} vendor has took the File" >> /tmp/EODFileAudit/result.txt

done

elif [[ !-f /home/qtscfp/outbox/${linesout[1]} ]]
then
ls /home/qtscfp/ndm/vendor/outbox/transferred/  | grep -E '^[^d]' | grep -E ^[^Total] |

if [[ -f  /home/qtscfp/ndm/vendor/outbox/transferred/${linesout[1]} ]]
then
until [ sed -n " ${linesout[1]} Broadride took the File /p "  /tmp/EODFileAudit/result.txt ]
do

printf "%s\n" "${linesout[1]} Broadride took the File " >> /tmp/EODFileAudit/result.txt

done

fi
fi


fi

Last edited by cbtshare; 02-07-2014 at 11:44 AM.
 
Old 02-08-2014, 09:36 AM   #7
Prince Imhotep
Member
 
Registered: Jan 2014
Posts: 45

Rep: Reputation: Disabled
I've written simple scripts before, and I know that a script conataining that number of conditions and regexes must have at least one mistake if written from the ground up without dissecting each condition, and each regexed command, and analysing it's output.

Your script looks neet, and it's complicated and I don't have the text files you're sedding so as to know whether the regexed sed lines are correct or not, also I dont know what output of the sed lines are you expecting.

So the best advice I can give you is take each condition you want to be satisfied to the console, and try it on it's own. Analyse the input, Analyse the output, and find what's wrong, or what is not happening, and I'm sure you'll get what you want.

Good luck.
 
Old 02-08-2014, 11:02 AM   #8
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,028

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
Quote:
Originally Posted by cbtshare View Post
Here is a simplified explanation.

(Do) Monitor Directory -Inbox
(test) ->If file(s) come in that match name in a textFile
(Action) ->Report that file(s) has been received .

<If files are gone, from INBOX,start monitoring PROCESSED directory>

(Do) Monitor Directory - PROCESSED
(test) -> If file(s) come in that match names on a textFile
(Action) -> Report that file(s) has been processed
<If files appear in PROCESSED start, monitoring OUTBOX> directory

(Do) Monitor Directory - OUTBOX
(test) ->If file(s) come in that matches name on textFile
(Action) ->Report that file(s) are ready to be transferred
<If file(s) are gone from OUTBOX start monitoring TRANSFERRED directory


(Do) Monitor Directory - TRANSFERRED
(test) -> If file(s) come in that matches name on a textFile
(Action) -> Report that file(s) have been delivered

^
|-Keep monitoring for more files
something like this mite work[untested]:
Code:
touch inbox2.lst processed2.lst outbox2.lst transferred2.lst
while [ 1 ]
do
 sleep 5
 find /path/to/inbox > inbox1.lst
 if [ "`diff inbox1.lst inbox2.lst | grep -f files.txt | grep ^\<`" ]
 then
  echo file "`diff inbox1.lst inbox2.lst | grep -f files.txt | grep ^\<`" received
 fi
 if [ "`diff inbox1.lst inbox2.lst | grep ^\>`" ]
 then
  sleep 5
  find /path/to/processed > processed1.lst
  if [ "`diff processed1.lst processed2.lst | grep -f files.txt | grep ^\<`" ]
  then
   echo file "`diff processed1.lst processed2.lst | grep -f files.txt | grep ^\<`" processed
   sleep 5
   find /path/to/outbox > outbox1.lst
   if [ "`diff outbox1.lst outbox2.lst | grep -f files.txt | grep ^\<`" ]
   then
    echo file "`diff outbox1.lst outbox2.lst | grep -f files.txt | grep ^\<`" ready to be transferred
   fi
   if [ "`diff outbox1.lst outbox2.lst | grep ^\>`" ]
   then
    sleep 5
    find /path/to/transferred > transferred1.lst
    if [ "`diff transferred1.lst transferred2.lst | grep -f files.txt | grep ^\<`" ]
    then
     echo files "`diff transferred1.lst transferred2.lst | grep -f files.txt | grep ^\<`" have been delivered
    fi
   fi
  fi
 fi
 mv inbox1.lst inbox2.lst
 mv processed1.lst processed2.lst
 mv outbox1.lst outbox2.lst
 mv transferred1.lst transferred2.lst
done
 
Old 02-09-2014, 10:12 AM   #9
Prince Imhotep
Member
 
Registered: Jan 2014
Posts: 45

Rep: Reputation: Disabled
Hi,


As I understood from the touch command at the begining, the files inbox2.lst, processed2.lst, outbox2.lst, transferred2.lst are all empty files, please correct me if I'm wrong.

Now in the conditions sections for inbox, processed, outbox, and transfered you're doing an unnecessary step -from my view point- which is diffing some file contents with an empty file, only to find lines written in it, please correct me if I'm wrong.

Instead you could have grepped whatever line inside all the files directly, for example:

Code:
grep -f files.txt inbox1.lst | grep ^\<`" received
You could've saved the piped grep by including the second pattern in your pattern file; files.txt.

Also you used the find command which is going to give a full path of a file inside the inbox1.lst, and I guess you only need stripped file names to facilitate comparisons, or string searching, please correct me if I'm wrong.

Instead you could have used
Code:
ls -1
for the directories you wanted to list their contents, also you could've included a line that creates a directory anywhere, just for the sake of executing your script and storing the temp data you need, then it gets deleted after printing out the information you need.

Also you could use system variables for the directories you wanted to list the contents of, to make your script more robust.

Hope this helped.

Good luck.
 
Old 02-10-2014, 09:43 AM   #10
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,028

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
Quote:
Originally Posted by Prince Imhotep View Post
Hi,


As I understood from the touch command at the begining, the files inbox2.lst, processed2.lst, outbox2.lst, transferred2.lst are all empty files, please correct me if I'm wrong.
yup, its to initialize the files so that diff doesnt complain about file not found the first time thru the loop. its probably not needed.
Quote:
Originally Posted by Prince Imhotep View Post

Now in the conditions sections for inbox, processed, outbox, and transfered you're doing an unnecessary step -from my view point- which is diffing some file contents with an empty file, only to find lines written in it, please correct me if I'm wrong.
the first time thru the loop it should find everything to be different. at the end we shift the files so each subsequent time thru the loop it should only find additions or removals to be different.
Quote:
Originally Posted by Prince Imhotep View Post

Instead you could have grepped whatever line inside all the files directly, for example:

Code:
grep -f files.txt inbox1.lst | grep ^\<`" received
i dont think this would find recent files that have been removed from the directory.
Quote:
Originally Posted by Prince Imhotep View Post
You could've saved the piped grep by including the second pattern in your pattern file; files.txt.
probably but i wouldve had to code a way to update finds output automatedly.
Quote:
Originally Posted by Prince Imhotep View Post

Also you used the find command which is going to give a full path of a file inside the inbox1.lst, and I guess you only need stripped file names to facilitate comparisons, or string searching, please correct me if I'm wrong.
either way, i figured not stripping the filenames was simpler.
Quote:
Originally Posted by Prince Imhotep View Post

Instead you could have used
Code:
ls -1
for the directories you wanted to list their contents, also you could've included a line that creates a directory anywhere, just for the sake of executing your script and storing the temp data you need, then it gets deleted after printing out the information you need.
i think its more portable to use find. some implimentations of ls (busybox) dont contain certain switches.
Quote:
Originally Posted by Prince Imhotep View Post

Also you could use system variables for the directories you wanted to list the contents of, to make your script more robust.
not sure what this means ?
Quote:
Originally Posted by Prince Imhotep View Post

Hope this helped.

Good luck.
this was more of a launchpad for the op to base their solution on but i am glad someone took the time to parse thru my errors. i honestly think that the op's pseudo-code was faulty but i would need their input to know for sure (in certain scenarios they start processing when files appear in the directory but in other scenarios they start processing when files are gone -- seems inconsistent).

Last edited by schneidz; 02-10-2014 at 09:47 AM.
 
Old 02-10-2014, 11:32 AM   #11
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 3,811
Blog Entries: 1

Rep: Reputation: 1346Reputation: 1346Reputation: 1346Reputation: 1346Reputation: 1346Reputation: 1346Reputation: 1346Reputation: 1346Reputation: 1346Reputation: 1346
Just want to point out that inotify was added to the kernel in 2.6.13. So if this is a newer system, you should have the ability to call on it.

You don't need the utility itself, you can create your own program to do the same.

Here is an example:

http://man7.org/tlpi/code/online/dis...inotify.c.html

Edit: You might need the headers file, and a compiler for this solution.

Last edited by szboardstretcher; 02-10-2014 at 11:39 AM.
 
1 members found this post helpful.
Old 02-10-2014, 11:39 AM   #12
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,028

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
also watch and/or screen mite work also.
 
Old 02-10-2014, 11:54 AM   #13
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
Thank you guys for all your help.

So far I've changed the appraoch for the script so many times lol...this is the approach I'm finally taking, this is just ehe logic for the first folder.

Code:
#!/bin/bash
set -e

IFS=$'\n'
lines=($(cat /tmp/EODFileAudit/inputFileName.txt))
linesout=($(cat /tmp/EODFileAudit/outputFileName.txt))


#Unitil results text file has in it , that the 2 files have been received, continue
#to monitor the directories

#until  sed -n ' File has been received /p' /tmp/EODFileAudit/result.txt    #&&  sed -n '"${lines[1]} File has been received /p "'  /tmp/EODFileAudit/result.txt | $? -eq "0"
#do

##If the name of the two files have not been reveived or logged in the result.txt, continue to monitor directory.

if grep -v "${lines[1]:0:7}$(date +"%Y%m%d").* File has been received" /tmp/EODFileAudit/result.txt && "${lines[1]:0:7$date +"%Y%m%d")} File has been received" /tmp/EODFileAudit/result.txt
then


ls  /home/qtscfp/sftp/iqplatform/processed | grep -E '^[^d]' | grep -E '^[^Total]' |




#if [ -f "/home/qtscfp/sftp/iqplatform/processed/${lines[0]:0:7$(date +"%Y%m%d")}*" ]  #&& [ sed -n "${lines[0]} File has been received/p " /tmp/EODFileAudit/result.txt | "$?" -ne "0"  ]
while read files
do
echo $files
if [[ "$files" = grep --include={*.txt,*.csv} -rnw '/home/qtscfp/sftp/iqplatform/processed/' -e "${lines[1]:0:7}$(date +"%Y%m%d")" ]]
then

printf "%s\n" "$files File has been received " > /tmp/EODFileAudit/result.txt

else

printf "%s\n" "$files File has NOT been received " > /tmp/EODFileAudit/result.txt

fi

done
 
Old 02-10-2014, 12:23 PM   #14
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
Also please note, that the name of the files expected are in the directory /tmp/EODFileAudit/inputFileName.txt ,the file names are kinda trick to I put them in an array and then extract then base names and then today's date as well as some additional numbering at then end.
(e.g) of sample file name =QUEOBTR20140207002_48638.txt

So this is my trouble , say QUEOBTR20140207002_48638.txt comes in and awaiting to be processed, I can get some of the name using the array
${lines[1]:0:7}$(date +"%Y%m%d") translates to QUEOBTR20140207 but I want 002_48638 which is randomly added, and if I do this line to check if the file is there.It doesnt work

ls /home/qtscfp/sftp/iqplatform/processed | grep -E '^[^d]' | grep -E '^[^Total]' |

while read files
do
echo $files
if [[ "$files" = grep --include={*.txt,*.csv} -rnw '/home/qtscfp/sftp/iqplatform/processed/' -e "${lines[1]:0:7}$(date +"%Y%m%d")*" ]]
then
printf "%s\n" "$files File has been received " > /tmp/EODFileAudit/result.txt
 
Old 02-10-2014, 12:25 PM   #15
cbtshare
Member
 
Registered: Jul 2009
Posts: 610

Original Poster
Rep: Reputation: 42
Quote:
Originally Posted by schneidz View Post
also watch and/or screen mite work also.
I did , or am doing it now, but I have a long list of files to search through to see if the files I want are there.

watch -n 1 -d ls -l sftp/iserver/inbox/ sftp/iserver/processed/ /home/server/outbox/ /home/server/vendor/outbox/transferred/

Last edited by cbtshare; 02-10-2014 at 12:28 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
real time sound monitoring niteen Programming 14 04-15-2013 01:56 PM
squid real time monitoring Ammad Linux - Networking 2 02-08-2010 12:46 PM
Squid Real-time Monitoring molybtek Linux - Networking 1 10-10-2008 10:42 PM
Real time Monitoring squid logs jatender Linux - Server 3 10-03-2007 04:08 AM
Keylogger / session monitoring in real time? houler Linux - Security 2 04-06-2005 10:21 PM


All times are GMT -5. The time now is 10:55 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration