LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 04-26-2010, 03:57 PM   #1
tuxtutorials
Member
 
Registered: Dec 2008
Location: New York
Distribution: RedHat, Solaris
Posts: 68

Rep: Reputation: 16
Efficient way of doing this?


Hello all, I would like some input on a small script I wrote to move a file that is received daily to one of our servers. My main question is regarding the file comparison piece where I do a compare between the two variables FILE_DATE and SYSTEM_DATE. I am pretty sure there is an easier way to do this but by brain couldn't figure it out. Any input would be great.


Code:
#!/bin/bash
#Description: Script to parse incoming file and archive if recent.


DATE=`date +%m-%d-%y`
INBOUND_DIR=/home/dropdir
ARCHIVE_DIR=/home/dropdir/archive
LOG=/home/dropdir/log



#Locate all files that have been placed in directory within last 10 hours.
cd $INBOUND_DIR
for FILE in `find $INBOUND_DIR/file* -mmin -600`
        do
        #Compare the day the file was placed in directory with date on system.
        FILE_DATE=`ls -l $FILE | awk '{ print $6,$7}'`
        SYSTEM_DATE=`date | awk '{ print $2,$3 }'`

        if [ "$FILE_DATE" = "$SYSTEM_DATE" ]; then
                echo "Moving received $FILE to archive on $DATE" >> $LOG
                mv $FILE $ARCHIVE_DIR
                /bin/mail -s "$FILE was recieved on $DATE and archived" me@example.com < /dev/null
        else
                echo "No file received on $DATE" >> $LOG
                /bin/mail -s "$FILE was not received on $DATE and not archived" me@example.com < /dev/null
        fi
done
exit

Thanks
 
Old 04-26-2010, 10:59 PM   #2
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
I am not quite sure how effective the test is for date?
Assuming I use the test on my machine for file dummy.txt:
Code:
 
FILE_DATE="Apr 27" #this is after awk command
SYSTEM_DATE="Apr 27" #as i have run this right now
My issue is that should I run this script at say 1am and have had a file created within the last 10 hours (as per your find)
then I will be moved to the "else" as the dates are not equal??

Wouldn't it be better to check if the files that are less than 10 hours old exist in the ARCHIVE_DIR?
So maybe it could look like:
Code:
#!/bin/bash
#Description: Script to parse incoming file and archive if recent.


DATE=`date +%m-%d-%y`
INBOUND_DIR=/home/dropdir
ARCHIVE_DIR=/home/dropdir/archive
LOG=/home/dropdir/log

#Locate all files that have been placed in directory within last 10 hours.
#cd $INBOUND_DIR #this is not required if your find is already looking there

for FILE in `find $INBOUND_DIR/file* -mmin -600`
do
        if [[ ! -e "$ARCHIVE_DIR/${FILE##*/}" ]]; then
                echo "Moving received $FILE to archive on $DATE" >> $LOG
                mv $FILE $ARCHIVE_DIR
                /bin/mail -s "$FILE was recieved on $DATE and archived" me@example.com < /dev/null
        else
                echo "No file received on $DATE" >> $LOG
                /bin/mail -s "$FILE was not received on $DATE and not archived" me@example.com < /dev/null
        fi
done
exit
 
Old 04-26-2010, 11:06 PM   #3
mrshanim
Member
 
Registered: Aug 2009
Posts: 30

Rep: Reputation: 16
Hi,
I have some thaught.
Find the files with mtime or -mmin/-cmin 600 (here 600 consider it is variable as temp)
IF you are running your script after 10am or later on the day then use temp as 600. if you are running your script before 10 am say 8am, in this case you need to find the files which created in last 8hrs because first 2 hrs falls in previous day.
Then out temp variable will be 480.

Algo

//Find temp variable
if current time is >>10am temp=600
else find temp value by getting current days minutes or last two variables of the time command

For find all files with -mmin temp (or -cmin temp)
move each file to target dir
end of for


Regards
Manish
 
Old 04-27-2010, 04:32 PM   #4
tuxtutorials
Member
 
Registered: Dec 2008
Location: New York
Distribution: RedHat, Solaris
Posts: 68

Original Poster
Rep: Reputation: 16
@grail regarding the tests for the date, the script would run at 10PM on the same day as the file being dropped. We usually get the file around 5PM on the same day. The logic behind this was that it compares the file with the current system date and if they match archive. Otherwise ignore. We had issues though when the cron job does not run on time and we have stray files dropped into the directory and files would not get archived. So my solution was to add the find command to only grab files 10 hours old. Not really sure if that test is necessary now since the find syntax only grabs files within the last 10 hours. I will take your logic for checking files in the archive directory and play with it.

Thanks for the input.
 
Old 04-28-2010, 04:25 AM   #5
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
A potential issue with this asynchronous file drop and copy procedure is copying files before they have been fully dropped. The most robust solution is for the dropper to drop the fie with a special name (example *.part) and to rename it on completion of the drop. If that is not possible, a less robust solution is for the copy program to note the length of the file, pause a while and check the length again, only copying when the length has not changed.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
cdbackup not efficient! ganninu Linux - General 2 01-18-2007 01:24 PM
[PDF] Is it efficient? Wim Sturkenboom General 8 01-11-2007 05:00 PM
C: storing string which is more efficient. debiant Programming 22 09-01-2006 01:39 AM
Simple is efficient ???? Maybe/Maybe not ! bigjohn General 21 07-08-2005 11:27 AM
a more efficient set up dr_zayus69 Linux - Newbie 3 12-03-2004 10:47 AM


All times are GMT -5. The time now is 03:31 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration