LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
Reload this Page [SOLVED] Shell script that modifies/touches files in an unappreciated manner ...
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-01-2010, 10:05 AM   #1
dr_strangelove
LQ Newbie
 
Registered: Jun 2010
Location: London, UK
Posts: 10

Rep: Reputation: 0
Shell script that modifies/touches files in an unappreciated manner ...

Hello

I am not a complete newbie to Linux, but I'm most definitely a newbie to linuxquestions.org and the solution to my query may well be obvious; my apologies if so.

I've written a BASH script (running on SuSE Server) that I've set to run at regular intervals using the CRON daemon. The script trawls through a pre-defined directory, peers into each subdirectory and then checks for the presence of certain files against the contents of a 'text' file within this directory. If all files are present, the script creates a 'TRIGGER' file that is used to stimulate another system picking up the entire contents of this sub-directory. Else, it does nothing. The script works in every way, except that it appears to 'touch' the subdirectories being checked and amends their modification date. This is an issue as I cannot use 'modified date' rules to purge this folder going forward.

The script can be found below:

Code:
#!/bin/bash
#This script confirms that each folder within the drop directory contains all files referenced in the text file.


DROPDIR="/home/ftp_root/"											#Define the dropdirectory, within which the subfolders, text files etc are dropped.  NB* Remember to include a forward oblique at the end of the location.
TRIGGER="TRIGGER"													#This is the 'TRIGGER' file that is created
LOGDIR="/home/ftp_root/log/"										#Define the log folder location/  NB* Remember to include a forward oblique at the end of the location.

#This section creates our variables for use later in the log.
THEDATE=$(date +"%Y-%m-%d")
THETIME=$(date +"%T")

#We also check to see if a log file already exists, if not, create one with headings.
if [ -e $LOGDIR$THEDATE.csv ];
	then
		echo $THEDATE,$THETIME,-,Running Script 1>>$LOGDIR$THEDATE.csv
	else
		echo Trigger Script Log $THEDATE 1>>$LOGDIR$THEDATE.csv
		echo Date,Time,File,Message/Event 1>>$LOGDIR$THEDATE.csv
		echo $THEDATE,$THETIME,-,Running Script 1>>$LOGDIR$THEDATE.csv
	fi
#echo Drop directory: $DROPDIR							# Un-comment to include additional logging.
cd $DROPDIR												# Change to the drop directory

for SUBDIR in `ls -d *` ; do							# Loop through the drop directory and cycle through each folder
cd $SUBDIR

#echo PWD: $PWD

# Now for each directory in the drop folder we check the text folder.
SWITCH=0												# We set the switch to 0 here for each directory iteration:

if [ -e $DROPDIR$SUBDIR/$SUBDIR.txt ];
	then
cp $DROPDIR$SUBDIR/$SUBDIR.txt /home/TRIGGER/$SUBDIR.txt
dos2unix /home/TRIGGER/$SUBDIR.txt

IFS='
'
file=( $( < /home/TRIGGER/$SUBDIR.txt ) )					# Here we grab the contents of the text file within the folder

for (( i = 0 ; i < ${#file[@]} ; i++ ))					# Here we loop through its contents and check for the presence of each line
do

tpath=$(echo ${file[$i]} | sed -e 's/\\/\//g')			# This line creates a new variable with any reverse obliques being substituted for forward oblique, as the text file was created on a Windows box

if [ -e $DROPDIR$SUBDIR/$tpath ];
	then
		THEMESSAGE="$DROPDIR$SUBDIR/${file[$i]},ARRIVED"
		#echo $THEDATE,$THETIME,$THEMESSAGE 1>>$LOGDIR$THEDATE.csv							# Un-comment to include additional logging.
     	else
        SWITCH=1							# ...  we set the switch to '1' if we can't find a file
		THEMESSAGE="$DROPDIR$SUBDIR/${file[$i]},ABSENT"
		#echo $THEDATE,$THETIME,$THEMESSAGE 1>>$LOGDIR$THEDATE.csv							# Un-comment to include additional logging.
    	fi
done
if [ $SWITCH == 0 ];							# ... if the switch remains '0' then we assume that all files are present
	then
		echo $SUBDIR is READY
		touch $DROPDIR/TRIGGER/$TRIGGER.$SUBDIR.txt			# ... if all files are ready then we touch $TRIGGER - this is the file we are to use as a trigger
		THEMESSAGE="$DROPDIR$SUBDIR/${file[$i]},TRIGGER CREATED"
		#echo $THEDATE,$THETIME,$THEMESSAGE 1>>$LOGDIR$THEDATE.csv							# Un-comment to include additional logging.
	else
		echo $SUBDIR is NOT READY
	fi
	
	
	else
		THEMESSAGE="$DROPDIR$SUBDIR/$SUBDIR.txt,TEXT FILE ABSENT"
		#echo $THEDATE,$THETIME,$THEMESSAGE 1>>$LOGDIR$THEDATE.csv							# Un-comment to include additional logging.
	fi
rm /home/TRIGGER/$SUBDIR.txt
cd $DROPDIR
done
echo $THEDATE,$THETIME,-,Script complete 1>>$LOGDIR$THEDATE.csv
I cannot seem to isolate the area of code which is amending the 'modified date' of the subdirectories. If someone could point me in the correct direction I would really appreciate it.

Thanks in advance for your patience,

dr_strangelove
Last edited by dr_strangelove; 06-01-2010 at 10:22 AM. Reason: ... I've got terrible spelling ...
 
Old 06-01-2010, 10:29 AM   #2
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 92
Hi

Can you post an example of running the script including ls -la before and after?

Looking through it, I don't see anywhere where you either add or remove anything within any of your $SUBDIR's - can't see how the sub-directories are being modified.
 
Old 06-01-2010, 10:36 AM   #3
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Quote:
Originally Posted by dr_strangelove View Post
I cannot seem to isolate the area of code which is amending the 'modified date' of the subdirectories.
Me neither. What are the symptoms which make you think it is the script which is changing the modification time of the subdirectories? AFAIK the modification time of a directory is the last time that a file was created in it or deleted from it.
 
Old 06-01-2010, 10:51 AM   #4
dr_strangelove
LQ Newbie
 
Registered: Jun 2010
Location: London, UK
Posts: 10

Original Poster
Rep: Reputation: 0
Hi catkin, alli_yas

Thanks to you both for your super-swift replies.

Quote:
Can you post an example of running the script including ls -la before and after?
The script last ran @ 16:18 via CRON, I then removed the script from the CRONTAB:
Code:
total 144
drwxrwxr-x 34 moveit_ftp xxxx_users 12288 Jun  1 11:40 .
drwxrwxrwx  8 moveit_ftp users           4096 May 24 17:27 ..
drwxrwxr-x  2 moveit_ftp users           4096 Mar 10 14:36 ._TMD EDL Extra JE test 26th Feb v5
drwxrwxr-x  2 moveit_ftp users           4096 May 29 23:23 1058014
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058021
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058087
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058127
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058145
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058152
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058163
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058193
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058293
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058306
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058329
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058348
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058360
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058368
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058375
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058433
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058476
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058573
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058582
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058633
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1058640
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1059073
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:38 1069969
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:38 1075455
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:38 1078425
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:38 1078501
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:38 1082625
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:38 1084113
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:38 1097394
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 14:16 TRIGGER
drwxrwxr-x  2 moveit_ftp users           4096 May 29 23:28 testSD
I manually ran the script at 16:43 and:
Code:
total 144
drwxrwxr-x 34 moveit_ftp xxxx_users 12288 Jun  1 11:40 .
drwxrwxrwx  8 moveit_ftp users           4096 May 24 17:27 ..
drwxrwxr-x  2 moveit_ftp users           4096 Mar 10 14:36 ._TMD EDL Extra JE test 26th Feb v5
drwxrwxr-x  2 moveit_ftp users           4096 May 29 23:23 1058014
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058021
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058087
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058127
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058145
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058152
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058163
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058193
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058293
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058306
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058329
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058348
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058360
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058368
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058375
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058433
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058476
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058573
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058582
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058633
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1058640
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:43 1059073
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 16:44 1069969
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:44 1075455
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:44 1078425
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:44 1078501
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:44 1082625
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:44 1084113
drwxrwxr-x  3 moveit_ftp users           4096 Jun  1 16:44 1097394
drwxrwxr-x  2 moveit_ftp users           4096 Jun  1 14:16 TRIGGER
drwxrwxr-x  2 moveit_ftp users           4096 May 29 23:28 testSD
Quote:
What are the symptoms which make you think it is the script which is changing the modification time of the subdirectories?
I'm checking the ftp folder in Filezilla and I can see the modified date/time increments 'in-sync' with the script CRON task running. The frequency of the updates change as I amend the frequency of the CRON task/script.

Hope to speak soon,

dr_strangelove
 
Old 06-01-2010, 10:53 AM   #5
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 92
Can you post your crontab for the user that's running the script?

Code:
user@yourbox$ crontab -l
 
Old 06-01-2010, 11:05 AM   #6
dr_strangelove
LQ Newbie
 
Registered: Jun 2010
Location: London, UK
Posts: 10

Original Poster
Rep: Reputation: 0
Quote:
Can you post your crontab for the user that's running the script?
Code:
CHSWK-FTP01:/ # crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.XXXXZrgr3g installed on Tue Jun  1 16:38:37 2010)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
*/20 * * * * /home/TRIGGER/THESCRIPT.sh
59 23 */1 * * find /home/ftp_root/MEDIA/Support/TRIGGER_log/* -mtime +7 -exec rm {} \;
 
Old 06-01-2010, 12:23 PM   #7
smoker
Senior Member
 
Registered: Oct 2004
Distribution: Fedora Core 4, 12, 13, 14, 15, 17
Posts: 2,279

Rep: Reputation: 250Reputation: 250Reputation: 250
Where does dos2unix make it's temporary file while working ? Does it make a temporary file ?
Because it is being run from each subdir.

You could try removing it from the loop and running it on all /home/TRIGGER/*txt files right at the end of the script.

Which version of SUSE are you running because I found this ...
Quote:
dos2unix used to create a temporary file in the current directory and rename it to the target file after conversion.
http://support.novell.com/techcenter...ee97957e7.html
Last edited by smoker; 06-01-2010 at 12:32 PM.
 
1 members found this post helpful.
Old 06-01-2010, 12:35 PM   #8
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Well spotted, smoker
 
Old 06-01-2010, 01:00 PM   #9
smoker
Senior Member
 
Registered: Oct 2004
Distribution: Fedora Core 4, 12, 13, 14, 15, 17
Posts: 2,279

Rep: Reputation: 250Reputation: 250Reputation: 250
Quote:
Originally Posted by catkin View Post
Well spotted, smoker
Fingers crossed, eh Charles ...
 
1 members found this post helpful.
Old 06-01-2010, 01:29 PM   #10
dr_strangelove
LQ Newbie
 
Registered: Jun 2010
Location: London, UK
Posts: 10

Original Poster
Rep: Reputation: 0
Thumbs up
That is indeed the most excellent spot smoker.

I've just tried changing the working directory at the appropriate point in the loop and the script appeared to not update the timings! I've re-ran it several times and it's tickety-boo.

Code:
cp $DROPDIR$SUBDIR/$SUBDIR.txt /home/TRIGGER/$SUBDIR.txt
cd /home/TRIGGER/
dos2unix /home/TRIGGER/$SUBDIR.txt
cd $DROPDIR$SUBDIR
<GUSHING>Without doubt, the most satisfactory forum experience. I am in awe.</GUSHING>

Thanks to you all for your time and expertise with this; it is appreciated muchly.

Quote:
... Which version of SUSE are you running ...
Linux version 2.6.27.7-9-pae (geeko@buildhost) (gcc version 4.3.2 [gcc-4_3-branch revision 141291] (SUSE Linux) )
Last edited by dr_strangelove; 06-01-2010 at 01:35 PM.
 
Old 06-01-2010, 01:47 PM   #11
smoker
Senior Member
 
Registered: Oct 2004
Distribution: Fedora Core 4, 12, 13, 14, 15, 17
Posts: 2,279

Rep: Reputation: 250Reputation: 250Reputation: 250
That seems like a fairly late version of SUSE, I would have thought the issue with dos2unix would have been fixed. Depends where you got it from I guess.

As for the spot, it wasn't immediate I can tell you. But in true Sherlock Holmes style, if you eliminate the impossible, then whatever remains must be the truth.

It's nice when it turns out to be someone elses fault isn't it !
Your script was fine all along.

Cheers


(Now I'm off to try and annoy a moderator JOKING !!!)
 
Old 06-02-2010, 06:08 AM   #12
dr_strangelove
LQ Newbie
 
Registered: Jun 2010
Location: London, UK
Posts: 10

Original Poster
Rep: Reputation: 0
It's the fact that it wasn't 'immediate' that makes me value your help to the n'th degree.

All the best,

dr_strangelove
 
Old 06-02-2010, 06:25 AM   #13
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Quote:
Originally Posted by smoker View Post
But in true Sherlock Holmes style, if you eliminate the impossible, then whatever remains must be the truth.
Elementary, my dear Watson! But none the less a commendable bit of techno-sleuthing.
 
  


Reply

Tags
bash, date, modified, script, shell, suse, touch



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Apache modifies files owned by root mrtndimitrov Linux - Newbie 6 03-24-2010 04:22 AM
how can I differentiate two large text files using shell script? Files are like below surya_gadde Linux - Software 1 01-20-2009 02:52 AM
Sabayon modifies files on other distro users during install Viper Chief Linux - Distributions 3 06-14-2008 04:03 PM
Ubuntu GUI/Shell in the manner of OS X/Mac Ainkut Linux - General 4 04-30-2008 07:05 PM
Force Linux to run shell script in FIFO manner _hadi_ Programming 3 12-17-2006 02:12 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 08:55 PM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration