Please Check Script

ehegagoka · 02-12-2007, 08:49 PM

Hi,
We have a cron entry that runs a script twice with different parameters, here is the cron entry:

Quote:

0,30 * * * * /prosbnfdv/cronjobs/purgeFiles.sh /prosbnfdv/dataview/temp 30 MINS /prosbnfdv/dataview/logs/purgeFiles

1,30 * * * * /prosbnfdv/cronjobs/purgeFiles.sh /apps/newton/bin/gat/temp 30 MINS /prosbnfdv/dataview/logs/purgefiles

Now the purgeFiles.sh script contains this:

Quote:

#!/bin/sh

#----------------------------------------
# purgeFiles.sh
# Purges a specified path recursively that are more than x minutes/days old.
# Also deletes directories if the directory is empty.
#
# Revision History:
# v1.0 (Jake Catayoc)
# - initial release
#
# v1.1 (Jake Catayoc)
# - Added DAYS or MINS parameter to specify whether to delete x minutes
# or x days old.
# - Added logging facility
# - Improved error handling
#----------------------------------------

#--For Windows-- set PATH so that current directory is searched first
# May be deleted sometime if full Cygwin has been installed in Windows
PATH=.:$PATH

showhelp ()
{
echo "purgeFiles"
echo "----------"
echo "This script purges a specified path recursively of files"
echo "that are of specified minutes old. It will also remove"
echo "empty directories after deleting the affected files."
echo
echo "CAUTION: USE THIS SCRIPT WITH CARE! DELETED FILES MAY NO LONGER BE RESTORED!"
echo
echo "Usage: ./purgeFiles.sh <path_to_purge> <age> DAYS|MINS [path_to_log_file]"
echo.
echo Note: [path_to_log_file] defaults to ./logs if no path is specified.
}

# Ensure that we have correct parameters
if [ -z $1 ] || [ -z $2 ] || [ -z $3 ]; then
showhelp
exit 1
fi

# Ensure that <path_to_purge> is a valid path
if [ ! -d $1 ]; then
echo "Not a valid path: $1. Run purgeFiles.sh without arguments for help."
exit 1
fi

declare -i AGE=$2

# Ensure that <age> is a valid integer
if [ "$2" != "0" ]; then
if [ $AGE -eq 0 ]; then
echo "Age is a non-valid integer: $2. Run purgeFiles.sh without arguments for help."
exit 1
fi
fi

# Ensure that <age> is a positive integer
if [ $AGE -lt 0 ]; then
echo "Age is not a positive integer: $2. Run purgeFiles.sh without arguments for help."
showhelp
exit 1
fi

# Ensure that 3rd parameter is either DAYS or MINS, and set accordingly
case "$3" in
DAYS)
SCOPE='-mtime'
;;
MINS)
SCOPE='-mmin'
;;
*)
echo "Invalid age: $3. Must specify DAYS or MINS. Run purgeFiles.sh without arguments for help."
exit 1
;;
esac

# If [path_to_log_file] is specified, ensure that it is a valid path
if [ -z $4 ]; then
LOGFILE="./logs"
else
LOGFILE="$4"
fi
if [ ! -d $LOGFILE ]; then
echo "Not a valid log path: $LOGFILE. Run purgeFiles.sh without arguments for help."
exit 1
fi

LOGFILE="$LOGFILE/`date +purgeFiles_%F.log`"

# Start purging
echo "[`date`] Start purging $2 $3 old files from $1" >>$LOGFILE
find $1 -type f $SCOPE +$AGE -printf "[`date`] Deleting file %P (filedate: %t)\n" -exec rm -f {} \; >>$LOGFILE
#find $1 -depth -mindepth 1 -type d -empty -printf "[`date`] Removing empty directory %P\n" -exec rmdir {} \; >>$LOGFILE 2>>/dev/null
find $1 -type d $SCOPE +$AGE -printf "[`date`] Deleting file %P (filedate: %t)\n" -exec rm -rf {} \; >>$LOGFILE 2>>/dev/null
echo "[`date`] Finished purging files from $1" >>$LOGFILE
echo "" >>$LOGFILE

as you can see there's a "find" command that is comment out, that is the original code, then I replace it with this:

Quote:

find $1 -type d $SCOPE +$AGE -printf "[`date`] Deleting file %P (filedate: %t)\n" -exec rm -rf {} \; >>$LOGFILE 2>>/dev/null

because the files dont get deleted for the folders inside the "temp" folder, so I change it so that it will only check the folders INSIDE the temp folder given as a parameter. Now on the first day of the script the "temp" folder inside "gat" was not deleted. It is running fine, but on the next days the "temp" folder inside "gat" always get's deleted at around 6pm, but we've check other scripts and it is not doing anything to the gat/temp folder. Could the cron entry and the script have a case of deleting the temp folder inside gat? or is not possible and I should concentrate checking other things that might cause the deletion of the folder? Sorry for a bit lengthy of my question. Thank you so much Sir/Mam.

Regards,
Rhani

xflow7 · 02-12-2007, 09:38 PM

I think the purgeFiles script could be deleting your tempdir.

I've just done a few experiments with the find command and I believe that if the first find command (with -type f) removes any files that are in the dir 'temp' then these deletions count as modifications for the purposes of the -mmin option for the second find command and so the 'temp' dir itself matches in the second find command and thus gets removed.

Edit: I think if you add -mindepth 1 to the second find command (the one that checks directories) it will work as you intend. That is assuming I understand your intent.

ehegagoka · 02-13-2007, 02:13 AM

hi!,
thanks for the reply, I made some test scripts like the one I posted before to see if it does delete the "temp" folder. Please check if there's anything I forgot to note, because base on my test it did not delete the "temp" folder. Here's the test:

Code:


prosbnfdv@sbnftsyn:~/rrkear/temp> !1020
touch -t 02131515 file1
prosbnfdv@sbnftsyn:~/rrkear/temp> !1023
touch -t 02131515 file2
prosbnfdv@sbnftsyn:~/rrkear/temp> ls -lt
total 0
-rw-r--r--    1 prosbnfdv newtadm         0 2007-02-13 15:15 file1
-rw-r--r--    1 prosbnfdv newtadm         0 2007-02-13 15:15 file2
prosbnfdv@sbnftsyn:~/rrkear> cat testFind.sh
#!/bin/bash

find /home/prosbnfdv/rrkear/temp/ -type f -mmin +5 -printf "[`date`] Deleting file %P (filedate: %t)\n" -exec rm -f {} \;
find /home/prosbnfdv/rrkear/temp/ -type d -mmin +5 -printf "[`date`] Deleting file %P (filedate: %t)\n" -exec rm -rf {} \;
prosbnfdv@sbnftsyn:~/rrkear>
prosbnfdv@sbnftsyn:~/rrkear> ls
temp  testFind.sh
prosbnfdv@sbnftsyn:~/rrkear> ./testFind.sh
[Tue Feb 13 15:53:12 MYT 2007] Deleting file file1 (filedate: Tue Feb 13 15:15:00 2007)
[Tue Feb 13 15:53:12 MYT 2007] Deleting file file2 (filedate: Tue Feb 13 15:15:00 2007)
prosbnfdv@sbnftsyn:~/rrkear> ls -lR
.:
total 8
drwxr-xr-x    2 prosbnfdv newtadm      4096 2007-02-13 15:53 temp
-rwx------    1 prosbnfdv newtadm       260 2007-02-13 15:53 testFind.sh

./temp:
total 0
prosbnfdv@sbnftsyn:~/rrkear>

I can't recreate why with the previous script I posted the temp folder get's deleted. Thanks =)

Regards,
Rhani

xflow7 · 02-13-2007, 07:44 AM

Hi

I think I mis-spoke in my last reply. I think the problem will arise if the first find command doesn't remove any files.

That is, if nothing has been deleted or created in the temp folder within the specified time period, then the temp folder itself matches the timestamp criteria in the second find command.

Try repeating the test in your most recent post, but with 'temp' containing only files that have been updated recently and so don't get deleted by the first find command. I think you will see 'temp' get deleted.

I did another little test on my machine where I created a temp dir with some sub-dirs and a file. The file gets updated with some data, but note that this does not affect the timestamp of the temp dir because there was no file creation or destruction.

I then did a simplified version of the 'find -type d' command and you can see that it identified 'temp' as well as the old sub-dirs.

Here's the console text (I added spaces after the command outputs to ease reading)

Code:

drw1@coulthard:~$ ls -l temp
total 8
-rw-r--r-- 1 drw1 users    0 2007-02-13 08:33 afile
drwxr-xr-x 2 drw1 users 4096 2007-02-13 08:30 anotherdir/
drwxr-xr-x 2 drw1 users 4096 2007-02-13 08:30 yetanotherdir/

drw1@coulthard:~$ ls -ld temp
drwxr-xr-x 4 drw1 users 4096 2007-02-13 08:33 temp/

drw1@coulthard:~$ echo << EOF >temp/afile
> Some changes to afile
> EOF

drw1@coulthard:~$ ls -l temp
total 12
-rw-r--r-- 1 drw1 users    1 2007-02-13 08:35 afile
drwxr-xr-x 2 drw1 users 4096 2007-02-13 08:30 anotherdir/
drwxr-xr-x 2 drw1 users 4096 2007-02-13 08:30 yetanotherdir/

drw1@coulthard:~$ ls -ld temp
drwxr-xr-x 4 drw1 users 4096 2007-02-13 08:33 temp/

drw1@coulthard:~$ date
Tue Feb 13 08:37:33 EST 2007

drw1@coulthard:~$ find testdir -type d -mmin +3
testdir
testdir/defaultdir
testdir/otherdir
testdir/anewdir

drw1@coulthard:~$

As noted above, I think -mindepth 1 would fix this.