Help with clean-up script
 

I'm new to shell scripting, and would like some guidance in solving a file cleanup problem we have with one of our applications.

We have an app that archives files in several different locations. The files are stored in the following manner:
A directory is created with the name format Y????M??
(i.e. Y2004M12)
A subdirectory is created with the name format D??
(i.e. D01 or D15)
archived files for a given day are stored in the D?? directory.

Managing the files is relatively simple -- we want to keep them for 3 days, so

find $SOURCEDIR -mtime +3 -type f -exec rm {} \;

will delete files older than 3 days. Where I get stuck is getting rid of directories that are emptied by deleting these files.

I've thought of counting files in directories and deleting any directories with a filecount of 3 (what you get if you do a ls -al | wc -l on an empty directory), but I can't figure out how to either loop through existing directories, or search for directories with the correct name format and delete them if they're empty.

Any help or suggestions would be greatly appreciated.

(As an aside, I went looking for online script archives to see if anyone had already solved this problem, but I couldn't find anything. I vaguely remember there used to be such sites -- do they still exist?)

Thanks,

fiservguy

Another type that find uses is 'd' for directory.

find $SOURCEDIR -mtime +3 -type d -exec rmdir "{}" \;

You could use: rm -rf "{}" \; instead. The double quotes help when a file name might contain whitespace or special characters.

Another type that find uses is 'd' for directory.

find $SOURCEDIR -mtime +3 -type d -exec rmdir "{}" \;

You could use: rm -rf "{}" \; instead. The double quotes help when a file name might contain whitespace or special characters.

You can also use more than one test if you want only directories matching a certain pattern removed.

find $SOURCEDIR -mtime +3 -type d ( iname "Y[[:digit:]][[:digit:]][[:digit:]][[:digit:]]M[[:digit:]][[:digit:]]" -o iname "D[[:digit:]][[:digit:]]" ) -exec rmdir {} \;

Thanks for the quick reply.

If I order the directory searches (D?? first, then Y????M??), and do both of those after deleting the files, and if I only use an rmdir command, I won't be able to delete any non-empty directories.

I had briefly considerd something like this, but wasn't sure if deleting files out of a directory modifies the mtime value. I'll test it and let you know my results.

Thanks again!

fiservguy

OK. Here's what I ran:

find $SOURCEDIR -mtime +3 -type f -exec rm {} \;

find $SOURCEDIR -name "D[0-9][0-9]" -type d -mtime +3 -exec rmdir {} \;

find $SOURCEDIR -name "Y[0-9][0-9][0-9][0-9]M[0-9][0-9]" -type d -mtime +3 -exec rmdir {} \;

Here's what happened:

The files were deleted fine
No D?? directories were found with an mtime of +3
No Y????M?? directories were deleted because none were empty

So, deleting files from a directory changes the mtime of the directory.

I looked at the man page for 'find' and found the flag -empty , which looks for empty files or directories. I made the following changes:

find $SOURCEDIR -mtime +3 -type f -exec rm {} \;

find $SOURCEDIR -name "D[0-9][0-9]" -type d -empty -exec rmdir {} \;

find $SOURCEDIR -name "Y[0-9][0-9][0-9][0-9]M[0-9][0-9]" -type d -empty -exec rmdir {} \;

This works, but oddly generates an error for each of the directories that is deleted:

find: <$SOURCEDIR>/Y2005M01: No such file or directory

The directories did exist, however, and now don't. So the find/delete worked. I'm not sure why this error is being generated for each directory deleted.

I've got some more testing to do, but this has put me on the right track. Thanks for your help. If you can explain the error, let me know.

fiservguy

Thanks for pointing out my mistake about the directories 'mtime' being modified. I should of tested it on directories before submitting. Oh, well, now you taught me something.

The 'find' command is possibly the most useful, but there are so many commands, a person needs to resort to scanning the manpage. I usually use -cmin when looking for files or directories, but when I tested it, the ctime test was also changed when a file was added to a directory. I guess the directory must be replaced when a change is made. I think that -ctime stands for 'create time'.

For your information, here is a LinuxPlanet tutorial on time in Linux:
http://linuxplanet.com/linuxplanet/tutorials/215/1/