LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 01-26-2005, 07:26 AM   #1
fiservguy
Member
 
Registered: Nov 2004
Distribution: RHEL 2.1, RHEL 3.0, SUSE 9.2
Posts: 31

Rep: Reputation: 16
Question Help with clean-up script


I'm new to shell scripting, and would like some guidance in solving a file cleanup problem we have with one of our applications.

We have an app that archives files in several different locations. The files are stored in the following manner:
A directory is created with the name format Y????M??
(i.e. Y2004M12)
A subdirectory is created with the name format D??
(i.e. D01 or D15)
archived files for a given day are stored in the D?? directory.

Managing the files is relatively simple -- we want to keep them for 3 days, so

find $SOURCEDIR -mtime +3 -type f -exec rm {} \;

will delete files older than 3 days. Where I get stuck is getting rid of directories that are emptied by deleting these files.

I've thought of counting files in directories and deleting any directories with a filecount of 3 (what you get if you do a ls -al | wc -l on an empty directory), but I can't figure out how to either loop through existing directories, or search for directories with the correct name format and delete them if they're empty.

Any help or suggestions would be greatly appreciated.

(As an aside, I went looking for online script archives to see if anyone had already solved this problem, but I couldn't find anything. I vaguely remember there used to be such sites -- do they still exist?)

Thanks,

fiservguy
 
Old 01-26-2005, 07:46 AM   #2
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
Another type that find uses is 'd' for directory.

find $SOURCEDIR -mtime +3 -type d -exec rmdir "{}" \;

You could use: rm -rf "{}" \; instead. The double quotes help when a file name might contain whitespace or special characters.
 
Old 01-26-2005, 07:53 AM   #3
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
Another type that find uses is 'd' for directory.

find $SOURCEDIR -mtime +3 -type d -exec rmdir "{}" \;

You could use: rm -rf "{}" \; instead. The double quotes help when a file name might contain whitespace or special characters.

You can also use more than one test if you want only directories matching a certain pattern removed.

find $SOURCEDIR -mtime +3 -type d ( iname "Y[[:digit:]][[:digit:]][[:digit:]][[:digit:]]M[[:digit:]][[:digit:]]" -o iname "D[[:digit:]][[:digit:]]" ) -exec rmdir {} \;
 
Old 01-26-2005, 08:41 AM   #4
fiservguy
Member
 
Registered: Nov 2004
Distribution: RHEL 2.1, RHEL 3.0, SUSE 9.2
Posts: 31

Original Poster
Rep: Reputation: 16
Thanks for the quick reply.

If I order the directory searches (D?? first, then Y????M??), and do both of those after deleting the files, and if I only use an rmdir command, I won't be able to delete any non-empty directories.

I had briefly considerd something like this, but wasn't sure if deleting files out of a directory modifies the mtime value. I'll test it and let you know my results.

Thanks again!

fiservguy
 
Old 01-26-2005, 11:10 AM   #5
fiservguy
Member
 
Registered: Nov 2004
Distribution: RHEL 2.1, RHEL 3.0, SUSE 9.2
Posts: 31

Original Poster
Rep: Reputation: 16
OK. Here's what I ran:

find $SOURCEDIR -mtime +3 -type f -exec rm {} \;

find $SOURCEDIR -name "D[0-9][0-9]" -type d -mtime +3 -exec rmdir {} \;

find $SOURCEDIR -name "Y[0-9][0-9][0-9][0-9]M[0-9][0-9]" -type d -mtime +3 -exec rmdir {} \;

Here's what happened:

The files were deleted fine
No D?? directories were found with an mtime of +3
No Y????M?? directories were deleted because none were empty

So, deleting files from a directory changes the mtime of the directory.

I looked at the man page for 'find' and found the flag -empty , which looks for empty files or directories. I made the following changes:

find $SOURCEDIR -mtime +3 -type f -exec rm {} \;

find $SOURCEDIR -name "D[0-9][0-9]" -type d -empty -exec rmdir {} \;

find $SOURCEDIR -name "Y[0-9][0-9][0-9][0-9]M[0-9][0-9]" -type d -empty -exec rmdir {} \;

This works, but oddly generates an error for each of the directories that is deleted:

find: <$SOURCEDIR>/Y2005M01: No such file or directory

The directories did exist, however, and now don't. So the find/delete worked. I'm not sure why this error is being generated for each directory deleted.

I've got some more testing to do, but this has put me on the right track. Thanks for your help. If you can explain the error, let me know.

fiservguy
 
Old 01-27-2005, 12:59 AM   #6
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
Thanks for pointing out my mistake about the directories 'mtime' being modified. I should of tested it on directories before submitting. Oh, well, now you taught me something.

The 'find' command is possibly the most useful, but there are so many commands, a person needs to resort to scanning the manpage. I usually use -cmin when looking for files or directories, but when I tested it, the ctime test was also changed when a file was added to a directory. I guess the directory must be replaced when a change is made. I think that -ctime stands for 'create time'.

For your information, here is a LinuxPlanet tutorial on time in Linux:
http://linuxplanet.com/linuxplanet/tutorials/215/1/
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
clean up after update wjacobusse Fedora 2 03-06-2005 07:03 AM
make clean Palamides Linux - Software 5 01-13-2005 01:42 AM
Simple script to clean up old file rbeckett Red Hat 2 09-09-2004 02:38 PM
Raid not clean u4113072 Linux - Software 0 10-20-2002 10:20 PM
How can I do a clean-up? J_Szucs Linux - Newbie 3 04-30-2002 04:17 PM


All times are GMT -5. The time now is 09:53 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration