cron to remove all files within directory except one file

restarea · 06-28-2010, 03:39 AM

Hi All,
I'm a newbie here, so please help

I would like to create a cronjob that will delete all files within a directory 1 hours after it is created to the folder

I found this cron

find /path/to/file/* -ctime +1 -exec rm {} \; but it's deleted all files.
I want to make an exception, all file should be deleted except one file (letsay file a.zip)

How can I do that?

Thank you

rical · 06-28-2010, 04:02 AM

There is 2 different approaches, ether you can match all file that you want to delete with a regexp (or name) or you can exclude the one you don't want with a regexp (or name).

Example of excluding a specific file:
find /path/to/file/* -not -iname "a.zip" -ctime +1 -exec rm {} \;

Hope this helps

colucix · 06-28-2010, 04:04 AM

Hi and welcome to LinuxQuestions!

Quote:

Originally Posted by restarea

I found this cron

Where did you find it? This is not a "cron", it's just a find command line (very dangerous if you don't really know how it works, because of the -exec rm part). Also it is malformed.

Look at this instead:

Code:

find /path/to/dir -mtime +1 -type f ! -name a.zip -exec echo rm {} \;

First it specifies just a directory as search path (it will be descended recursively). Second I assume you know the difference between ctime (that is not the creation time), mtime and atime and also what +1 really mean for time tests in the find command. Third, it uses echo before the rm command for testing purposes: it does not actually removes anything before you've checked the results and re-run the command again without the echo statement. This is a good practice. On the contrary, what if you accidentally had left a space between "/" and "path/to/file"? It would have removed all your root directory tree and/or your user's home in a bunch of seconds!

Finally, to answer your question, you might add the -name test specifying the name of the file you want to exclude from the search results, using the negation "!" operator, as in my example above. Hope this helps.

restarea · 06-28-2010, 04:38 AM

wow thanks for fast response @colucix and rical

Quote:

Where did you find it? This is not a "cron", it's just a find command line (very dangerous if you don't really know how it works, because of the -exec rm part). Also it is malformed.

mmmm I found it somewhere after a google-ing, I put that line on my cron field at cpanel

@colucix I'll try it first and let you know the result

Thanks

restarea · 06-28-2010, 05:13 AM

Quote:

Originally Posted by colucix

Hi and welcome to LinuxQuestions!

Second I assume you know the difference between ctime (that is not the creation time), mtime and atime and also what +1 really mean for time tests in the find command.

@colucix I think -ctime -1 means :
all files were created 1 hour before. Is that correct? sorry

colucix · 06-28-2010, 05:32 AM

Quote:

Originally Posted by restarea

@colucix I think -ctime -1 means :
all files were created 1 hour before. Is that correct? sorry

Not really. -ctime is not the creation time, since most linux filesystems don't store that information. An exception is the recent ext4, but most applications don't take it in account yet.

Please, take a look here for a little but exhaustive explanation of time test in GNU find. As you can see the n, -n or +n specifications are multiples of 24-hours, hence -1 means less than 24 hours ago (but you also have to take rounding effects in account).

If you really want to look for files modified less than 1 hour ago, you may consider -newer. First you may create an empty file with the proper timestamp, then look for more recent files and finally remove the newly created temporary file. Example:

Code:

touch -t $(date -d "- 1 hour" +%Y%m%d%H%M) /tmp/my_dummy_file
find /path/to/dir -type f -newer /tmp/my_dummy_file ! -name a.zip -exec echo rm {} \;
rm /tmp/my_dummy_file

restarea · 06-28-2010, 06:03 AM

thx @colucix

btw, this is my condition

I set up a web space where my friend and I will upload files via ftp or via web then sharing them with other friends. Some kinda private file hosting.
And I want any files uploaded after one hours should be deleted automatically to save space

I tried your previous suggestion but looks like it's not working.
The files are still there

Thanks

colucix · 06-28-2010, 06:28 AM

Ok, sorry. So you want to remove files older than one hour, don't you? That is after one hour from the upload, they should be deleted. This should do the work:

Code:

#
#  Create an empty file with timestamp "1 hour ago"
#
touch -t $(date -d "- 1 hour" +%Y%m%d%H%M) /tmp/my_dummy_file
#
#  Find all files 1 or more hours old (exclude some of them based on their name)
#
find /path/to/dir -type f ! -newer /tmp/my_dummy_file ! -name file_to_exclude -exec echo rm {} \;
#
#  Make clean (remove the temporary empty file)
#
rm /tmp/my_dummy_file

Note the negation before -newer, now. Please, test the above from the command line (it should just print out the resulting rm commands without actually remove anything) and see if the result is satisfactory.

Note that you can exclude more than one file from the search using the -o operator in the following way:

Code:

find /path/to/dir -type f ! -newer /tmp/my_dummy_file ! \( -name first_to_exclude -o -name second_to_exclude \) -exec echo rm {} \;

and add more of them inside the escaped parentheses. You can also use globbing (*) to match files with a common pattern or the -regexp test, as previously suggested by rical.

Finally if you want to run this through cron I suggest to put the statements inside a script and execute it from the crontab, but - if you prefer - you can always put it in one command line, e.g.

Code:

0 * * * * touch -t $(date -d "- 1 hour" +\%Y\%m\%d\%H\%M) /tmp/my_dummy_file && find /path/to/dir -type f ! -newer /tmp/my_dummy_file ! -name file_to_exclude -exec rm {} \; && rm /tmp/my_dummy_file

rical · 06-28-2010, 06:43 AM

Quote:

Originally Posted by restarea

thx @colucix

btw, this is my condition

I set up a web space where my friend and I will upload files via ftp or via web then sharing them with other friends. Some kinda private file hosting.
And I want any files uploaded after one hours should be deleted automatically to save space

I tried your previous suggestion but looks like it's not working.
The files are still there

Thanks

You probably could do it with a script.

Code:

#!/bin/bash
time_stamp=`stat -c '%Y' $1`
time_stamp=$((`date +%s`-$time_stamp))
if [ $time_stamp -gt 3600 ];then
        # Older then 1h, remove
        rm -r $1
fi

and execute it (ON YOUR OWN RISK) with

find /path/to/file/ -not -iname "a.zip" -exec ./script.sh {} \;

restarea · 06-28-2010, 07:09 AM

@colucix
thank you for your kindness and help

so this is what I wrote :

Code:

touch -t $(date -d "- 1 hour" +\%Y\%m\%d\%H\%M) /home/xxx/public_html/dummy_file && find /home/xxx/public_html/upload -type f ! -newer /home/xxx/public_html/dummy_file ! -name file_to_exclude -exec rm {} \; && rm /home/xxx/public_html/dummy_file

is that correct?

is the dummy_file folder created automatically or I have to create it manually?

Thanks

colucix · 06-28-2010, 07:32 AM

Yes, it looks correct. The folder containing the dummy file must be existent, on the contrary you have to create it manually. Indeed, the touch command creates only non-existent files (not directories).

Two notes: 1. don't forget to put the actual name of the file to exclude in your command line, 2. note the escaped % in the date command: they are necessary only in crontab entries, since the % sign has a special meaning there (see man 5 crontab for details), 3. I would actually test the cron job again using echo before rm and sending the output (along with any standard error) to a file for further checking, for example:

Code:

0 * * * * touch blah blah && find blah blah > $HOME/cron.log 2>&1 && rm blah blah

In this way you can avoid surprises. Am I a bit paranoid?

restarea · 06-28-2010, 08:15 AM

I've tested it and looks it works properly
but i got this warning msg in my email
rm: cannot remove 'home/xxx/public_html/dummy_file' : Is a directory

colucix · 06-28-2010, 08:36 AM

Most likely you've previously created a directory with that name. The rest works properly, because the touch command changes the timestamp of existing directories, as well. Maybe I misunderstood your last question but I meant that dummy_file should be a regular file (inside a certain directory). Now you can simply rmdir the dummy_file directory and let the cron job do its work.

restarea · 06-28-2010, 12:53 PM

it works!!!
thanks alot @colucix and also @rical