Deleting files after x days from the latest date file

sriramsreedhars · 03-01-2007, 05:03 AM

I need to keep files for X days from latest date (not just one day)on a linux system then delete rest:

More Details :

Can someone suggest on this script :

let's say the logs files are available for Jan 07, Dec 06, Nov 06, Oct 06

the script should identify the latest months logs, i.e Jan 07

it should then delete anything older than 2 months, which will be logs for Nov 06 & Oct 06.

stress_junkie · 03-01-2007, 07:09 AM

The problem that you face here is that Linux and Unix do not record the creation date of a file. The date that the file was last accessed and the date that the file was last modified are saved. You can either depend on one of those settings with the chance that old files will not be removed because they were last modified or accessed some time since they were created or you can record the creation date inside the file. I would choose to use the last modified date as the criteria. In that case you can use the find command to find all of the files in a directory tree that were modified more than 60 days ago, for example. The syntax of the find command is:

Code:

find <search directory> <criteria> [ -exec command [ parameters ] {} \;

So if you want to search the /var/log/mylogs directory tree for all files that were modified more than 60 days ago and delete these files the command would look like this.

Code:

find /var/log/mylogs -type f -ctime +1440 -exec rm {} \;

Here the ctime is measured in hours so 60 days times 24 hours is 1440. The plus sign in front of the 1440 means "more than". So this command will find files that were last modified/changed more than 60 days ago and it will remove them.

More information about the find command is available in the man pages.

Code:

man find

sriramsreedhars · 03-01-2007, 09:03 AM

No thats not what I want

1) Find the latest date file
Say 2003-march-21 is the last file available in that directory,
then it should look for files older than that for 2 months and delete it.

Which means it should delete 2003-jan-21 and above files

I hope you are getting it this time.

Heres a script I have worked out on which finds the latest date and deletes restall file, but I want it to delete x no of days from the latest date file :

bash-3.00$ cat script.sh
#!/bin/bash
Keepdate=`/bin/ls -ult --full-time| sed 1d | head -1 | awk '{ print $6 }'`

echo $Keepdate

/bin/ls -ult --full-time | grep -v "$Keepdate" | awk '{print $NF}' | xargs -i rm
-rf {}

It would be gr8 if some one could help me.

jim mcnamara · 03-01-2007, 03:09 PM

goto www.unix.com

Go to the FAQ.
Download the datecalc script.

Unix shell does NOT do date arithemetic very well.

Write a script:

Code:

1. find newest *.log file in the /path: youngfile= `ls -l -rt | tail -1`
2. use datecalc to take your filedate, subtract 60 days from it, and
return another date 60 days further in the past.
3. touch -t <older date> ./testfile
4. find /path ! -newer ./testfile -name '*.log' -exec rm -f {{} \;
Repeat steps 1 -4 for each directory

colucix · 03-01-2007, 07:42 PM

Here's my

Code:

ls -l --time-style=+%Y-%m-%d\ %T | gawk '{print $6,$7}'

to format timestamp in ls -l, and

Code:

date --date "2007-03-02 02:40:18 2 months ago"

to retrieve "2 months ago" from any date and time.

stress_junkie · 03-01-2007, 07:48 PM

Quote:

Originally Posted by sriramsreedhars

I hope you are getting it this time.

Are you kidding? That sounds a bit rude.

It is clear to me that you aren't "getting" the solution that I proposed and the reasons behind it. Your script is no better than my solution but it is a lot more complicated. Your script is still using the last modified date of the files to select the ones to delete.

ygloo · 03-01-2007, 09:42 PM

Code:

#!/bin/bash
latest_file=`/bin/ls -ult --full-time| sed 1d | head -1 | awk '{ print $9 }'`

echo `/bin/ls -ult --full-time $latest_file`

find . ! -newer $latest_file -print0 | xargs -0 ls -l 
(check results with "ls" before deleting!)

find . ! -newer $latest_file -print0 | xargs -0 rm

jim mcnamara · 03-02-2007, 10:47 AM

Or a perl version

Code:

#!/bin/ksh

cd /PATH/to/logfiles
ls -rt *.log | tail -1 | read newfile
touch_value=$(perl -e '
	$mtime=(stat $ARGV[0])[9];
	$mtime = $mtime - (86400*60);
	($sec,$min,$hour,$mday,$mon,$year,$wday,
					 $yday,$isdst)
					   = localtime($mtime);
	 printf "%d%02d%02d%02d%02d\n", $year+1900, $month+1, $day+1, $hour, $min;
			 ' $newfile )
touch -t $touch_value touchfile
find . ! -newer touchfile -exec rm -f {} \;

makyo · 03-03-2007, 09:30 AM

Hi.

A few thoughts on this.

I agree that trusting the times (ctime ,mtime, atime) can lead to incorrect (and worse, unwanted) removals, although it can be convenient to code. Solutions that use a time-stamp recorded as data in a log file are less risky, but they require more thought and work. One would not necessarily need to have only the most recent time, but one could keep track of any file and its creation date.

I think that sriramsreedhars was not being rude. I suspect that English is not his first language, and that he was using a phrase that was neutral to him.

My practice for associating creation dates with important files and directories is to make the creation date part of the name, e.g.

Code:

/backup/debian/makyo/2007-03-03

This makes date calculations with the arithmetic facilities of GNU/Linux command date very easy -- although -- as I customarily rant -- those capabilities are not noted in the man page for date, but rather as a few examples in:

Code:

info coreutils date

If portability is a concern, Dave Taylor posted a short c code that may serve: http://www.askdavetaylor.com/date_ma...ll_script.html , and once you test it, you can modify it to eliminate the explanatory string to get simply the date calculation (a small effort is required).

Best wishes ... cheers, makyo