LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-01-2007, 06:03 AM   #1
sriramsreedhars
LQ Newbie
 
Registered: Mar 2007
Posts: 11

Rep: Reputation: 0
Cool Deleting files after x days from the latest date file


I need to keep files for X days from latest date (not just one day)on a linux system then delete rest:

More Details :

Can someone suggest on this script :

let's say the logs files are available for Jan 07, Dec 06, Nov 06, Oct 06

the script should identify the latest months logs, i.e Jan 07

it should then delete anything older than 2 months, which will be logs for Nov 06 & Oct 06.
 
Old 03-01-2007, 08:09 AM   #2
stress_junkie
Senior Member
 
Registered: Dec 2005
Location: Massachusetts, USA
Distribution: Ubuntu 10.04 and CentOS 5.5
Posts: 3,873

Rep: Reputation: 331Reputation: 331Reputation: 331Reputation: 331
The problem that you face here is that Linux and Unix do not record the creation date of a file. The date that the file was last accessed and the date that the file was last modified are saved. You can either depend on one of those settings with the chance that old files will not be removed because they were last modified or accessed some time since they were created or you can record the creation date inside the file. I would choose to use the last modified date as the criteria. In that case you can use the find command to find all of the files in a directory tree that were modified more than 60 days ago, for example. The syntax of the find command is:
Code:
find <search directory> <criteria> [ -exec command [ parameters ] {} \;
So if you want to search the /var/log/mylogs directory tree for all files that were modified more than 60 days ago and delete these files the command would look like this.
Code:
find /var/log/mylogs -type f -ctime +1440 -exec rm {} \;
Here the ctime is measured in hours so 60 days times 24 hours is 1440. The plus sign in front of the 1440 means "more than". So this command will find files that were last modified/changed more than 60 days ago and it will remove them.

More information about the find command is available in the man pages.
Code:
man find

Last edited by stress_junkie; 03-01-2007 at 08:56 PM.
 
Old 03-01-2007, 10:03 AM   #3
sriramsreedhars
LQ Newbie
 
Registered: Mar 2007
Posts: 11

Original Poster
Rep: Reputation: 0
No thats not what I want

1) Find the latest date file
Say 2003-march-21 is the last file available in that directory,
then it should look for files older than that for 2 months and delete it.

Which means it should delete 2003-jan-21 and above files

I hope you are getting it this time.

Heres a script I have worked out on which finds the latest date and deletes restall file, but I want it to delete x no of days from the latest date file :


bash-3.00$ cat script.sh
#!/bin/bash
Keepdate=`/bin/ls -ult --full-time| sed 1d | head -1 | awk '{ print $6 }'`

echo $Keepdate

/bin/ls -ult --full-time | grep -v "$Keepdate" | awk '{print $NF}' | xargs -i rm
-rf {}


It would be gr8 if some one could help me.
 
Old 03-01-2007, 04:09 PM   #4
jim mcnamara
Member
 
Registered: May 2002
Posts: 964

Rep: Reputation: 34
goto www.unix.com

Go to the FAQ.
Download the datecalc script.

Unix shell does NOT do date arithemetic very well.

Write a script:
Code:
1. find newest *.log file in the /path: youngfile= `ls -l -rt | tail -1`
2. use datecalc to take your filedate, subtract 60 days from it, and
return another date 60 days further in the past.
3. touch -t <older date> ./testfile
4. find /path ! -newer ./testfile -name '*.log' -exec rm -f {{} \;
Repeat steps 1 -4 for each directory
 
Old 03-01-2007, 08:42 PM   #5
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Here's my
Code:
ls -l --time-style=+%Y-%m-%d\ %T | gawk '{print $6,$7}'
to format timestamp in ls -l, and
Code:
date --date "2007-03-02 02:40:18 2 months ago"
to retrieve "2 months ago" from any date and time.

Last edited by colucix; 03-01-2007 at 08:47 PM.
 
Old 03-01-2007, 08:48 PM   #6
stress_junkie
Senior Member
 
Registered: Dec 2005
Location: Massachusetts, USA
Distribution: Ubuntu 10.04 and CentOS 5.5
Posts: 3,873

Rep: Reputation: 331Reputation: 331Reputation: 331Reputation: 331
Quote:
Originally Posted by sriramsreedhars
I hope you are getting it this time.
Are you kidding? That sounds a bit rude.

It is clear to me that you aren't "getting" the solution that I proposed and the reasons behind it. Your script is no better than my solution but it is a lot more complicated. Your script is still using the last modified date of the files to select the ones to delete.

Last edited by stress_junkie; 03-01-2007 at 08:52 PM.
 
Old 03-01-2007, 10:42 PM   #7
ygloo
Member
 
Registered: Aug 2006
Distribution: slack
Posts: 323

Rep: Reputation: 30
Code:
#!/bin/bash
latest_file=`/bin/ls -ult --full-time| sed 1d | head -1 | awk '{ print $9 }'`

echo `/bin/ls -ult --full-time $latest_file`

find . ! -newer $latest_file -print0 | xargs -0 ls -l 
(check results with "ls" before deleting!)

find . ! -newer $latest_file -print0 | xargs -0 rm

Last edited by ygloo; 03-01-2007 at 10:44 PM.
 
Old 03-02-2007, 11:47 AM   #8
jim mcnamara
Member
 
Registered: May 2002
Posts: 964

Rep: Reputation: 34
Or a perl version
Code:
#!/bin/ksh

cd /PATH/to/logfiles
ls -rt *.log | tail -1 | read newfile
touch_value=$(perl -e '
	$mtime=(stat $ARGV[0])[9];
	$mtime = $mtime - (86400*60);
	($sec,$min,$hour,$mday,$mon,$year,$wday,
					 $yday,$isdst)
					   = localtime($mtime);
	 printf "%d%02d%02d%02d%02d\n", $year+1900, $month+1, $day+1, $hour, $min;
			 ' $newfile )
touch -t $touch_value touchfile
find . ! -newer touchfile -exec rm -f {} \;
 
Old 03-03-2007, 10:30 AM   #9
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 719

Rep: Reputation: 72
Hi.

A few thoughts on this.

I agree that trusting the times (ctime ,mtime, atime) can lead to incorrect (and worse, unwanted) removals, although it can be convenient to code. Solutions that use a time-stamp recorded as data in a log file are less risky, but they require more thought and work. One would not necessarily need to have only the most recent time, but one could keep track of any file and its creation date.

I think that sriramsreedhars was not being rude. I suspect that English is not his first language, and that he was using a phrase that was neutral to him.

My practice for associating creation dates with important files and directories is to make the creation date part of the name, e.g.
Code:
/backup/debian/makyo/2007-03-03
This makes date calculations with the arithmetic facilities of GNU/Linux command date very easy -- although -- as I customarily rant -- those capabilities are not noted in the man page for date, but rather as a few examples in:
Code:
info coreutils date
If portability is a concern, Dave Taylor posted a short c code that may serve: http://www.askdavetaylor.com/date_ma...ll_script.html , and once you test it, you can modify it to eliminate the explanatory string to get simply the date calculation (a small effort is required).

Best wishes ... cheers, makyo
 
  


Reply

Tags
delete


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to get logrotate to only rotate files 2 days old. yoda_mon Linux - Software 1 05-18-2006 09:55 PM
bash sort files by date in file name thedude2010 Programming 6 05-12-2006 12:07 PM
Changing File Permission in a FTP directory to prevent deleting of files shawnbishop Linux - Software 3 01-10-2006 06:41 AM
Deleting data from file by date? mephitic Linux - Newbie 3 09-22-2004 01:33 PM
Script for deleting files based on date MaverickApollo Linux - General 3 02-03-2004 08:54 PM


All times are GMT -5. The time now is 10:11 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration