logrotate files with time stamp

williebens · 10-10-2010, 12:56 AM

Hello All:

I would like to know how I should go about rotating files that end with a date stamp. This is the configuration I have to rotate my Apache access files, but it is not working:

/var/log/httpd/access_log.* {

compress

daily

rotate 1

copytruncate

missingok

notifempty

}

The files are created with a date extension like the ones below:

access_log.100810-00_00
access_log.092810-00_00

access_log.100510-00_00

I need to find to rotate them so that I get this:

access_log.00810-00_00.bz2
access_log.092810-00_00.bz2
access_log.100510-00_00.bz2

Thanks.
--Willie

frankbell · 10-10-2010, 09:10 PM

A couple of years ago, I wrote a script to rotate my log files daily. At the time I was self-hosting using the lampp server with all security enabled.

I don't have a copy of the script any more, but this is what I did:

1. Stopped my server services.

2. Declared a variable for date and time and created a directory to hold the backup named by date and time.

Code:

#Define now as today’s day-month-year-hour-minute.

NOW=$(date +%m-%d-%Y-%H-%M)

#Make the directory to hold the backup file.

mkdir /root/backups/log/$NOW

3. Gzipped the files into the directory /root/backups/log/$NOW. Had I needed to, I could have added an mv command to rename the files including the date and time, but having them separated in the named directories suited my needs.

4. Restarted my server services.

I know nothing about scripting. I did it all with the help of this book.

It took me about a week to get it right, but it then worked flawlessly as a cron job until I moved my website out to a hosting service.

Hope this helps.

williebens · 10-10-2010, 10:05 PM

Thanks for the response frankell.

This is what I have done so far. I have this script:

LOGDIR="/var/log/httpd"
BZIP=`whereis bzip2`
cd $LOGDIR
LISTACC=$(ls access*)
LISTERR=$(ls error*)

LISTACC1=$(ls access*.bz2)
LISTERR1=$(ls error*.bz2)

for files in $LISTACC $LISTERR
do
if [[ $files != *.bz2 ]] && [[ -f $files ]]
then
bzip2 $files
fi
done
for files1 in $LISTACC1 $LISTERR1
do
#if [[ $files1 == *.bz2 ]]
#then
#echo "files ones are: $files1"
$newfilenames=`echo $files1 | sed 's/bz2/1.bz2/g'`
#echo "the new names are $newfilenames"
#mv $files1 $newfilenames
#fi
done

However, for the second for loop, it is not working. These are my log files:

access_log.073010-00_00.bz2
access_log.082810-00_00.bz2
access_log.083010-00_00.bz2
access_log.bz2
error_log.042310-00_00.bz2
error_log.070910-00_00.bz2
error_log.080510-00_00.bz2
error_log.bz2

When I run the script, it gives me and error as follows:

[root@scripts willie]# bash newnanme.sh
access_log.073010-00_00.1.bz2
access_log.073010-00_00.2.bz2
access_log.073010-00_00.3.bz2
newnanme.sh: line 38: =access_log.073010-00_00.1.bz2: command not found
newnanme.sh: line 38: =access_log.082810-00_00.1.bz2: command not found
newnanme.sh: line 38: =access_log.083010-00_00.1.bz2: command not found
newnanme.sh: line 38: =access_log.1.bz2: command not found
newnanme.sh: line 38: =error_log.042310-00_00.1.bz2: command not found
newnanme.sh: line 38: =error_log.070910-00_00.1.bz2: command not found
newnanme.sh: line 38: =error_log.080510-00_00.1.bz2: command not found
newnanme.sh: line 38: =error_log.1.bz2: command not found
[root@scripts willie]#

I want the logs to be renamed to

access_log.073010-00_00.1.bz2
access_log.082810-00_00.1.bz2
access_log.083010-00_00.1.bz2
access_log.1.bz2
error_log.070910-00_00.1.bz2
error_log.080510-00_00.1.bz2

and so on.

Please someone guide me to fix this error.
Thanks.

frankbell · 10-11-2010, 07:38 PM

You are way ahead of me as regards scripting, but puzzling over this might help me learn stuff, so I'll try asking a few questions.

I can't figure out from your post exactly which line is line 38, but that seems to be where the script chokes.

When you look at the results of the script, the actual files in the directory, does the script choke before or after the bzip operation?

Another way of putting this is, is it choking on the bzip or on the rename?

sleddog · 10-11-2010, 08:55 PM

Why not focus on getting logrotate working instead of trying to re-invent it?

Use logrotate's debug/verbose mode to see what's happening:

logrotate -dv /path/to/configfile

Could be that logrotate doesn't think your logs are due for rotation. You can force it with it:

logrotate -f /path/to/configfile

williebens · 10-13-2010, 02:21 PM

Hello frankbell and sleddog:

Frankbell, this part is working properly as far as I know, but now that you are asking the question, I will double check to make sure. I don't do it at this moment because I am kind of in a hurry.

Works (but I will double check):
for files in $LISTACC $LISTERR
do
if [[ $files != *.bz2 ]] && [[ -f $files ]]
then
bzip2 $files
fi
done

This is the part that I think is not working:

$newfilenames=`echo $files1 | sed 's/bz2/1.bz2/g'`

It fails when it tries to rename the file. The funny story is that it works if I run it manually on the command line.

sleddog:

The situation is that the person who wants these logs rotated is using the apache rotatelogs utility. This person is not using /etc/logroate.d. Also, I tested logrotate -f, and tried it, but it did not work. However, I will try it again. I will try ogrotate -dv /path/to/configfile to see what it says.

Thanks
--Willie

frankbell · 10-13-2010, 09:09 PM

Quote:

Originally Posted by williebens

It fails when it tries to rename the file. The funny story is that it works if I run it manually on the command line.

This is good news and here's why: You know the issue is something to do with the "mv" command and the associated elements of the script. 95% of trouble-shooting is finding the trouble.

I can only tell you what I would do:

Set up a test directory.

Run the script. See what fails (not just in the error messages, but also in the results on the hard drive).

Make ONE change.

Try again.

Repeat, one chance at a time.

By the way, "make one change" is why I declared the my $NOW variable to the minute. I could make one change, wait a minute, try again, and retest.

One change at a time.

If you make two changes, you will not know which one worked.

I wish I knew enough about scripting to just look at the script and say, "do this," but I don't. But I do know trouble-shooting.