how to backup & truncate the log file while the process is running

mvenkat_in · 10-14-2007, 03:25 AM

Hi all,
I am in urgent need of a solution for my below problem. My problem definition is as follows:

I have an application (or process) which runs forever and writes the log the output into a log file say X.log. This log file size is becoming huge. So I want to take a backup of this X.log file into another file say Xbackup.log and truncate the X.log while the process is running and without hampering the process. The X.log file can't be moved or deleted because the process always keep on logging into this file.So that I can take the backup of the original file into some other place and reduce the original file size by truncating it.

Can anyone please suggest me or give me the script to my above problem.

Thanks in Advance,
Venkatesh M

jschiwal · 10-14-2007, 05:16 AM

This is on my system using "logrotate". You system may even be using it.

unSpawn · 10-14-2007, 05:34 AM

If the process keeps the log open you may not be able to truncate it in place?

mvenkat_in · 10-14-2007, 07:49 AM

Yeah,You are right.The process keeps the log file open writes into it most of the time.Because the process runs 24*7.
So how can I solve this problem.
Will logrotate or logadm (in solaris) solve this problem?

unSpawn · 10-14-2007, 11:43 AM

A logrotate post-script still would need to -HUP the process to make it see the changed descriptor stuff.
You said "writes into it most of the time". Could you be more specific why the app can't be restarted? Would be easiest.

Alien_Hominid · 10-14-2007, 12:57 PM

It's not the best sollution (probably) but you could use some db, which allows multiple access at the same time.

mvenkat_in · 10-14-2007, 10:06 PM

Hi
The process can not be restarted or disturbed because its a very crucial application running in a production servers.So I should rotate the logs without disturbing the process.

Can I try like this?
1) Take a backup of original file ( cp X.log /backup/X.log)
2) Truncate the original file ( truncate X.log)

Will this work if the process is busy with writing into X.log?

chrism01 · 10-14-2007, 11:39 PM

Use the logrotate util as mentioned: it'll do all that stuff for you, if properly invoked.

ta0kira · 10-15-2007, 07:46 AM

I would open a new file descriptor with a second file then dup2 to the log descriptor:

Code:

int log_file = open("x.log", O_RDWR | O_CREAT | O_TRUNC, 0600);

/*... write to file ...*/

/*other threads may continue to write to log_file during this process*/
int new_log_file = open("x2.log", O_RDWR | O_CREAT | O_TRUNC, 0600);
if (dup2(new_log_file, log_file) < 0) /* error! */;
if (new_log_file != log_file) close(new_log_file);

rename("x.log", "x.log~");
rename("x2.log", "x.log");

ta0kira

mvenkat_in · 10-19-2007, 12:01 AM

Hi
I can not do this in the application. I need to write an external script to backup the existing file and truncate it to 0 bytes. I have tried small script like below
cp X.log /backup/X.log1
> X.log

but the second truncate is not working properly. The file is getting truncated to 0 bytes but as soon as the application writes into the file, the file is coming again the previous size.

Can any one suggest..

ta0kira · 10-19-2007, 07:08 AM

The application needs to write to the file in append mode, otherwise it won't work. The application never closes the file, so its write index doesn't reset to 0 when the file is truncated. That means if the program last wrote at 1024 it will write at 1025 the next time, restoring the file size.

If the program opens the file with O_APPEND it should always write to the end regardless of other programs changing the file. That's how multiple processes can all write to the same file without tripping over each other.
ta0kira

mvenkat_in · 10-20-2007, 04:07 AM

Hi,
Thanks a lot for your correct information.
I will try to solve this problem in any another way.

Thanks a lot

mvenkat_in · 10-23-2007, 11:55 PM

Hi,
I could resolve this problem.
Actually the application was opening the log in WRITE mode. The Append mode was disabled in the configuration file (log4j.xml for Oracle App Server).
I enabled the APPEND mode and its working fine now.

Thanks all for your valuable suggestions.

p_michalczyk · 10-30-2007, 07:03 AM

You can do something like this with copytruncate option in logrotate. See man logrotate for details.