LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 10-14-2007, 03:25 AM   #1
mvenkat_in
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Rep: Reputation: 0
how to backup & truncate the log file while the process is running


Hi all,
I am in urgent need of a solution for my below problem. My problem definition is as follows:

I have an application (or process) which runs forever and writes the log the output into a log file say X.log. This log file size is becoming huge. So I want to take a backup of this X.log file into another file say Xbackup.log and truncate the X.log while the process is running and without hampering the process. The X.log file can't be moved or deleted because the process always keep on logging into this file.So that I can take the backup of the original file into some other place and reduce the original file size by truncating it.

Can anyone please suggest me or give me the script to my above problem.

Thanks in Advance,
Venkatesh M
 
Old 10-14-2007, 05:16 AM   #2
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
This is on my system using "logrotate". You system may even be using it.
 
Old 10-14-2007, 05:34 AM   #3
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,264
Blog Entries: 54

Rep: Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841
If the process keeps the log open you may not be able to truncate it in place?
 
Old 10-14-2007, 07:49 AM   #4
mvenkat_in
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Yeah,You are right.The process keeps the log file open writes into it most of the time.Because the process runs 24*7.
So how can I solve this problem.
Will logrotate or logadm (in solaris) solve this problem?
 
Old 10-14-2007, 11:43 AM   #5
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,264
Blog Entries: 54

Rep: Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841Reputation: 2841
A logrotate post-script still would need to -HUP the process to make it see the changed descriptor stuff.
You said "writes into it most of the time". Could you be more specific why the app can't be restarted? Would be easiest.
 
Old 10-14-2007, 12:57 PM   #6
Alien_Hominid
Senior Member
 
Registered: Oct 2005
Location: Lithuania
Distribution: Hybrid
Posts: 2,247

Rep: Reputation: 53
It's not the best sollution (probably) but you could use some db, which allows multiple access at the same time.
 
Old 10-14-2007, 10:06 PM   #7
mvenkat_in
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Hi
The process can not be restarted or disturbed because its a very crucial application running in a production servers.So I should rotate the logs without disturbing the process.

Can I try like this?
1) Take a backup of original file ( cp X.log /backup/X.log)
2) Truncate the original file ( truncate X.log)

Will this work if the process is busy with writing into X.log?
 
Old 10-14-2007, 11:39 PM   #8
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,265

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
Use the logrotate util as mentioned: it'll do all that stuff for you, if properly invoked.
 
Old 10-15-2007, 07:46 AM   #9
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
I would open a new file descriptor with a second file then dup2 to the log descriptor:
Code:
int log_file = open("x.log", O_RDWR | O_CREAT | O_TRUNC, 0600);

/*... write to file ...*/

/*other threads may continue to write to log_file during this process*/
int new_log_file = open("x2.log", O_RDWR | O_CREAT | O_TRUNC, 0600);
if (dup2(new_log_file, log_file) < 0) /* error! */;
if (new_log_file != log_file) close(new_log_file);

rename("x.log", "x.log~");
rename("x2.log", "x.log");
ta0kira
 
Old 10-19-2007, 12:01 AM   #10
mvenkat_in
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Hi
I can not do this in the application. I need to write an external script to backup the existing file and truncate it to 0 bytes. I have tried small script like below
cp X.log /backup/X.log1
> X.log

but the second truncate is not working properly. The file is getting truncated to 0 bytes but as soon as the application writes into the file, the file is coming again the previous size.

Can any one suggest..
 
Old 10-19-2007, 07:08 AM   #11
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
The application needs to write to the file in append mode, otherwise it won't work. The application never closes the file, so its write index doesn't reset to 0 when the file is truncated. That means if the program last wrote at 1024 it will write at 1025 the next time, restoring the file size.

If the program opens the file with O_APPEND it should always write to the end regardless of other programs changing the file. That's how multiple processes can all write to the same file without tripping over each other.
ta0kira
 
Old 10-20-2007, 04:07 AM   #12
mvenkat_in
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Hi,
Thanks a lot for your correct information.
I will try to solve this problem in any another way.

Thanks a lot
 
Old 10-23-2007, 11:55 PM   #13
mvenkat_in
LQ Newbie
 
Registered: Feb 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Hi,
I could resolve this problem.
Actually the application was opening the log in WRITE mode. The Append mode was disabled in the configuration file (log4j.xml for Oracle App Server).
I enabled the APPEND mode and its working fine now.

Thanks all for your valuable suggestions.
 
Old 10-30-2007, 07:03 AM   #14
p_michalczyk
LQ Newbie
 
Registered: Oct 2007
Posts: 1

Rep: Reputation: 0
You can do something like this with copytruncate option in logrotate. See man logrotate for details.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Does K3B automatically truncate file names? General Linux - Software 2 05-08-2007 05:37 PM
Truncate A File Dovetails Linux - Newbie 2 10-22-2005 06:58 PM
truncate a text file in CLI? hottdogg Linux - General 2 09-06-2005 09:32 PM
Zipslack 9.1 long file names truncate jimk Slackware - Installation 5 05-16-2004 06:38 AM
system backup process 4 win2k & linux hank43 Linux - General 5 05-11-2004 09:56 PM


All times are GMT -5. The time now is 08:51 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration