LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 08-02-2009, 10:56 PM   #1
njozwiak
LQ Newbie
 
Registered: Jul 2008
Location: Manchester, NH
Distribution: Fedora Core
Posts: 22

Rep: Reputation: 1
System Logging - File Writing


Hey all,

I'm writing some system log software for Linux for a project and I am almost done but there is one piece that I am missing. The software needs to write the application messages to the log file and depending on user requirements (max log file size, number of messages, etc) remove the oldest message from the log. So I'm using a circular buffer to hold the messages. The question is when should I write to the log file? I cannot write every time data is modified in the buffer, that would be too taxing on the system. I am aware of the syslog software and was looking through the source to see when it writes to file, but was not able to find what I was looking for. Any ideas?

Thanks,
Nate
 
Old 08-02-2009, 11:20 PM   #2
Meson
Member
 
Registered: Oct 2007
Distribution: Arch x86_64
Posts: 606

Rep: Reputation: 66
I know this isn't the answer you are looking for, but why don't you just use syslog?

As far as taxing the system goes, what language are you writing this in? Writing isn't necessarily done synchronously - for this reason some languages have commands like flush() and close() which force writing to be done.

Last edited by Meson; 08-02-2009 at 11:22 PM.
 
Old 08-03-2009, 09:42 AM   #3
njozwiak
LQ Newbie
 
Registered: Jul 2008
Location: Manchester, NH
Distribution: Fedora Core
Posts: 22

Original Poster
Rep: Reputation: 1
Good question. The company I'm working for wants some features that syslog has. To include these features I would have needed to make a front end application to intercept messages. At that point I was basically writing the entire application anyhow, so we decided that it'd be best just to leave the busybox version currently running on the system there and utilize our own.
 
Old 08-03-2009, 11:09 AM   #4
paulsm4
Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
Hi -

Reinventing the wheel is seldom a good solution ... unless you have absolutely no alternative.

Please scan this article, and see if it might be applicable:

http://www.ducea.com/2006/06/06/rota...nux-log-files/

IMHO .. PSM
 
Old 08-03-2009, 11:24 AM   #5
njozwiak
LQ Newbie
 
Registered: Jul 2008
Location: Manchester, NH
Distribution: Fedora Core
Posts: 22

Original Poster
Rep: Reputation: 1
Thanks for the article Paul, while informative, it wasn't quite what I'm looking for. I'll try to be a little clearer. What I need is an algorithm for when to write to a file in a system log setting. When writing to disk is not possible every time data changes, what is the best approach?
 
Old 08-03-2009, 11:46 AM   #6
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,396
Blog Entries: 2

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
Before you make the assumption that 'I cannot write every time data is modified in the buffer, that would be too taxing on the system', why don't you simply do it that way and see if your assumption is correct. You may be surprised to find how little load this adds. The filesystem is generally able to use algorithms employing buffering that does a good job of optimizing disk access efficiency.
Another factor that may make a difference is the type of filesystem used. Most have some particular emphasis on performance, and using the one that matches your requirements may be all the fine-tuning that your purpose requires.
---- rod.
 
Old 08-03-2009, 04:33 PM   #7
njozwiak
LQ Newbie
 
Registered: Jul 2008
Location: Manchester, NH
Distribution: Fedora Core
Posts: 22

Original Poster
Rep: Reputation: 1
Quote:
Originally Posted by theNbomr View Post
Before you make the assumption that '...', why don't you simply do it that way and see if your assumption is correct.
The purpose of this system log is to monitor applications that are running on this board. We have a lot of them and they could all be spitting out a lot of messages at once. Writing every time a message is received is just not an option.

I guess what I'll do is create a series of flags (log file hasn't been written to in x minutes, buffer has x bytes of data more, etc) that will trigger a file write. It was my initial idea and seems very primitive, but I don't see any other solutions out there.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Linpus Fedora after saving files to file system, CAN'T open FILE SYSTEM from desktop! remiwa Fedora 2 01-07-2009 08:28 AM
Writing an ISO file for cd audio from a cue and wave file? spaz-atk Linux - Software 5 12-01-2008 02:52 AM
Writing a proxy file system driver in linux appas Linux - Software 0 07-22-2004 01:45 AM
writing a simple file system in linux Deepthi_57 Linux - Newbie 1 05-03-2004 01:42 AM


All times are GMT -5. The time now is 07:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration