LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 04-01-2010, 03:21 PM   #31
rigor
Member
 
Registered: Sep 2003
Location: 19th moon ................. ................Planet Covid ................Another Galaxy;............. ................Not Yours
Posts: 705

Rep: Reputation: Disabled

Quote:
Originally Posted by dnaqvi View Post
I it ok?


0 * * * * /home/dn/error_message.

will it generate every hour?
Yes, if there's no period at the end of the crontab line, that will run the error_message program every hour.

But, it will count ALL the messages in the file. Not just those from a single hour. Is that what you want?
 
Old 04-05-2010, 11:22 AM   #32
dnaqvi
Member
 
Registered: Oct 2009
Posts: 117

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by kakaka View Post
Yes, if there's no period at the end of the crontab line, that will run the error_message program every hour.

But, it will count ALL the messages in the file. Not just those from a single hour. Is that what you want?
Yes or no.
Thing has change on demand.

What about every hour?

But if restart server before cutoff time,
the error counts will start from zero but we want total errors every hour.

for eg:

1hr -- 20 counts
2hr -- 45 counts (25 from current hours and 20 from previos hr)
3hr -- 45 counts (0 from current hours and 45 from previous hrs)

Thanks
 
Old 04-06-2010, 12:14 AM   #33
rigor
Member
 
Registered: Sep 2003
Location: 19th moon ................. ................Planet Covid ................Another Galaxy;............. ................Not Yours
Posts: 705

Rep: Reputation: Disabled
Hi dnaqvi,

If understand correctly you've stated that you start a new log file every 12 hours.

The pattern we've been talking about will count the overall total of whatever type of messages you specify, in a single log file, each time you use the pattern, without any concern for the hour in the message .

Let's say that you are able to use the pattern in a script, and manage to run the script, exactly at the time, after the last message for one hour is put in the log file, and before the first message for the next hour is put in the log file. Then, for a particular type of message, with this list of counts for the first few hours of the day as an example, the counts would work like this:

Code:
hour    count which occurred *that* hour    pattern would return this count
====    ==================================  ===============================
   0                                    10                               10
   1                                    15                               25
   2                                    25                               50
   3                                    10                               60

It will only do that for a single log file. As soon as you start a new log file, the count returned by a pattern used for a particular type of message, will start over at zero.

So, if you want a continuously increasing count, then you'll need to save the count you got from one log file, and add it to the count from the next log file.

If for some reason, it's easier for you to add up the total, by getting the count for each specific hour according to the time in the message for a particular type of message, that can be done using commands like these:

Code:
$ the_hour=`date +%H`
$ echo $the_hour
23
$ egrep '^(([^\ ]+)([\ ]+))'${the_hour}'(((:[0-9]+)){3})((([\ ]+)([^\ ]+)){3})([\ ]+)(A)' sys.log
[3/29/10 23:01:46:113 PDT] 00000093 LdapRegistryI A SECJ0419I: The user registry is currently connected to the LDAP server ldap://00.00.00.00:123.
Again, I'm using the log messages you've provided, but I changed the hour in some of them so that there are some messages for each hour of some day. When I started typing this it was between the 23rd hour ( 11 PM ) and the end of the day, in the local time zone. So the special format on the date command, just outputs the hour of the day from 0 through 23, in this case 23, which is then assigned to variable the_hour. If you put the command in the script, it could used to select the same hour for each pattern for each type of message.

If you were to use that with what we'd talked about before, and just using only error and advisory messages as an example, with the pattern for a specific hour according to the time in the message, it might look something like:

Code:
the_hour=`date +%H`

error_count=`egrep '^(([^\ ]+)([\ ]+))'${the_hour}'(((:[0-9]+)){3})((([\ ]+)([^\ ]+)){3})([\ ]+)(E)' sys.log | sort -k6 -u | wc -l`

advisory_count=`egrep '^(([^\ ]+)([\ ]+))'${the_hour}'(((:[0-9]+)){3})((([\ ]+)([^\ ]+)){3})([\ ]+)(A)' sys.log | sort -k6 -u | wc -l`

total_count=`expr $advisory_count  \+  $error_count`

echo "There were $error_count error messages, $advisory_count advisory messages, $total_count together."
But in case you did need to do something like that, please keep in mind that by using the date command, the patterns would look for messages from whatever hour it is when the script runs, even if it's run near the end of an hour, something delays it's running, and it actually runs just after the start of a new hour.

You could also pass in the hour for which you want to search to be absolutely sure you get the right hour.

Whichever way you need to do things, hope this helps.
 
Old 04-07-2010, 08:47 AM   #34
dnaqvi
Member
 
Registered: Oct 2009
Posts: 117

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by kakaka View Post
Hi dnaqvi,

If understand correctly you've stated that you start a new log file every 12 hours.

The pattern we've been talking about will count the overall total of whatever type of messages you specify, in a single log file, each time you use the pattern, without any concern for the hour in the message .

Let's say that you are able to use the pattern in a script, and manage to run the script, exactly at the time, after the last message for one hour is put in the log file, and before the first message for the next hour is put in the log file. Then, for a particular type of message, with this list of counts for the first few hours of the day as an example, the counts would work like this:

Code:
hour    count which occurred *that* hour    pattern would return this count
====    ==================================  ===============================
   0                                    10                               10
   1                                    15                               25
   2                                    25                               50
   3                                    10                               60

It will only do that for a single log file. As soon as you start a new log file, the count returned by a pattern used for a particular type of message, will start over at zero.

So, if you want a continuously increasing count, then you'll need to save the count you got from one log file, and add it to the count from the next log file.

If for some reason, it's easier for you to add up the total, by getting the count for each specific hour according to the time in the message for a particular type of message, that can be done using commands like these:

Code:
$ the_hour=`date +%H`
$ echo $the_hour
23
$ egrep '^(([^\ ]+)([\ ]+))'${the_hour}'(((:[0-9]+)){3})((([\ ]+)([^\ ]+)){3})([\ ]+)(A)' sys.log
[3/29/10 23:01:46:113 PDT] 00000093 LdapRegistryI A SECJ0419I: The user registry is currently connected to the LDAP server ldap://00.00.00.00:123.
Again, I'm using the log messages you've provided, but I changed the hour in some of them so that there are some messages for each hour of some day. When I started typing this it was between the 23rd hour ( 11 PM ) and the end of the day, in the local time zone. So the special format on the date command, just outputs the hour of the day from 0 through 23, in this case 23, which is then assigned to variable the_hour. If you put the command in the script, it could used to select the same hour for each pattern for each type of message.

If you were to use that with what we'd talked about before, and just using only error and advisory messages as an example, with the pattern for a specific hour according to the time in the message, it might look something like:

Code:
the_hour=`date +%H`

error_count=`egrep '^(([^\ ]+)([\ ]+))'${the_hour}'(((:[0-9]+)){3})((([\ ]+)([^\ ]+)){3})([\ ]+)(E)' sys.log | sort -k6 -u | wc -l`

advisory_count=`egrep '^(([^\ ]+)([\ ]+))'${the_hour}'(((:[0-9]+)){3})((([\ ]+)([^\ ]+)){3})([\ ]+)(A)' sys.log | sort -k6 -u | wc -l`

total_count=`expr $advisory_count  \+  $error_count`

echo "There were $error_count error messages, $advisory_count advisory messages, $total_count together."
But in case you did need to do something like that, please keep in mind that by using the date command, the patterns would look for messages from whatever hour it is when the script runs, even if it's run near the end of an hour, something delays it's running, and it actually runs just after the start of a new hour.

You could also pass in the hour for which you want to search to be absolutely sure you get the right hour.

Whichever way you need to do things, hope this helps.
Later error in the log could be different than the past error message in the log.
"00000093 LdapRegistryI A SECJ0419I: The user registry is currently connected to the LDAP server"
 
Old 04-07-2010, 10:40 PM   #35
rigor
Member
 
Registered: Sep 2003
Location: 19th moon ................. ................Planet Covid ................Another Galaxy;............. ................Not Yours
Posts: 705

Rep: Reputation: Disabled
Are you saying the message has no time stamp? It looks like this:

Code:
00000093 LdapRegistryI A SECJ0419I: The user registry is currently connected to the LDAP server
rather than like this:

Code:
[3/29/10 0:01:45:464 PDT] 00000093 LdapRegistryI A SECJ0419I: The user registry is currently connected to the LDAP server ldap://00.00.00.00:123.
 
Old 04-15-2010, 11:08 AM   #36
dnaqvi
Member
 
Registered: Oct 2009
Posts: 117

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by kakaka View Post
Are you saying the message has no time stamp? It looks like this:

Code:
00000093 LdapRegistryI A SECJ0419I: The user registry is currently connected to the LDAP server
rather than like this:

Code:
[3/29/10 0:01:45:464 PDT] 00000093 LdapRegistryI A SECJ0419I: The user registry is currently connected to the LDAP server ldap://00.00.00.00:123.
yes
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Trying to understand pipes - Can't pipe output from tail -f to grep then grep again lostjohnny Linux - Newbie 15 03-12-2009 10:31 PM
how to grep multiple filters with grep LinuxLover Linux - Enterprise 1 10-18-2007 07:12 AM
grep output on stdout and grep output to file don't match xnomad Linux - General 3 01-13-2007 04:56 AM
bash script with grep and sed: sed getting filenames from grep odysseus.lost Programming 1 07-17-2006 11:36 AM
ps -ef|grep -v root|grep apache<<result maelstrombob Linux - Newbie 1 09-24-2003 11:38 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 08:30 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration