LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-05-2011, 02:41 PM   #1
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Rep: Reputation: Disabled
how to add number for each day from log


looks like im keeping this forum busy these days: sorry

i have a log like:

Jul 4 23:19:02 Node-1 node-KPI: [INFO] KPI [proc.count = 11 files]
Jul 4 23:22:46 Node-1 node-KPI: [INFO] KPI [proc.count = 27 files]
Jul 4 23:37:49 Node-1 node-KPI: [INFO] KPI [proc.count = 27 files]
Jul 4 23:49:01 Node-1 node-KPI: [INFO] KPI [proc.count = 12 files]
Jul 4 23:50:50 Node-1 node-KPI: [INFO] KPI [proc.count = 19 files]
Jul 5 23:34:01 Node-1 node-KPI: [INFO] KPI [proc.count = 11 files]
Jul 5 23:37:10 Node-1 node-KPI: [INFO] KPI [proc.count = 12 files]
Jul 5 23:48:59 Node-1 node-KPI: [INFO] KPI [proc.count = 12 files]
Jul 5 23:50:48 Node-1 node-KPI: [INFO] KPI [proc.count = 19 files]
Jul 5 23:52:00 Node-1 node-KPI: [INFO] KPI [proc.count = 7 files]
Jul 6 00:03:59 Node-1 node-KPI: [INFO] KPI [proc.count = 12 files]
Jul 6 00:05:51 Node-1 node-KPI: [INFO] KPI [proc.count = 21 files]
Jul 6 00:06:55 Node-1 node-KPI: [INFO] KPI [proc.count = 5 files]



I would like to add up the numbers in the last column for each day and get that as the output. It could look like:

Jul 5 : 54
Jul 6 : 38


how can this be done with a bash script?
thanks

Last edited by alirz; 07-05-2011 at 03:36 PM.
 
Old 07-05-2011, 03:22 PM   #2
opnsrc
LQ Newbie
 
Registered: Dec 2005
Posts: 28

Rep: Reputation: 1
gawk is your friend here.

You can print out the columns you desire and even add up a total: gawk '{ sum += $10 }; END { print sum }' filename - You get that example if you just type gawk by itself.

With gawk, you can create some predefined variables and then use echo, you can get what you want. It's not terribly difficult and if you can put it together you'll far more than just someone creating it for you.
 
Old 07-05-2011, 03:35 PM   #3
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by opnsrc View Post
gawk is your friend here.

You can print out the columns you desire and even add up a total: gawk '{ sum += $10 }; END { print sum }' filename - You get that example if you just type gawk by itself.

With gawk, you can create some predefined variables and then use echo, you can get what you want. It's not terribly difficult and if you can put it together you'll far more than just someone creating it for you.
well so far i have this:

sum=0
for num in $(cat /var/log/messages |grep "node" |grep "proc.count" |awk -F " " '{print $10}')
do
((sum+=num))

done
echo $sum



the result of that script is that gives me just one big total of all values regardless of the date. If i wasnt a total noob i would figure it out but i dont know. i want to get total for each day separately.
 
Old 07-05-2011, 04:36 PM   #4
opnsrc
LQ Newbie
 
Registered: Dec 2005
Posts: 28

Rep: Reputation: 1
Well first you need some variables for the day(s). Since you're talking about the messages log file, you can use the current date as your starting point and go back:

#Date is output of short Month and day - 'Jul 5'
TODAY=$(date +%b%e)
YESTERDAY=$(date --date="yesterday" +"%b%e")
2_DAYS_AGO=$(date --date="2 days ago" +"%b%e")
#.
#.
#However far you want to go back before your messages file rotates.

#I would use your initial grep and place the output in a temporary file getting only the columns you really want:

cat /var/log/messages |grep "node" |grep "proc.count" |gawk '{print $1" "$2" \: "$10}' > /tmp/myout.log 2>&1 #the 2>&1 redirects STDERR


TODAY_TOTAL=$(cat /tmp/myout.log |grep $TODAY |gawk '{ sum += $4 }; END { print sum }')
YESTERDAY_TOTAL=$(cat /tmp/myout.log |grep $YESTERDAY |gawk '{ sum += $4 }; END { print sum }')
...and so on



echo "$TODAY TOTAL : $TODAY_TODAY" > logfile.log
echo "$YESTERDAY TOTAL : $YESTERDAY_TODAY" >> logfile.log

You can play with the output to meet your needs.


This is the beef of what you want, a total for each day. Let me know if I'm missing something, but I think you can figure it out, you're not a total noob, you've got the know how.
 
Old 07-05-2011, 07:14 PM   #5
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,417

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
See these HOWTOs http://tldp.org/LDP/Bash-Beginners-G...tml/index.html, http://www.tldp.org/LDP/abs/html/, http://rute.2038bug.com/index.html.gz
 
Old 07-05-2011, 09:42 PM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,565

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
How about we just stay with opnsrc initial advice:
Code:
awk '/[nN]ode/ && /proc[.]count/{sum[$1,$2]+=$(NF-1)}END{for(i in sum){split(i,a,SUBSEP);print a[1],a[2],sum[i]}' /var/log/messages
This is not tested but you should get the idea
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Process a log file on an hourly basis but the log only rotates once a day hattori.hanzo Programming 4 10-28-2010 09:55 AM
Postfix - Set max number of messages sent per day mangueJOE Linux - Server 1 10-22-2009 08:55 AM
/var/log/procmail.log.1 being mailed every day soylentgreen Linux - Server 3 07-10-2007 03:47 PM
Add Date (day) to Filename greengrass Programming 1 01-15-2007 07:48 PM
Hoto limit the number of minutes a user may be loged on in a day? Mandrake 9 tmillard Linux - Software 1 04-13-2004 08:04 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration