LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-30-2011, 02:16 PM   #1
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Rep: Reputation: Disabled
help with a bash script


Hi,
ok so this might not be a newbie questions. I need help in creating a bash script that can do some addition of some numbers for me based on a certain criteria. This will be hard to explain

e.g. My data looks like(from a log)

Jun 14 03:15:06 node-1 speedflow: [24003][kpi.cpp:57][INFO] KPI [netCap.speed.real = 1011 mbs]
Jun 14 03:15:06 node-1 speedflow: [24004][kpi.cpp:57][INFO] KPI [netCap.speed.real = 1081 mbs]
Jun 14 03:30:06 node-1 speedflow: [24003][kpi.cpp:57][INFO] KPI [netCap.speed.real = 1027 mbs]
Jun 14 03:30:06 node-1 speedflow: [24004][kpi.cpp:57][INFO] KPI [netCap.speed.real = 1102 mbs]
Jun 14 03:45:06 node-1 speedflow: [24004][kpi.cpp:57][INFO] KPI [netCap.speed.real = 1039 mbs]
Jun 14 03:45:06 node-1 speedflow: [24003][kpi.cpp:57][INFO] KPI [netCap.speed.real = 960 mbs]
Jun 14 04:00:06 node-1 speedflow: [24003][kpi.cpp:57][INFO] KPI [netCap.speed.real = 864 mbs]
Jun 14 04:00:06 node-1 speedflow: [24004][kpi.cpp:57][INFO] KPI [netCap.speed.real = 964 mbs]
Jun 15 03:15:06 node-1 speedflow: [24003][kpi.cpp:57][INFO] KPI [netCap.speed.real = 2011 mbs]
Jun 15 03:15:06 node-1 speedflow: [24004][kpi.cpp:57][INFO] KPI [netCap.speed.real = 2081 mbs]
Jun 15 03:30:06 node-1 speedflow: [24003][kpi.cpp:57][INFO] KPI [netCap.speed.real = 2027 mbs]
Jun 15 03:30:06 node-1 speedflow: [24004][kpi.cpp:57][INFO] KPI [netCap.speed.real = 2102 mbs]

Here the sixth column has two repeating process ID's(24004 and 24003). Based on this, i want to get the sum of xxx mbs
e.g
for jun 14, 3:15:06 i would like to add 1001 and 1081 because for them the process ID's are different. The goal is not to add the values where the process ID is the same. in this case i can see that after every 15min the two process ID's repeat. That what i want to add!
Then for the next one, Jun 14 03:30:06 i would like to get the sum of 1027 and 1102
.
.
.
.
for Jun 15 03:30:06 get the sum of 2027 and 2102

Thanks. I will really appreciate help with this.

Last edited by alirz; 06-30-2011 at 02:26 PM.
 
Old 06-30-2011, 02:23 PM   #2
Zmyrgel
Senior Member
 
Registered: Dec 2005
Location: Finland
Distribution: Slackware, CentOS, RHEL, OpenBSD
Posts: 1,006

Rep: Reputation: 37
I tried to read your description but couldn't figure out what your trying to do here.

You need to add the digits in "speed = xxx mbs" which are reported at certain time together?
 
Old 06-30-2011, 02:27 PM   #3
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Zmyrgel View Post
I tried to read your description but couldn't figure out what your trying to do here.

You need to add the digits in "speed = xxx mbs" which are reported at certain time together?

Thanks for your answer, i've re edited my questions so it makes more sense hopefully
 
Old 06-30-2011, 02:52 PM   #4
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,066
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Code:
awk -F"[][ ]" '{pid[$7]+=$17}END{for(i in pid){print i" "pid[i]}}' alirz.log
24003 7900
24004 8369


Cheers,
Tink
 
Old 06-30-2011, 03:07 PM   #5
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Tinkster View Post
Code:
awk -F"[][ ]" '{pid[$7]+=$17}END{for(i in pid){print i" "pid[i]}}' alirz.log
24003 7900
24004 8369


Cheers,
Tink
Thanks Tink, Although your script is adding the numbers for their respective ID's, it seems to be adding them all up. I'm trying to only get the sum for each time period. i.e the sum at Jun 14 03:15:06 and then for the next occurence at Jun 14 03:30:06 and so on for each day in the log.

The idea is to get the sum of the speed at each 15min interval for both the(24004 and 24003) processes and then at the end of the day, see what and when was the max speed total.

the output of the bash script could look something like

Jun 14 03:15:06 2092
Jun 14 03:30:06 2129
Jun 14 03:45:06 1999
Jun 14 04:00:06 1828

Last edited by alirz; 06-30-2011 at 03:23 PM.
 
Old 06-30-2011, 03:22 PM   #6
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,066
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Ooops. Epic fail on my side. Only read half of what you were asking.
Let me think ...


Cheers,
Tink
 
Old 06-30-2011, 03:24 PM   #7
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Tinkster View Post
Ooops. Epic fail on my side. Only read half of what you were asking.
Let me think ...


Cheers,
Tink
No problem at all buddy...You are definitely wayyyy better than me at Linux, hence my need to ask for help on a forum for this
i further explained above what im trying to get as an output.
 
Old 06-30-2011, 03:31 PM   #8
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,066
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Code:
awk -F"[][ ]" '{if(NR%2!=0){hold=$17}else{print $1" "$2" "$3" "hold+$17}}' alirz.log
Jun 14 03:15:06 2092
Jun 14 03:30:06 2129
Jun 14 03:45:06 1999
Jun 14 04:00:06 1828
Jun 15 03:15:06 4092
Jun 15 03:30:06 4129
Better?


Cheers,
Tink
 
Old 06-30-2011, 03:37 PM   #9
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Tinkster View Post
Code:
awk -F"[][ ]" '{if(NR%2!=0){hold=$17}else{print $1" "$2" "$3" "hold+$17}}' alirz.log
Jun 14 03:15:06 2092
Jun 14 03:30:06 2129
Jun 14 03:45:06 1999
Jun 14 04:00:06 1828
Jun 15 03:15:06 4092
Jun 15 03:30:06 4129
Better?


Cheers,
Tink
Freaking genius, that works perfectly. Thanks alot. If you dont mind would you care to explain very briefly what "{if(NR%2!=0){hold=$17}" is doing for my own learning
 
Old 06-30-2011, 03:41 PM   #10
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,066
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
It does a modulo on the record numbers ... which reminds me:
if any of your logs don't have two matching time-stamps at
the beginning this will NOT do what you want. You could re-
write it to hold & compare time-stamps ;}

On the first line we hold the value, on the second we add
and print.


Cheers,
Tink
 
Old 07-01-2011, 02:33 AM   #11
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,255

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
Just for a slightly different take on the same process:
Code:
awk '{arr[$1" "$2" "$3]+=$(NF-1)}END{n=asorti(arr,arr2);while(++i <= n)print arr2[i],arr[arr2[i]]}' alirz.log
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Variables and Mkvextract in a bash script and a good resource for bash help? gohmifune Linux - General 9 04-13-2011 09:37 AM
[SOLVED] Setting function default in a bash script - Bash sixtensixone Linux - General 13 01-08-2011 02:44 PM
SSH connection from BASH script stops further BASH script commands tardis1 Linux - Newbie 3 12-06-2010 09:56 AM
[SOLVED] Using a long Bash command including single quotes and pipes in a Bash script antcore Linux - General 9 07-22-2009 12:10 PM
Strange if statement behaviour when using bash/bash script freeindy Programming 7 08-04-2008 07:00 AM


All times are GMT -5. The time now is 02:27 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration