LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 11-22-2012, 05:42 AM   #1
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,797
Blog Entries: 4

Rep: Reputation: 285Reputation: 285Reputation: 285
Help understanding "awk" code


Hello, I have a script, containing following awk code:
Code:

Something...
Something....

grep " RESULT " ${IFILE} \
awk 'BEGIN{sum=0; cat1=0; cat2=0; cat3=0;}
{sum++}
/value=0\.[^0]/{cat2++;}                   ## 4th line
/value=0\.0/{cat3++;}                      ## 5th line
END{cat1=sum-(cat2+cat3); print cat1, cat2, cat3;}'

Something...
Something....
And this script returns output as:
Code:
458 0 0
I can understand this awk code, but I couldn't understand it's 4th and 5th lines. I am a beginnner in awk, so could you help me, how it's calculating values of cat1, cat2, and cat3. It may be little time taking for you, but am expecting a well explained answer. Thanks a bunch!

Note: ${IFILE} file mentioned in code, contains values like:
Code:
value=0
value=0.01
value=0.01
value=0 and so on...

Last edited by shivaa; 11-22-2012 at 05:47 AM.
 
Old 11-22-2012, 05:50 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
Code:
/value=0\.[^0]/{cat2++;}
increment cat2 if not value=0.0
[^0] -> all but a zero

Code:
/value=0\.0/{cat3++;}
increment cat3 if value=0.0

BTW: Please stop using large font, there's really no need for that.
 
1 members found this post helpful.
Old 11-22-2012, 05:52 AM   #3
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,373

Rep: Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962
I presume the confusion is from teh lack of formal structures around the confiditional operations there.

/value=0\.0/{cat3++;}

means something like

Code:
if {string matches regex "value=0\.0" ) 
{
  cat3 = cat3 + 1
}
in pseudo code. as they both end up as zero, those regexs never match
 
1 members found this post helpful.
Old 11-22-2012, 10:45 AM   #4
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,797
Blog Entries: 4

Original Poster
Rep: Reputation: 285Reputation: 285Reputation: 285
Code:
grep " RESULT " ${IFILE} \
awk 'BEGIN{sum=0; cat1=0; cat2=0; cat3=0;}
{sum++}
/value=0\.[^0]/{cat2++;}                   ## 4th line
/value=0\.0/{cat3++;}                      ## 5th line
END{cat1=sum-(cat2+cat3); print cat1, cat2, cat3;}'
It means, first it intialized all sum, cat1, cat2, cat3 to 0, then in 3rd line, it increases value of sum by one i.e sum=1. then what happens in next 4th line? Is it searching for values that are 0.0 and incrementing cat2 by 1? What's relation between pattern i.e. /value=0\.[^0]/ and action {cat2++;} in this line? Could you explain little more...
 
Old 11-22-2012, 10:52 AM   #5
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,373

Rep: Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962
I think my explanation covers that just fine.
 
1 members found this post helpful.
Old 11-22-2012, 11:42 AM   #6
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,797
Blog Entries: 4

Original Poster
Rep: Reputation: 285Reputation: 285Reputation: 285
Quote:
Originally Posted by acid_kewpie View Post
I think my explanation covers that just fine.
Apparently you're right & I understood what you explained, but my question is, what's it doing in 4th line in pattern portion? Is it searching for all values that contains or begin with 0.0 and if it finds any such matching value then adding 1 count to cat2? Am I correct?

BTW, problem is that I have got an old script created by someone who left the job, and I have been given a task to write a new script which should have same functionality like this script doing, and that is why I want to fully understands its code.

Last edited by shivaa; 11-22-2012 at 11:45 AM.
 
Old 11-22-2012, 12:53 PM   #7
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371Reputation: 2371
Would this help;
Code:
awk 'BEGIN{sum=0; cat1=0; cat2=0; cat3=0;}
{sum++}
/value=0\.[^0]/{cat2++;}
/value=0\.0/{cat3++;}
END{cat1=sum-(cat2+cat3); print cat1, cat2, cat3;}'
The blue part, setting some counters, is done once, when awk starts and before any input is parsed.

The green part is done for every line of input (lines that contain RESULT (with extra spaces)).
sum++ line -> counts all the lines it gets (total number of lines)
cat2++ line -> counts all the lines that contain value=0.<whatver> as long as it is _not_ value=0.0
cat3++ line ->counts all the lines that contain value=0.0

the brown part, some calculations and then printing the 3 entries, is done when all the lines are parsed.

Quote:
BTW, problem is that I have got an old script created by someone who left the job, and I have been given a task to write a new script which should have same functionality like this script doing, and that is why I want to fully understands its code.
The snippet of code posted in post #1 is in need of some re-writing.

- What you posted will not work (grep " RESULT " ${IFILE} \ should be grep " RESULT " ${IFILE} | \
- No need for the grep part, this can be done by awk
- the sum counter isn't needed in this case (the awk NR variable can be used)

That's without knowing what exactly needs to be done with the lines it gets.

Some resources that might help:

Bash resources:
Sed/Awk resources:
General resources:

Last edited by druuna; 11-22-2012 at 12:55 PM. Reason: added some resources
 
1 members found this post helpful.
Old 11-22-2012, 07:53 PM   #8
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,797
Blog Entries: 4

Original Poster
Rep: Reputation: 285Reputation: 285Reputation: 285
Special thanks to @druuna. You've explained it very well. One more last question:-
Can we devide this code in two parts (for understanding purpose) as follow:
Part-1:
Code:
grep " RESULT " ${IFILE} \
awk 'BEGIN{sum=0; cat1=0; cat2=0; cat3=0;}
{sum++}
/value=0\.[^0]/{cat2++;}                   ## 4th line
/value=0\.0/{cat3++;}                      ## 5th line
Part-2:
Code:
END{cat1=sum-(cat2+cat3); print cat1, cat2, cat3;}'
Then can we say?
(1) In first part, it's searching for specified patterns & then counting values. And after finishing, it simply prints those values in second part of the code?
(2) Is it looping between BEGIN and END of the code? Or calculating all specified patterns in one shot, e.g. in 4th line it searches for all values that are equal to 0.0<whatever>, and go to next line?
(3) After calculating values e.g. in 4th line, does it store that total connt in cat2? Or is it like a loop in which it search for a pattern, get pattern, and add 1 count?
 
Old 11-23-2012, 12:38 AM   #9
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,373

Rep: Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962Reputation: 1962
the BEGIN{} block runs before any of the input data is processed, the END{} block runs afterwards, everything else in the main {} block is executed on a per input line basis. so after each input line is read, the 4th line does get executed completely, but of course the value of cat2 persists over these executions, increasing each time it is matched.
 
1 members found this post helpful.
Old 11-23-2012, 04:43 AM   #10
linosaurusroot
Member
 
Registered: Oct 2012
Distribution: OpenSuSE,RHEL,Fedora,OpenBSD
Posts: 742
Blog Entries: 2

Rep: Reputation: 193Reputation: 193
Code:
BEGIN{sum=0; cat1=0; cat2=0; cat3=0;}
A BEGIN block like that is unnecessary as variables default to 0 and can be used without initialisation.
 
Old 12-01-2012, 05:25 AM   #11
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,797
Blog Entries: 4

Original Poster
Rep: Reputation: 285Reputation: 285Reputation: 285
Many thanks everyone, my purpose of asking this question has solved.
Special thanks to @druuna for always being helpful
 
Old 12-02-2012, 01:40 PM   #12
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Quote:
Originally Posted by linosaurusroot View Post
Code:
BEGIN{sum=0; cat1=0; cat2=0; cat3=0;}
A BEGIN block like that is unnecessary as variables default to 0 and can be used without initialisation.
While this is mostly correct, in truth uninitialized variables are only treated as having a value of zero in math operations. They are still actually null in regards to other operations. This means that there are times where initializing them first is required.

I had a case just the other day where I needed it to print an actual "0" if the variable never incremented, which it won't do unless explicitly set first. I think the OP code here may come up against the same issue.

Speaking of which, the above section can be shortened a bit:

Code:
BEGIN{ sum=cat1=cat2=cat3=0 }
 
  


Reply

Tags
awk


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Grabbing the last field of a delimited entry without using "awk" or "sed" drandre Linux - Newbie 17 04-18-2012 10:54 PM
awk question on handling *.CSV "text fields" in awk jschiwal Programming 8 05-27-2010 06:23 AM
printing hh in hh:mm using "awk '{FS=":";print $1}'" misses first line of output!! mayankmehta83 Linux - Newbie 2 12-03-2009 02:55 AM
Facing error while running "g++ 3.4.4" code in "g++ 4.3.3" Suranjit Ubuntu 2 10-08-2009 11:22 PM
Replacing "function(x)" with "x" using sed/awk/smth Griffon26 Linux - General 3 11-22-2006 10:47 AM


All times are GMT -5. The time now is 11:58 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration