LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-04-2016, 06:49 PM   #1
lamont0
LQ Newbie
 
Registered: Feb 2016
Posts: 20

Rep: Reputation: Disabled
Strange output from awk


I execute a script where I initialize a variable and it effects the output. The script does a recursive search thru a directory list all files less than 32 bytes. This works. I then wanted to add a counter to sum up all the bytes. When I initialize a byte counter to "0" a directory is allowed to be processed as output data. If I set the byte counter to "1", the output is as expected. The counter is off by 1 byte and the directory is not listed.

Code:
  ls -lR /home  2> /dev/null | awk '
  BEGIN{print "LIST OF FILES WITH LESS THEN 32 BYTES", bytes=0}
    NF == 9 && $1 !~ /^d/ && $5 < 32, bytes=bytes+$5
  END{print "Total " bytes " bytes"}' > ~/Chapter7/source/awk_data4
Code:
-rw-rw-r-- 1 linux1 linux1    0 Feb 20 20:01 accounta
-rw-rw-r-- 1 linux1 linux1    0 Feb 20 20:01 accountb
-rw-rw-r-- 1 linux1 linux1    0 Feb 20 20:01 accountc
drwxrwxr-x 2 linux1 linux1 4096 Jan 29 13:37 Chapter1
Also, when I initialize the variable it is printed whit the output. How do you suppress that behavior?

Code:
LIST OF FILES WITH LESS THEN 32 BYTES 0
 
Old 04-04-2016, 07:11 PM   #2
lamont0
LQ Newbie
 
Registered: Feb 2016
Posts: 20

Original Poster
Rep: Reputation: Disabled
I resolved the output of the initialized variable by moving the byte counter after the awk invocation: -v bytes=0. The other behavior has not changed.
 
Old 04-04-2016, 07:44 PM   #3
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
You need to go and look at the User guide site for awk as you have several issues in your code.

Specifically:

1. What does a comma do to a print statement
Code:
print "LIST OF FILES WITH LESS THEN 32 BYTES", bytes=0
2. What is the default action if one is not defined
Code:
NF == 9 && $1 !~ /^d/ && $5 < 32, bytes=bytes+$5
3. What happens when there is no delimiter between information and a keyword
Code:
bytes=bytes+$5
  END
4. What does a comma do to a set of tests (I would be interested to know as well as it never occurred to me to put one in??)
Code:
NF == 9 && $1 !~ /^d/ && $5 < 32, bytes=bytes+$5
 
Old 04-04-2016, 08:22 PM   #4
lamont0
LQ Newbie
 
Registered: Feb 2016
Posts: 20

Original Poster
Rep: Reputation: Disabled
Thank you for your reply,
1 - Without the comma I received a syntax error. However, this is not a problem now.
I moved this after "awk" and before the BEGIN.
Quote:
ls -lR /home 2> /dev/null | awk -v counter=0 '
2 - If the bytes counter is not initialized I still get the one directory listed in
the output.
3- If I remove everything after the END then I receive a syntax error.
Quote:
awk: cmd. line:4: END blocks must have an action part
4 - If I remove the "," I receive a syntax error.
Quote:
awk: cmd. line:2: NF == 9 && $1 !~ /^d/ && $5 < 32 counter=counter+$5
awk: cmd. line:2: ^ syntax error
 
Old 04-04-2016, 10:30 PM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Hence you need to go and look at more examples. Specifically you need to look up what character to use to delimit one section from another ... I can tell you it is not a comma
 
Old 04-05-2016, 09:26 AM   #6
mikenash
Member
 
Registered: Dec 2014
Posts: 84

Rep: Reputation: Disabled
Thanks grail! I contemplated about your directions and a light bulb went off.
This is what I finally changed.
Quote:
ls -lR /home 2> /dev/null | awk -v counter=0 '
BEGIN{print "LIST OF FILES WITH LESS THEN 32 BYTES"}
NF == 9 && $1 !~ /^d/ && $5 < 32 && counter=counter+$5
END{print "Total " counter " bytes"}' > ~/Chapter7/source/awk_data4
 
Old 04-05-2016, 11:43 AM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Not bad, although you are going to run into problems
Code:
NF == 9 && $1 !~ /^d/ && $5 < 32 && counter=counter+$5
The above reads, if there are 9 fields in the line and the first character of the first field is not 'd' and the fifth field is less than 32. . . so far so good. Now lets assume the first
file(s) you find are all of 0 bytes in length. As the counter is set to 0 initially, each of these will make the total for counter equal to 0, which is regarded as false, hence none of
these files will be printed and yet they all should be ... ie. this is not what you wanted.

I am also still surprised that having no delimiter between $5 and END doesn't cause an issue, but I do believe it then does not show the correct results.
Try creating a small subset in a single directory and test your results.
 
Old 04-05-2016, 03:33 PM   #8
mikenash
Member
 
Registered: Dec 2014
Posts: 84

Rep: Reputation: Disabled
Hello grail, thank you for catching the omissions. I am still thinking about that one. I understand why it happens but have not though of the solution. Well, I could cheat! Anyway, I am curious about your comment about the delimiter. If I remove the counter then output is correct. Are you referring to parentheses; ()?
 
Old 04-05-2016, 05:13 PM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Hmmm ... turns out I have not read the doco in a while either Previously you needed a semi-colon between items not delimited by curly brackets. This would still be required
if you placed all of the command on a single line, but not with the new lines in the places you have them. The semi-colon would have been your solution though to your first print issue
and it would have told the print command to stop and then setting your variable would not have been included in the output.

You will need to incorporate curly braces into the main part of your script so as to not worry about the zero values causing incorrect output.
 
Old 04-05-2016, 05:44 PM   #10
mikenash
Member
 
Registered: Dec 2014
Posts: 84

Rep: Reputation: Disabled
Hello grail, thank you for responding. I apologize but I am not quite understanding the explanation. When you state all the command are you stating all 4 lines on only 1 line? Then a semi-colon before the increment of the counter? Also, I have tried curly brace everywhere but I always get syntax errors.
 
Old 04-05-2016, 06:04 PM   #11
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Code:
NF == 9 && $1 !~ /^d/ && $5 < 32{counter=counter+$5;print}
 
Old 04-05-2016, 07:36 PM   #12
lamont0
LQ Newbie
 
Registered: Feb 2016
Posts: 20

Original Poster
Rep: Reputation: Disabled
Well, I had tried this and the listing for the files was suppressed. Then I thought, if one line provides the file listing and the other the counter then why not use both! That is what I did. Thanks grail, I hope this is correct!
Quote:
NF == 9 && $1 !~ /^d/ && $5 < 32
NF == 9 && $1 !~ /^d/ && $5 < 32{counter=counter+$5}
 
Old 04-05-2016, 07:45 PM   #13
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,711

Rep: Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279
Or you could include a "print $0;" in the block containing counter=counter=$5.
 
Old 04-05-2016, 09:18 PM   #14
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Quote:
I had tried this and the listing for the files was suppressed.
Not sure I follow ... you had tried what I wrote and you were getting no output?? The print statement should print the current line
 
Old 04-06-2016, 10:55 AM   #15
mikenash
Member
 
Registered: Dec 2014
Posts: 84

Rep: Reputation: Disabled
Hello grail, once again thank you. I had tried the curly braces around the counter and I would not get any file listings. If I left the curly braces off then I would get the file listing but lose the zero byte file listings until the counter became positive. The print $0 resolved these two issues. I never saw or used awk before this past Sunday so this was a great learning exercise. Thank you.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Pass search results to awk, and use awk output to search other files bspears1 Linux - Newbie 8 07-21-2012 10:17 AM
[SOLVED] Bash; awk or sed output to variable: how keep newline at end of each output line porphyry5 Programming 3 06-10-2011 06:50 PM
awk does not > output Alkass Programming 4 04-25-2010 06:10 PM
awk strange behavior in bash bingmou Linux - Software 7 09-18-2008 12:27 PM
awk strange behaviour in tcsh flyingalex Linux - Software 5 10-17-2003 11:27 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:32 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration