Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 10-14-2012, 11:04 PM   #1
LQ Newbie
Registered: Oct 2012
Posts: 4

Rep: Reputation: Disabled
Does "wait" function exist in (g)awk?

It is a general question.

Using a awk or gawk script, let's say we have to process a large input file in two steps for example.

STEP 1: The first step processes every input records and add "1" in a new field at the end if the record contains a regex or "0" if not.

STEP 2: The second step processes each record differently depending on the last field we created just before:
If $NF = "1" then it does the operation A to the record,
If $NF = "0" then it does a different operation B to the record.

(This is a simple example of course)

Instead of taking input records one by one and applying STEP 1 and STEP 2 successively, is there a way to tell awk to apply the STEP 1 to all the input records, wait until they are all processed, and then apply the STEP 2?

Thanks for your help !
Old 10-15-2012, 12:46 AM   #2
Senior Member
Registered: Sep 2009
Location: Orange County, CA
Distribution: Kubuntu x64, Raspbian, CentOS
Posts: 1,849
Blog Entries: 36

Rep: Reputation: 455Reputation: 455Reputation: 455Reputation: 455Reputation: 455
As far as I know that would be using Awk incorrectly. You can treat awk like the stream editor (sed) in that it can only see one line at a time. That means if you have output from a command:
somecommand | awk 'TEST1 { RUN }; TEST2 { RUN }'
Then if the same line matches both TEST1 and TEST2 then both expressions will be run on the same line at the time the line is output. If you would like to have more serialized execution then you may have to consider either using a different language or running two awk commands like the following.
somecommand | awk 'TEST1 { RUN }' | awk 'TEST2 { RUN }'
Note my second "test" case has not actually been tested so it depends on how awk behaves. If it behaves like sed then my "test" case will likely not work either. However, if it behaves more like the sort command (processes the entire stream and then outputs at the end) then my "test" case will work. Try a couple of ways and maybe someone else could provide a better answer.


Last edited by sag47; 10-15-2012 at 12:48 AM.
1 members found this post helpful.
Old 10-15-2012, 02:04 AM   #3
Thad E Ginataom
Registered: Mar 2011
Distribution: Ubuntu 12.04 with KXStudio, MATE & Compiz
Posts: 46

Rep: Reputation: 7
Instead of taking input records one by one and applying STEP 1 and STEP 2 successively, is there a way to tell awk to apply the STEP 1 to all the input records, wait until they are all processed, and then apply the STEP 2?
I think this is one of those obvious-when-you-think-about-it answers: you output your first awk command to a temporary file, and run your second awk on the temporary file.

One has to ask why would you do this? I assume that that 1 or 0 is going to be used by something else: if it is just for this process, then it is an unnecessary stage.

If you must do this in two steps, within a single iteration of awk, I guess you could read the whole file into an array, and then process the array, possibly in an END section. But: keep it simple should be the whole philosophy of shell and awk work.

(awk has two special patterns: BEGIN and END. Whilst it works on a line by line basis, it executes stuff for the BEGIN pattern before it starts to read its input, and the END section after the line-by-line stuff is finished. You can use a BEGIN pattern in awk to write code that has no input: think awk "Hello World"!

By the way: NF is an awk built-in variable. If you try to use it for something else, I think confusion will ensue.

Last edited by Thad E Ginataom; 10-15-2012 at 02:07 AM.
1 members found this post helpful.
Old 10-15-2012, 04:18 AM   #4
Registered: Feb 2012
Posts: 89

Rep: Reputation: Disabled
Thank you guys for your answers!

As I was thinking more about doing this from a single awk script, Thad E Ginataom's method might work better by reading the all file into an array or doing the second step in the END section maybe.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Why do the "£inux" and "PSinux" satirical misspellings even exist?! LOL! Kenny_Strawn General 19 12-09-2010 02:01 PM
Qt "cannot call constructor for""a function-style cast remove the redundant ::QBrush" Peterius Linux - Software 4 11-24-2010 04:57 PM
printing hh in hh:mm using "awk '{FS=":";print $1}'" misses first line of output!! mayankmehta83 Linux - Newbie 2 12-03-2009 02:55 AM
Replacing "function(x)" with "x" using sed/awk/smth Griffon26 Linux - General 3 11-22-2006 10:47 AM
"My posts" and "My threads" should be exempt from the wait time before searches vharishankar LQ Suggestions & Feedback 6 03-06-2006 08:10 PM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:08 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration