LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-30-2012, 02:00 AM   #1
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Rep: Reputation: Disabled
Can we run a third party program from an awk script?


The usage of the third-party (or external) program, normally run from the shell, is as follow:
Code:
program [input] [output] [options]
(The output argument being compulsory)

My gawk script is made of three parts:
Code:
gawk '{
    <do stuff...>

    print result > "file1"        # part1: produce a file to be used as input for the third party program

    "program file1 file2 [options]"           # part2: run the third-party program
    close("program file1 file2 [options]")

     while((getline < file2) > 0)        # part3: use the output of the third-party program within the script
     <do stuff...>

}'
part 1 works but nothing is displayed on my terminal.

That's why I am wondering if it is possible at least?

Thanks for your help !

Last edited by lcvs; 09-30-2012 at 02:56 AM. Reason: add gawk' in the code
 
Old 09-30-2012, 02:37 AM   #2
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978
In GNU awk the correct way to run an external program (apart the system function) is:
Code:
  print COMMAND | "/bin/sh"
  close("/bin/sh")
that is you print the command line you want to execute and pipe it to the shell. Use parentheses to embed the COMMAND if you build it from different parts (substrings), e.g.
Code:
  command = "ncks"
  options = "-O -h"
  options = ( options " -d time,0,0" )
  print ( command, options, "file1", "file2" ) | "/bin/bash"
  close("/bin/bash")
Hope this helps.
 
1 members found this post helpful.
Old 09-30-2012, 02:55 AM   #3
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
* If I am understanding well you run COMMAND (i.e. the external program) and pipe its output to the shell instead of the terminal.

But the external program I am using must redirect its output to a file (or "error:argument is missing"). I don't see how redirecting the output to the shell will allow me to use it again in the same gawk script...

* I also read about the "system" function but it says that it doesn't return the output, but just a value to show if the command was executed properly.
 
Old 09-30-2012, 03:32 AM   #4
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,505

Rep: Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890
I think you have slightly missed the point. Both the option shown by colucix and the system command will both execute the line you pass to it within the quotes or by variables.

So to use your example, here are 3 alternatives:
Code:
system("program file1 file2 [options]")

print "program file1 file2 [options]" | /bin/SHELL # which ever SHELL you like

print | "program file1 file2 [options]" # this will do the same as previous but use the current SHELL
After performing one of the above you can then use getline to loop over 'file2' to process the output file.

May I also suggest a fourth option, which is you can open your command like a pipe and then communicate with it directly using awk's two-way IO (as discussed in link below):

http://www.gnu.org/software/gawk/man...02dway-I_002fO

The advantage here is that you do not have to complete your external command and then open the output, but rather deal with the output immediately as it is delivered
line by line by your command.
 
2 members found this post helpful.
Old 09-30-2012, 09:03 PM   #5
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
grail: I tried the 4 options you mentioned.
I make some progress, with every options the program runs properly and returns the correct output in the file "file2". However, I cannot redirect "file2" into the awk script, except for option4 which returns a blank line. I must make a mistake in the syntax...

(It seems the external program must use a file as input. Storing "file1" in a variable and give it to the program, returns an error message.
But it doesn't matter if I have to produce "file1" I can remove it after.)

Based on the code I posted in my first post, I tried:
Option 1:
Code:
gawk '{

        <do stuff...>        

	print result > "file1"                              # file1 is the input for the external program
	
	system("program file1 file2 [options]")         # file2 is the output of the external program
	
	while ((getline var < "file2") > 0)          # use file2 do work on it
	print var
	
	close("program file1 file2 [options]")

        <do stuff...>
    
}'
Option 2:
Code:
gawk '{

        <do stuff...>

	print result > "file1"
	
	print "program file1 file2 [options]" | /bin/bash
	
	while ((getline var < "file2") > 0)
	print var
	
	close("program file1 file2 [options]")

        <do stuff...>
   
}'
Option 3:
Code:
gawk '{

        <do stuff...>       

	print result > "file1"
	
	print | "program file1 file2 [options]"
	
	while ((getline var < "file2") > 0)
	print var
	
	close("program file1 file2 [options]")

        <do stuff...>
    
}'
Option 4 (according to the link you provided) is maybe the one that works the best because it returns a blank line to the terminal (in my test, file1 contains only 1 line).
Code:
gawk '{

<do stuff...>

do {
         print result > "file1"
         "program file1 file2 [options]" |& getline var < "file2"
         print var
     } while ((getline var < "file2") > 0)
     close("program file1 file2 [options])
}

<do stuff...>

}'

Last edited by lcvs; 09-30-2012 at 09:15 PM.
 
Old 09-30-2012, 11:51 PM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,505

Rep: Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890
Ok, so the gotcha with options 1 - 3 is that you have closed the system call after trying to read from the newly created file. This means that for all intents and purposes, the new file
has yet to be created or created fully were it large.

So solution to all of them is:
Code:
gawk '{

        <do stuff...>        

	print result > "file1"                              # file1 is the input for the external program
	
	system("program file1 file2 [options]")         # file2 is the output of the external program

        close("program file1 file2 [options]")       # must close call here if you are going to reuse any of the files
	
	while ((getline var < "file2") > 0)          # use file2 do work on it
	print var
	
	close("program file1 file2 [options]")

        <do stuff...>
    
}'
As for option 4, I think you need to first consider how you would perform the same task on the command line with your program with the following 2 criteria:

1. The program receives data from stdin, probably via a pipe

2. The output of the program is sent to stdout, ie the screen

This would negate the need for either file1 or file2 in the code which is where option 4 comes in, ie it gets the data from awk and transfers the output of the command back to awk.
 
1 members found this post helpful.
Old 10-01-2012, 12:52 AM   #7
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
* It works for options 1 - 3. Thanks !
So I assume the second "close("program")" is not necessary. But why we don't have to close the redirections of "file1/2"?

* The option 4 is the most interesting I guess, but cannot probably be realised with this particular program. The program doesn't seem to work within a pipe and needs both input and output files arguments or it returns an error.
So if it can be run in a pipe there is not reason to use the option 4 in the code I suppose.
 
Old 10-01-2012, 05:17 AM   #8
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,505

Rep: Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890
Quote:
But why we don't have to close the redirections of "file1/2"?
Strictly speaking you should but awk allows you to be lazy here as the files are not being used for anything else so it does the tidy
up for you when the program closes.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Run a script from a C program? shilpa rangappa Programming 14 01-12-2010 07:54 AM
Run AWK script on contents of a variable EmrldDrgn Linux - Newbie 3 07-10-2009 05:05 AM
awk run script on field before output Geneset Programming 3 08-26-2008 04:59 AM
How do I run an awk script? davee Programming 2 08-12-2003 08:46 AM
How to run a shell command containing awk, and grep within a C program Linh Programming 3 06-05-2003 07:05 PM


All times are GMT -5. The time now is 02:09 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration