LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 09-25-2010, 09:50 PM   #1
C.L.
LQ Newbie
 
Registered: Jul 2010
Posts: 17

Rep: Reputation: 1
Awk: Input from one line, execute program; input from next line, execute program


I have a file with two fields of numbers that I want to use as input for another program.

Code:
awk '{system("./inputbashangle $1 $2")}' outfailtest
The above code does not work, as I think it would take the whole first and second fields as the input for one particular instance of the program 'inputbashangle'. What I want is to get the first two numbers from the first line of the file 'outfailtest', execute 'inputbashangle' with them, then move on to the first two numbers of the second line for all the lines of 'outfailtest'.

How would I do this?
 
Old 09-25-2010, 10:10 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
In scripting, commands are executed in serial fashion; in general, a command runs only after the command preceding it terminates (barring some complicated workarounds).

The usual way to proceed is to first build lists of entries you want to work on, holding them in variables or temp files, then use a separate loop to iterate through them.

In order to give you more detailed advice though, I think we'd need to see an example of the contents of the files in question, as well as what you want the actual output to look like. Your current description is a little too vague for me to understand fully.

Last edited by David the H.; 09-25-2010 at 10:11 PM. Reason: reworded for clarity.
 
1 members found this post helpful.
Old 09-25-2010, 10:14 PM   #3
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,006

Rep: Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191
Well ultimately using awk here seems to be a complete waste as it is doing nothing only awkish.
In saying that it could be used like so:
Code:
awk '{system("./inputbashangle" $1 $2)}' outfailtest
This now tells awk that the items being referenced are awk fields.

As I said thought, it would just be better off either within your existing script or in a bash script to call yours:
Code:
while read -r one two
do
    ./inputbashangle one two
done<outfailtest
 
1 members found this post helpful.
Old 09-25-2010, 10:19 PM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
To follow up, perhaps something like this would work. It's untested though.
Code:
while read line; do
./inputbashangle $line
done <<<"$(awk '{print $1 $2}' outfailtest)"
 
1 members found this post helpful.
Old 09-25-2010, 10:35 PM   #5
C.L.
LQ Newbie
 
Registered: Jul 2010
Posts: 17

Original Poster
Rep: Reputation: 1
Quote:
Originally Posted by David the H. View Post
In scripting, commands are executed in serial fashion; in general, a command runs only after the command preceding it terminates (barring some complicated workarounds).

The usual way to proceed is to first build lists of entries you want to work on, holding them in variables or temp files, then use a separate loop to iterate through them.

In order to give you more detailed advice though, I think we'd need to see an example of the contents of the files in question, as well as what you want the actual output to look like. Your current description is a little too vague for me to understand fully.
Note: I was wrong about what was wrong in the OP. Read the end for details of what's actually wrong.

Ok. I have a simulator called '3bodyangle', which takes in two coordinates and outputs many lines of ten or so numbers each. Right now I have two scripts: the first one merely reads two numbers as input for the simulator:
Code:
 more inputbashangle
#!/bin/bash
read A B
         ./3bodyangle "$A" "$B" 1 0.001 0.002 >> outputangle/outangle"$1"-"$2"
The second one sends the prior script to a remote cluster. This is the script mentioned in my OP.
Code:
 more massinputangle
awk '{system("bsub -q bss24 -e angleerr -o angletest ./inputbashangle $1 $2")}' outfailtest
I have two scripts because the default cluster output has some extraneous information in it (such as what the submitted job's name is, the CPU time, etc) that I do not want. The first script, 'inputbashangle', gives me the raw data. The file 'outfailtest' looks like this (apologies, the formatting is odd when copied and pasted):
Code:
0.06    0.01    4       0.0100025       0       0      8.48798e-314    5
0.06    0.02    4       0.0100018       0       0       8.48798e-314    5
0.07    0.03    4       0.010002        0       0       8.48798e-314    5
0.08    0.05    4       0.0100014       0       0       8.48798e-314    5
0.09    0.06    4       0.0100004       0       0       8.48798e-314    5
0.09    0.07    4       0.0100012       0       0       8.48798e-314   5
This is what I need help with:
Right now, if I execute 'massinputangle' the correct number of jobs is submitted, which is one job per line in the text file 'outfailtest', which 'massinputangle' uses as input. However, I only get one output file with the information from all of the jobs, which is called 'outangle-'. So it seems that the output from each file is not getting sent to a new output file called 'outangle"$1"-"$2", but rather the "$1" and "$2" are getting ignored in the output file's name, so everything gets sent to one file. Executing the first script, 'inputbashangle', by itself gives the correct output naming (ie if A=2 and B=2, then the output file's name is outangle2-2). So it seems that it's the second script, 'massinputangle', that messes something up.
 
Old 09-25-2010, 10:44 PM   #6
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,187

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Read the section about pipes in the info gawk file, and use gawk rather that awk. In gawk. your programs would simply be gawk '{print $1 " " $2 | "./inputbashangle"}' outfailtest

(This use of a pipe is, if I correctly recall the documentation, a gawk extension. Bi-directional pipes are also available if needed.)

Note: I think your code would, perhaps, have worked if you remembered that $1 $2 concatenates those two strings into a single string. Try awk '{c="./inputbashangle " $1 " " $2;system(c)}' outfailtest

Last edited by PTrenholme; 09-25-2010 at 10:46 PM.
 
1 members found this post helpful.
Old 09-25-2010, 10:58 PM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,006

Rep: Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191Reputation: 3191
@PTrenholme - knew there was something I forgot in my haste ... spaces
 
Old 09-26-2010, 12:36 AM   #8
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
generate your statements first
Code:
awk '{cmd="./inputbashangle "$1" "$2; system(cmd)}' outfailtest
 
1 members found this post helpful.
Old 09-26-2010, 11:44 AM   #9
C.L.
LQ Newbie
 
Registered: Jul 2010
Posts: 17

Original Poster
Rep: Reputation: 1
Quote:
Originally Posted by ghostdog74 View Post
generate your statements first
Code:
awk '{cmd="./inputbashangle "$1" "$2; system(cmd)}' outfailtest
This worked. Thank you very much.

Thank you to everyone who replied; I shall try to figure out why your suggestions worked while mine did not.
 
Old 09-27-2010, 12:06 AM   #10
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
Quote:
Originally Posted by C.L. View Post
I shall try to figure out why your suggestions worked while mine did not.
yours did not work because the forming up of your string is wrong.
Code:
"./inputbashangle" $1 $2
you should put spaces in between.
Code:
"./inputbashangle " $1" "$2
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Scripting question - feed an input file into an if statement line-by-line kmkocot Linux - Newbie 10 01-18-2010 11:49 AM
input new line for every match input packets Programming 1 06-01-2009 03:18 AM
how to read input line by line (awk) slcalice Programming 4 03-09-2009 02:12 PM
How add input value to some varieble in awk program this is probably funny for some of you but I do not know what to do. I have sarajevo Programming 2 05-25-2007 07:54 AM
HELP connecting MIC Input/Line Input with Jack and Qsynth animehair Linux - Software 0 07-28-2006 07:47 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:02 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration