LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 05-11-2007, 05:23 PM   #1
the_imax
Member
 
Registered: Apr 2004
Distribution: RHEL 4/ CentOS
Posts: 152

Rep: Reputation: 30
Putting awk in bash script


I have written a awk script which is working fine when I use it with

Code:
awk -f script_file datafile
however I also want to grep the awk output and to complete this in a single step I want to put both awk and grep it in a bash script file

so how can i put it in a bash script ?

I tried and it kept on giving the error "
Code:
Error: no such instruction:"
I had only put #!/bin/bash at the top of the file and added awk before the start of the awk commands

Last edited by the_imax; 05-11-2007 at 05:25 PM.
 
Old 05-11-2007, 05:42 PM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Too bad you did not post what you have tried. Here's how it's done most of the time:

Quote:
#!/bin/bash

... some 'normal' bash code

awk '{

.. your awk code goes here...

}' infile | grep <something>


...some more bash code...

exit 0
Although you could use the awk -f script_file infile line, I prefer to put the awk code that's in the script_file in the bash script. It makes it more readable (together with 'all' the other bash stuff). It's also a bit more resource friendly (no extra file that needs to be opened).

Hope this clears things up a bit.
 
Old 05-11-2007, 06:28 PM   #3
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655
You can execute the awk script the same way in the shell as you do from the console.

Code:
#!/bin/bash
awk -f script_file datafile | grep ...
That will allow you edit the awk script on its own or still use the awk command as before.
You might want to have a standard place for your awk scripts, such as ~/bin/ or ~/bin/awk/.
Code:
#!/bin/bash
AWKDIR=$HOME/bin/awk
awk -f $AWKDIR/script_file datafile | grep ...

Last edited by jschiwal; 05-11-2007 at 06:31 PM.
 
Old 05-11-2007, 10:59 PM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by the_imax
I have written a awk script which is working fine when I use it with

Code:
awk -f script_file datafile
however I also want to grep the awk output and to complete this in a single step I want to put both awk and grep it in a bash script file

so how can i put it in a bash script ?

I tried and it kept on giving the error "
Code:
Error: no such instruction:"
I had only put #!/bin/bash at the top of the file and added awk before the start of the awk commands
and most probably what you want to do with grep, it can be done in your awk code as well. awk is also a "grepping" tool.
 
Old 05-12-2007, 01:44 AM   #5
the_imax
Member
 
Registered: Apr 2004
Distribution: RHEL 4/ CentOS
Posts: 152

Original Poster
Rep: Reputation: 30
here is the script which I am trying
Code:
#!/bin/bash

awk' {BEGIN {

FS =  " " }

print $1 , "it is working"

END
}'
I am executing it from command line using

Code:
./script_file  data_file
and it is giving error

Code:
[root@localhost ~]# ./script_file data_file
./script_file: line 10: awk {BEGIN {

FS =  " " }

print $1 , "it is working"

END
}: command not found
so where am I wrong

Last edited by the_imax; 05-12-2007 at 01:58 AM.
 
Old 05-12-2007, 04:43 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

There are 2 things 'wrong' with the script:

- the awk syntax used is not correct,
- you don't feed the infile to awk.

Quote:
#!/bin/bash

awk '
{ print $1, "is it working" }
' $1
The $1 represent the first argument given to the script at the command line. I.e:
./script_file data_file
The part in italics is seen as $1 inside the script. This needs to be given to the awk code (hence the $1 after the last ').

The awk syntax you used is also not correct:

The BEGIN and END part are not needed in your case.
The BEGIN part you use sets FS (the File Seperator) to a space, this is the awk default (ok, its a space or a tab).

Both BEGIN and END are special. Everything inside the BEGIN part is executed once _before_ the input is read and the END part is executed once _after_ the input is read.

Here are 2 examples of both, first the use of the BEGIN part:
Quote:
#!/bin/bash

awk '
BEGIN { FS = ":" }
{ print $1, "has the following shell:", $7 }
' /etc/passwd
The above code sets the FS to : and print the user and the shell from the /etc/passwd file. The user is on position 1 and the shell is on position 7 in the passwd file. The FS is set to : because that is the field seperator used in the passwd file.

The END part:
Quote:
#!/bin/bash

awk '
/bash/ { ++x }
END { print x " user(s) with a bash shell" }' /etc/passwd
This code checks the /etc/passwd for the existence of bash (the /bash/ part). If this is the case the counter (x) is increased by 1 (++x). When the whole file is checked, the END part will print the total.

Here are 3 awk related links that could come in handy:
GNU awk user guide
(another) GNU awk user guide
awk primer

Hope this clears things up a bit.

Last edited by druuna; 05-12-2007 at 05:35 AM. Reason: Fixed a typo
 
Old 05-12-2007, 05:51 AM   #7
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by the_imax
here is the script which I am trying
Code:
#!/bin/bash

awk' {BEGIN {

FS =  " " }

print $1 , "it is working"

END
}'
I am executing it from command line using

Code:
./script_file  data_file
and it is giving error

Code:
[root@localhost ~]# ./script_file data_file
./script_file: line 10: awk {BEGIN {

FS =  " " }

print $1 , "it is working"

END
}: command not found
so where am I wrong
most prob you want this:
Code:
#!/bin/bash
awk 'BEGIN {FS =  " " }
     {print $1 , "it is working" }
END {}' $1
so that you can call your script like this: ./yourscript "file"
or you can put your file in the awk argument itself:
Code:
#!/bin/bash
awk 'BEGIN {FS =  " " }
     {print $1 , "it is working" }
END {}' "file"

Last edited by ghostdog74; 05-12-2007 at 06:39 AM.
 
Old 05-12-2007, 06:17 AM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

@ghostdog74: Both examples have a typo/mistake: There's a { before BEGIN, the code will not execute because of this.

Also, the END part isn't needed (the same for the BEGIN part), as stated/explained in my previous reply.

EDIT
The mentioned mistakes are fixed by the poster

Last edited by druuna; 05-12-2007 at 06:54 AM. Reason: mistakes are fixed
 
Old 05-12-2007, 06:44 AM   #9
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by druuna
Hi,

@ghostdog74: Both examples have a typo/mistake: There's a { before BEGIN, the code will not execute because of this.

Also, the END part isn't needed (the same for the BEGIN part), as stated/explained in my previous reply.
thanks. missed that one out. i also merely wanted just to correct the {print} part, and didn't really bother about BEGIN/END, as OP might have other purposes for them..
 
Old 05-12-2007, 04:50 PM   #10
the_imax
Member
 
Registered: Apr 2004
Distribution: RHEL 4/ CentOS
Posts: 152

Original Poster
Rep: Reputation: 30
Thanks , it is working now , as druuna said, all the problem was due to the wrong feeding of datafile in the script

Last edited by the_imax; 05-12-2007 at 04:51 PM.
 
  


Reply

Tags
awk


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
HELP - bash script - cron job - not out putting in 132 boyd98 Programming 12 05-01-2007 07:14 PM
Using global variable in awk (bash script) kopeda Programming 2 04-24-2007 02:47 AM
can awk see bash script arguments ? sharapchi Programming 7 12-14-2006 09:03 PM
Bash script question (grep and awk) hamish Linux - Software 6 04-06-2005 04:14 PM
cannot export result from awk into a variable in a bash script Emmanuel_uk Linux - Newbie 4 03-07-2005 02:54 AM


All times are GMT -5. The time now is 09:08 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration