LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 04-22-2010, 08:08 AM   #1
manya
Member
 
Registered: Apr 2004
Posts: 194

Rep: Reputation: 15
Need help in bash script and grep


Hi All,

I have very long file divided into different sections and I need to match the pattern from that.
I have already done this part that but looking at the file structure
the pattern may be present in different sections and I need to present that section name as well.
Any thoughts on how to handle that..

e.g.

********************************
section 1
******************************
asadffrwrwr
aeererere
afrwrww
dfswwrwwfw
fwrwtwet
sfwrtrw
matching partern
afrwrww
dfswwrwwfw
fwrwtwet
sfwrtrw
********************************
section 2
******************************
asadffrwrwr
aeererere
afrwrww
dfswwrwwfw
fwrwtwet
sfwrtrw
matching partern 2
afrwrww
dfswwrwwfw
fwrwtwet
sfwrtrw



grep -i -E '^match' file

matching partern
matching partern 2



But I am looking something like this.

********************************
section 1
******************************
matching partern


********************************
section 2
******************************
matching partern 2
 
Old 04-22-2010, 08:51 AM   #2
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,809

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
Let's confirm what you need to do....
As I read it, you want to find a pattern and then print out the line which contains it AND the name of the section in which it appears.

How about this?
Code:
sed -n -r '/(section|pattern)/p' filename
 
Old 04-22-2010, 10:00 AM   #3
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,008

Rep: Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193
I like the simplicity, but what happens if the section does not contain the pattern?
I am not sure how complicated the file is but if there are multiple "sections" with some not containing
you will still get the header. (I would like to see the sed alternative)
This seems to work:
Code:
awk 'BEGIN{FS=RS}/^\*/{var=$0;for(i=1;i<3;i++){getline;var=var"\n"$0}}/^match/{print var"\n"$NF}' filename
 
Old 04-22-2010, 10:34 AM   #4
manya
Member
 
Registered: Apr 2004
Posts: 194

Original Poster
Rep: Reputation: 15
pixellany, you got me right.
let me try both the options and see which one is convenient. Will keep you posted.
 
Old 04-22-2010, 10:43 AM   #5
manya
Member
 
Registered: Apr 2004
Posts: 194

Original Poster
Rep: Reputation: 15
grail, would you please explain your script to me? I am bit confused in the for loop and not sure how modify it according to my need
Also, the section title name it may not be same so when the patten is match not sure how to extract the section title.
 
Old 04-22-2010, 09:37 PM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,008

Rep: Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193
No probs:
Quote:
BEGIN{FS=RS}
In awk the default record separator(RS) is "\n", so here I am setting the field separator(FS) to be the same so that each line is treated as the text I want to see. The default FS is white space.
Quote:
/^\*/
If the line starts with an asterix(*) then do what is in the braces({})
Quote:
{var=$0;for(i=1;i<3;i++){getline;var=var"\n"$0}}
Set variable "var" equal to the whole line($0), which due to previous criteria will be a line full of asterixes.
The for loop then simply says to append the next two(2) lines to the variable with a newline(\n) in between.
So basically this gives you the header in "var", which for your file the first header will be:
Quote:
********************************
section 1
******************************
Quote:
/^match/
Starting with the line after the last row of asterixes in the header, find the first whole line that starts with match.
Again, if true, do the stuff in the braces({})
Quote:
{print var"\n"$NF}
Found what we are looking for so print the header followed by a newline and the field we are in.
Because our FS (from above) is the newline then the number of fields (NF) per line is always 1

Let me know if I have lost you anywhere?
 
Old 04-23-2010, 12:49 AM   #7
Kenhelm
Member
 
Registered: Mar 2008
Location: N. W. England
Distribution: Mandriva
Posts: 360

Rep: Reputation: 170Reputation: 170
Using GNU sed
Code:
sed -rn '/^\*/{N;N;h}; /^match/{G;s/([^\n]*\n)(.*)/\2\n\1\n/p}' file
 
Old 04-23-2010, 02:10 AM   #8
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,008

Rep: Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193
@Kenhelm - thank you very much, my sed skills are still progressing
 
Old 04-23-2010, 08:38 AM   #9
manya
Member
 
Registered: Apr 2004
Posts: 194

Original Poster
Rep: Reputation: 15
grail, its kindaa working for me but I have to tweak the lines a bit and not sure how do I do that. Your help would be much appreciated.
Since there are multiple entries found how do I put separator in between?Is it possible for me to put anything like that
and other would it be possible to add "|" "or" keyword in first awk statement?

like this
awk 'BEGIN{FS=RS}/^; Project|Company/{var=$0;for(i=1;i<3;i++){getline;var=var"\n"$0}}/match/{print var"\n"$NF}' file
 
Old 04-23-2010, 08:56 AM   #10
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,008

Rep: Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193
Yes the pipe (|) can be used to alternate choices. If I understand correctly, you are asking for:
1. any line starting with a semicolon and followed by a space(^; )
2. followed by either Project or Company
Assuming above is true, you just need to add some brackets:
Code:
awk 'BEGIN{FS=RS}/^; (Project|Company)/{var=$0;for(i=1;i<3;i++){getline;var=var"\n"$0}}/match/{print var"\n"$NF}' file
Let me know if I am on the right track?
 
Old 04-23-2010, 09:06 AM   #11
manya
Member
 
Registered: Apr 2004
Posts: 194

Original Poster
Rep: Reputation: 15
That's correct and how do I use output separators? so you know visually they will look good.
 
Old 04-23-2010, 09:10 AM   #12
manya
Member
 
Registered: Apr 2004
Posts: 194

Original Poster
Rep: Reputation: 15
like this, see by giving normal command I am getting this output

; Company Name.....: XYZ (Total Systems)
allow client-ip 1.1.1.1 and server-name abcd.xyz.com
; Company Name.....: XYZ (Total Systems)
allow client-ip 1.1.1.1 and server-name abcd.xyz.com

And I am looking for something like this.

; Company Name.....: XYZ (Total Systems)
allow client-ip 1.1.1.1 and server-name abcd.xyz.com

########################################

; Company Name.....: XYZ (Total Systems)
aallow client-ip 1.1.1.1 and server-name abcd.xyz.com
 
Old 04-23-2010, 09:37 AM   #13
manya
Member
 
Registered: Apr 2004
Posts: 194

Original Poster
Rep: Reputation: 15
Ok - The last one grail

How do I use variable in awk statement? I am not getting the desired result when I am trying to pass a variable value in awk statement

awk 'BEGIN{FS=RS}/^; (Project|Company)/{var=$0;for(i=1;i<2;i++){getline;var=var"\n"$0}}/$PARA/{print var"\n"$NF}' file

I am sorry but I am really not aware of the awk so much.
 
Old 04-23-2010, 09:39 AM   #14
manya
Member
 
Registered: Apr 2004
Posts: 194

Original Poster
Rep: Reputation: 15
Ok -Sorry i got it..

I should have used single tick "'"

Thanks for your all help grail.
 
Old 04-23-2010, 09:58 AM   #15
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,008

Rep: Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193Reputation: 3193
Ok, post 12 - OFS is output field separator and can be accessed by placing a comma (,) between items

post 13 - where is the variable coming from? by the look you are inside a bash script so you need to do one of two things:
1. use awk variable - awk -v var="$PARA"
2. use bash variable inline - /$PARA/ becomes - /'"$PARA"'/
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
bash script grep without double saavik Programming 2 05-14-2009 10:15 AM
Bash: Script with tail and grep Primsi Linux - General 2 11-16-2006 05:30 AM
bash script with grep and sed: sed getting filenames from grep odysseus.lost Programming 1 07-17-2006 11:36 AM
Bash script question (grep and awk) hamish Linux - Software 6 04-06-2005 03:14 PM
bash script and grep syros Programming 4 01-13-2005 03:04 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 01:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration