LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
LinkBack Search this Thread
Old 06-13-2010, 11:32 PM   #1
brianmcgee
Member
 
Registered: Jun 2007
Location: Munich, Germany
Distribution: RHEL, CentOS, Fedora, SLES (...)
Posts: 399

Rep: Reputation: 38
Using sed to modify command output


The output of a command changed and I need to extract the data and print it out in a different fassion:

Code:
abcd1=aaaa xx
abcd 2 aaa xx bbb
abcd2=aaaa xy
ab 2 xx aaa bbb ccc xxx
should be transformed to:

Code:
output 1 = xx
item 2 = xx
output 2 = xy
item 2 = xx xxx
Currently I used sed "search1|search2|search3" to get the lines that need to be transformed. But I also need to search for substrings in those lines and I need to print those substrings in a specific order together with other characters.

How is this done with sed? Can someone please enlighten me?
 
Old 06-14-2010, 12:55 AM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
I think we need to see a bit more of the context here, because I'm not exactly sure what you're trying to accomplish. For example, does this command output one line at a time, so that they can be filtered individually, or a block of lines that needs to be formatted all at once? Are the "output" and "item" in the desired output literal strings, or something that's fetched from the input? You mention searching for substrings, but I'm not sure what you mean by that here. A real world example might be more helpful.

Offhand, from the examples given, I imagine awk might be a better tool to use here than sed, since the input can be read as a series of separate fields.
 
Old 06-14-2010, 01:15 AM   #3
vikas027
Senior Member
 
Registered: May 2007
Distribution: RHEL, CentOS
Posts: 1,241

Rep: Reputation: 96
Lightbulb

I have made a script for this purpose.

Code:
cat data | grep = | sed 's/abcd/output /' | awk -F"=" '{print $1}' | sed 's/.*/& =/' > file1
cat data | grep = | sed 's/abcd/output /' | awk '{print $NF}' > file2
paste file1 file2 | sed -e 's/  */ /g'  > final1

rm -f file1 file2  > /dev/null

cat data  | grep -v = | awk '{print "item" " " $2 " = "}' > file1
cat data |  grep -v = > tmp

while read line
do
echo $line | tr " " "\n" | grep x | tr "\n" " " >> file2
echo -e "\n" >> file2
done < tmp

sed -i '/./!d' file2
paste file1 file2 > final2
rm -f file1 file2 tmp

This might not be the best script, but will at least serve your purpose.

Here, "data" is your raw file. Afterm running this script, it will give you two files final1 and final2.

Code:
[linux1@HMLINUX1 abc]$ cat final1
output 1 =      xx
output 2 =      xy
[linux1@HMLINUX1 abc]$
[linux1@HMLINUX1 abc]$ cat final2
item 2 =        xx
item 2 =        xx xxx
Now, I am looking some way out to merge these files as
Code:
output 1 =      xx
item 2 =        xx
output 2 =      xy
item 2 =        xx xxx
I am unable to do this part. Hope someone else will help us out in this.
 
Old 06-14-2010, 02:28 AM   #4
brianmcgee
Member
 
Registered: Jun 2007
Location: Munich, Germany
Distribution: RHEL, CentOS, Fedora, SLES (...)
Posts: 399

Original Poster
Rep: Reputation: 38
Since RHEL 5.3, the SAN WWNs could only be retrieved via the sys filesystem in the following fashion:

Code:
# (systool -c fc_host -v && systool -c fc_remote_ports -v) | grep -E '\b+Class Device path = "/sys/class/fc_host/host[0-9]+"$|\b+node_name.*= "0x[0-9a-f]{16}"$|\b+Class Device path = "/sys/class/fc_remote_ports/rport-[0-9+:[0-9]+-[0-9]+"'
  Class Device path = "/sys/class/fc_host/host0"
    node_name           = "0x50060b0000c28e05"
  Class Device path = "/sys/class/fc_host/host1"
    node_name           = "0x50060b0000c28e07"
  Class Device path = "/sys/class/fc_remote_ports/rport-0:0-0"
    node_name           = "0x500a098089fb3c2e"
  Class Device path = "/sys/class/fc_remote_ports/rport-1:0-0"
    node_name           = "0x500a098089fb3c2e"
  Class Device path = "/sys/class/fc_remote_ports/rport-0:0-1"
    node_name           = "0x500a098089fb3c2e"
  Class Device path = "/sys/class/fc_remote_ports/rport-1:0-1"
    node_name           = "0x500a098087499c23"
  Class Device path = "/sys/class/fc_remote_ports/rport-0:0-2"
    node_name           = "0x500a098087499c23"
  Class Device path = "/sys/class/fc_remote_ports/rport-1:0-2"
    node_name           = "0x500a098087499c23"
  Class Device path = "/sys/class/fc_remote_ports/rport-0:0-3"
    node_name           = "0x500a098087499c23"
  Class Device path = "/sys/class/fc_remote_ports/rport-1:0-3"
    node_name           = "0x500a098089fb3c2e"
This output needs to be formatted to the old representation and should look as follows:

Code:
scsi-qla0-adapter-node=50060b0000c28e05;
scsi-qla0-target-0=500a098089fb3c2e;
scsi-qla0-target-1=500a098087499c23;
scsi-qla1-adapter-node=50060b0000c28e07;
scsi-qla1-target-0=500a098089fb3c2e;
scsi-qla1-target-1=50060b0000c28e07;
As you can see, first I need to search for specific lines in the output. Then the line needs to be rewritten and printed in this specific order.

e.g.:

'host0' should be rewritten to 'scsi-qla0-adapter-node' and its wwn should be extracted from the next line ' node_name = "0x50060b0000c28e05"' so that it looks like '50060b0000c28e05'. Together: 'scsi-qla0-adapter-node=50060b0000c28e05;'.

The scsi-qla{0,1}-target-0 and scsi-qla{0,1}-target-1 are the uniqed wwn values of the rport Device paths. rport-0 meaning, that it is connected to qla0 and 'rport-1' means that it is connected to qla1.

Certainly this would be an easy task by writing a complete script that transforms the output. Even easier when using a higher level language such as Python or Perl.

But I don't want to install a script on the target systems and I have no guarantee, that Perl or Python would be available on all systems. Therefor, I want to use simple bash for extracting the info via SSH command.

I think that this is very doable with a sed command | sort | uniq.

But I need a little help how to first search for a line, then find a substring in that line and rewriting the resulting output to the old format of pre RHEL 5.3.

Last edited by brianmcgee; 06-14-2010 at 02:32 AM.
 
Old 06-14-2010, 02:36 AM   #5
vikas027
Senior Member
 
Registered: May 2007
Distribution: RHEL, CentOS
Posts: 1,241

Rep: Reputation: 96
Dear Brian,

I would recommend you not to post a new query in the same thread.

Has your previous problem resolved ?
 
Old 06-14-2010, 02:53 AM   #6
brianmcgee
Member
 
Registered: Jun 2007
Location: Munich, Germany
Distribution: RHEL, CentOS, Fedora, SLES (...)
Posts: 399

Original Poster
Rep: Reputation: 38
I'm sorry that my posting was confusing. Actually it is the very same problem of reformatting data.

The real world problem is more complex, though.

My current solution is as follows:
Code:
(systool -c fc_host -v && systool -c fc_remote_ports -v) | grep -E '\b+Class Device path = "/sys/class/fc_host/host[0-9]+"$|\b+node_name.*= "0x[0-9a-f]{16}"$|\b+Class Device path = "/sys/class/fc_remote_ports/rport-[0-9]+:[0-9]+-[0-9]+"' | sed -e 's#.*host\([0-9]*\).*"#scsi-qla\1-adapter-node=#' -e 's#.*rport-\([0-9]*\):[0-9]*-\([0-9]*\).*#scsi-qla\1-target-\2=#' -e '$!N;s#\n##' -e 's#=.*node_name.*= "0x#=#g' -e 's#"$#;#g' | sort
scsi-qla0-adapter-node=50060b0000c28e05;
scsi-qla0-target-0=500a098089fb3c2e;
scsi-qla0-target-1=500a098089fb3c2e;
scsi-qla0-target-2=500a098087499c23;
scsi-qla0-target-3=500a098087499c23;
scsi-qla1-adapter-node=50060b0000c28e07;
scsi-qla1-target-0=500a098089fb3c2e;
scsi-qla1-target-1=500a098087499c23;
scsi-qla1-target-2=500a098087499c23;
scsi-qla1-target-3=500a098089fb3c2e;
However I think the sed commands can be shortened and I only want to list targets that have a uniq hexadecimal wwn.

Instead of

Code:
scsi-qla1-target-0=500a098089fb3c2e;
scsi-qla1-target-1=500a098087499c23;
scsi-qla1-target-2=500a098087499c23;
scsi-qla1-target-3=500a098089fb3c2e;
I want:

Code:
scsi-qla1-target-0=500a098089fb3c2e;
scsi-qla1-target-1=500a098087499c23;
Maybe I should try uniq with sipping the first characters....

Has anyone an idea how to shorten the sed commandline?

Last edited by brianmcgee; 06-14-2010 at 03:33 AM. Reason: posting my current solution
 
Old 06-14-2010, 06:21 AM   #7
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,191

Rep: Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784
So based on your input given above I came up with the following, let me know if it helps:
Code:
#!/usr/bin/awk -f

BEGIN{
	FS="\""
}

/host/{
	f=substr($2,length($2))
	getline
	qla[f]="scsi-qla"f"-adapter-node="gensub(/0x/,"","g",$2)
}

/rport/{
	g=gensub(/.*rport-|:.*/,"","g",$2)
	getline
	tmp = gensub(/0x/,"","g",$2)

	if(qla[g] !~ tmp)
		qla[g] = qla[g]"\nscsi-qla"g"-target-"i[g]++"="tmp
}

END{for(x in qla)print qla[x]}
 
Old 06-14-2010, 07:49 AM   #8
brianmcgee
Member
 
Registered: Jun 2007
Location: Munich, Germany
Distribution: RHEL, CentOS, Fedora, SLES (...)
Posts: 399

Original Poster
Rep: Reputation: 38
Hello grail,

thank you for your time and input! This is indeed a very good solution!

Code:
#(systool -c fc_host -v && systool -c fc_remote_ports -v) | grep -E '\b+Class Device path = "/sys/class/fc_host/host[0-9]+"$|\b+node_name.*= "0x[0-9a-f]{16}"$|\b+Class Device path = "/sys/class/fc_remote_ports/rport-[0-9]+:[0-9]+-[0-9]+"' | ./test.awk 
scsi-qla0-adapter-node=50060b0000c28e05
scsi-qla0-target-0=500a098089fb3c2e
scsi-qla0-target-1=500a098087499c23
scsi-qla1-adapter-node=50060b0000c28e07
scsi-qla1-target-0=500a098089fb3c2e
scsi-qla1-target-1=500a098087499c23
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
modify a file that contains \\ using sed or awk j-me Linux - Newbie 7 05-14-2010 08:30 AM
how to modify the output of df command anurupr Linux - Newbie 14 03-21-2010 01:21 PM
Embed command output in sed expression. AwesomeMachine Linux - Newbie 7 02-20-2010 02:48 PM
[SOLVED] Need help with sed to modify only lines of text meeting certain criteria kmkocot Programming 12 08-17-2009 11:50 AM


All times are GMT -5. The time now is 01:21 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration