LinuxQuestions.org - [SOLVED] Using sed to modify command output

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Using sed to modify command output (https://www.linuxquestions.org/questions/programming-9/using-sed-to-modify-command-output-813974/)

Using sed to modify command output

The output of a command changed and I need to extract the data and print it out in a different fassion:

Code:

abcd1=aaaa xx

abcd 2 aaa xx bbb

abcd2=aaaa xy

ab 2 xx aaa bbb ccc xxx

should be transformed to:

Code:

output 1 = xx

item 2 = xx

output 2 = xy

item 2 = xx xxx

Currently I used sed "search1|search2|search3" to get the lines that need to be transformed. But I also need to search for substrings in those lines and I need to print those substrings in a specific order together with other characters.

How is this done with sed? Can someone please enlighten me?

I think we need to see a bit more of the context here, because I'm not exactly sure what you're trying to accomplish. For example, does this command output one line at a time, so that they can be filtered individually, or a block of lines that needs to be formatted all at once? Are the "output" and "item" in the desired output literal strings, or something that's fetched from the input? You mention searching for substrings, but I'm not sure what you mean by that here. A real world example might be more helpful.

Offhand, from the examples given, I imagine awk might be a better tool to use here than sed, since the input can be read as a series of separate fields.

I have made a script for this purpose.

Code:

cat data | grep = | sed 's/abcd/output /' | awk -F"=" '{print $1}' | sed 's/.*/& =/' > file1

cat data | grep = | sed 's/abcd/output /' | awk '{print $NF}' > file2

paste file1 file2 | sed -e 's/  */ /g'  > final1



rm -f file1 file2  > /dev/null



cat data  | grep -v = | awk '{print "item" " " $2 " = "}' > file1

cat data |  grep -v = > tmp



while read line

do

echo $line | tr " " "\n" | grep x | tr "\n" " " >> file2

echo -e "\n" >> file2

done < tmp



sed -i '/./!d' file2

paste file1 file2 > final2

rm -f file1 file2 tmp

This might not be the best script, but will at least serve your purpose.

Here, "data" is your raw file. Afterm running this script, it will give you two files final1 and final2.

Code:

[linux1@HMLINUX1 abc]$ cat final1

output 1 =      xx

output 2 =      xy

[linux1@HMLINUX1 abc]$

[linux1@HMLINUX1 abc]$ cat final2

item 2 =        xx

item 2 =        xx xxx

Now, I am looking some way out to merge these files as

Code:

output 1 =      xx

item 2 =        xx

output 2 =      xy

item 2 =        xx xxx

I am unable to do this part. Hope someone else will help us out in this.

Since RHEL 5.3, the SAN WWNs could only be retrieved via the sys filesystem in the following fashion:

Code:

# (systool -c fc_host -v && systool -c fc_remote_ports -v) | grep -E '\b+Class Device path = "/sys/class/fc_host/host[0-9]+"$|\b+node_name.*= "0x[0-9a-f]{16}"$|\b+Class Device path = "/sys/class/fc_remote_ports/rport-[0-9+:[0-9]+-[0-9]+"'

  Class Device path = "/sys/class/fc_host/host0"

    node_name          = "0x50060b0000c28e05"

  Class Device path = "/sys/class/fc_host/host1"

    node_name          = "0x50060b0000c28e07"

  Class Device path = "/sys/class/fc_remote_ports/rport-0:0-0"

    node_name          = "0x500a098089fb3c2e"

  Class Device path = "/sys/class/fc_remote_ports/rport-1:0-0"

    node_name          = "0x500a098089fb3c2e"

  Class Device path = "/sys/class/fc_remote_ports/rport-0:0-1"

    node_name          = "0x500a098089fb3c2e"

  Class Device path = "/sys/class/fc_remote_ports/rport-1:0-1"

    node_name          = "0x500a098087499c23"

  Class Device path = "/sys/class/fc_remote_ports/rport-0:0-2"

    node_name          = "0x500a098087499c23"

  Class Device path = "/sys/class/fc_remote_ports/rport-1:0-2"

    node_name          = "0x500a098087499c23"

  Class Device path = "/sys/class/fc_remote_ports/rport-0:0-3"

    node_name          = "0x500a098087499c23"

  Class Device path = "/sys/class/fc_remote_ports/rport-1:0-3"

    node_name          = "0x500a098089fb3c2e"

This output needs to be formatted to the old representation and should look as follows:

Code:

scsi-qla0-adapter-node=50060b0000c28e05;

scsi-qla0-target-0=500a098089fb3c2e;

scsi-qla0-target-1=500a098087499c23;

scsi-qla1-adapter-node=50060b0000c28e07;

scsi-qla1-target-0=500a098089fb3c2e;

scsi-qla1-target-1=50060b0000c28e07;

As you can see, first I need to search for specific lines in the output. Then the line needs to be rewritten and printed in this specific order.

e.g.:

'host0' should be rewritten to 'scsi-qla0-adapter-node' and its wwn should be extracted from the next line ' node_name = "0x50060b0000c28e05"' so that it looks like '50060b0000c28e05'. Together: 'scsi-qla0-adapter-node=50060b0000c28e05;'.

The scsi-qla{0,1}-target-0 and scsi-qla{0,1}-target-1 are the uniqed wwn values of the rport Device paths. rport-0 meaning, that it is connected to qla0 and 'rport-1' means that it is connected to qla1.

Certainly this would be an easy task by writing a complete script that transforms the output. Even easier when using a higher level language such as Python or Perl.

But I don't want to install a script on the target systems and I have no guarantee, that Perl or Python would be available on all systems. Therefor, I want to use simple bash for extracting the info via SSH command.

I think that this is very doable with a sed command | sort | uniq.

But I need a little help how to first search for a line, then find a substring in that line and rewriting the resulting output to the old format of pre RHEL 5.3.

Dear Brian,

I would recommend you not to post a new query in the same thread.

Has your previous problem resolved ?

I'm sorry that my posting was confusing. Actually it is the very same problem of reformatting data.

The real world problem is more complex, though.

My current solution is as follows:

Code:

(systool -c fc_host -v && systool -c fc_remote_ports -v) | grep -E '\b+Class Device path = "/sys/class/fc_host/host[0-9]+"$|\b+node_name.*= "0x[0-9a-f]{16}"$|\b+Class Device path = "/sys/class/fc_remote_ports/rport-[0-9]+:[0-9]+-[0-9]+"' | sed -e 's#.*host\([0-9]*\).*"#scsi-qla\1-adapter-node=#' -e 's#.*rport-\([0-9]*\):[0-9]*-\([0-9]*\).*#scsi-qla\1-target-\2=#' -e '$!N;s#\n##' -e 's#=.*node_name.*= "0x#=#g' -e 's#"$#;#g' | sort

scsi-qla0-adapter-node=50060b0000c28e05;

scsi-qla0-target-0=500a098089fb3c2e;

scsi-qla0-target-1=500a098089fb3c2e;

scsi-qla0-target-2=500a098087499c23;

scsi-qla0-target-3=500a098087499c23;

scsi-qla1-adapter-node=50060b0000c28e07;

scsi-qla1-target-0=500a098089fb3c2e;

scsi-qla1-target-1=500a098087499c23;

scsi-qla1-target-2=500a098087499c23;

scsi-qla1-target-3=500a098089fb3c2e;

However I think the sed commands can be shortened and I only want to list targets that have a uniq hexadecimal wwn.

Instead of

Code:

scsi-qla1-target-0=500a098089fb3c2e;

scsi-qla1-target-1=500a098087499c23;

scsi-qla1-target-2=500a098087499c23;

scsi-qla1-target-3=500a098089fb3c2e;

I want:

Code:

scsi-qla1-target-0=500a098089fb3c2e;

scsi-qla1-target-1=500a098087499c23;

Maybe I should try uniq with sipping the first characters....

Has anyone an idea how to shorten the sed commandline?

So based on your input given above I came up with the following, let me know if it helps:

Code:

#!/usr/bin/awk -f



BEGIN{

        FS="\""

}



/host/{

        f=substr($2,length($2))

        getline

        qla[f]="scsi-qla"f"-adapter-node="gensub(/0x/,"","g",$2)

}



/rport/{

        g=gensub(/.*rport-|:.*/,"","g",$2)

        getline

        tmp = gensub(/0x/,"","g",$2)



        if(qla[g] !~ tmp)

                qla[g] = qla[g]"\nscsi-qla"g"-target-"i[g]++"="tmp

}



END{for(x in qla)print qla[x]}

Hello grail,

thank you for your time and input! This is indeed a very good solution!

Code:

#(systool -c fc_host -v && systool -c fc_remote_ports -v) | grep -E '\b+Class Device path = "/sys/class/fc_host/host[0-9]+"$|\b+node_name.*= "0x[0-9a-f]{16}"$|\b+Class Device path = "/sys/class/fc_remote_ports/rport-[0-9]+:[0-9]+-[0-9]+"' | ./test.awk 

scsi-qla0-adapter-node=50060b0000c28e05

scsi-qla0-target-0=500a098089fb3c2e

scsi-qla0-target-1=500a098087499c23

scsi-qla1-adapter-node=50060b0000c28e07

scsi-qla1-target-0=500a098089fb3c2e

scsi-qla1-target-1=500a098087499c23