LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-13-2018, 10:56 AM   #1
SasikumarBharanikumar
LQ Newbie
 
Registered: Nov 2018
Posts: 2

Rep: Reputation: Disabled
How to extract value from XML payload using Shell script


I have runtime XML file which has predefined format and Its being created in Jenkins workspace. I just want to parse the XML payload by using shell script.

Code:
<test-results>
     <test-method status="FAIL" name="flowVISA" duration-ms="843329" data-provider="533w43werwer" finished-at="2018-11-01T23:30:48Z">
       <params>
         <param index="0">
           <value>
           <![CDATA[ account_number:22988419 ]]>
           </value>
         </param>
         <param index="1">
           <value>
           <![CDATA[ txn_id:6wdadfsad2134330L ]]>
           </value>
         </param>
         <param index="2">
           <value>
           <![CDATA[ amount:1100 ]]>
           </value>
          </param>
          <param index="3">
            <value>
            <![CDATA[ currency:USD ]]>
            </value>
          </param>
          <param index="4">
            <value>
            <![CDATA[Id:11a09 ]]>
            </value>
          </param>
          <param index="5">
            <value>
            <![CDATA[Name:Consumer [testId=AS1-TC2, description=Txn amount; wallet -Bal, BA,CC,VISA Credit; Consumer - CC;No other preference set]]]>
            </value>
          </param>
        </params>
     </test-method>
     <test-method status="PASS" name="flowVISA" duration-ms="843329" data-provider="533w43werwer" finished-at="2018-11-01T23:30:48Z">
         <params>
           <param index="0">
              <value>
              <![CDATA[ account_number:22988419 ]]>
              </value>
           </param>
           <param index="1">
             <value>
             <![CDATA[ txn_id:6wdadfsad2134330L ]]>
             </value>
           </param>
        </params>
      </test-method>
      <test-method status="FAIL" name="flowVISA" duration-ms="843329" data-provider="533w43werwer" finished-at="2018-11-01T23:30:48Z">
        <params>
           <param index="0">
              <value>
              <![CDATA[ account_number:22988419 ]]>
              </value>
           </param>
           <param index="1">
              <value>
              <![CDATA[Name:Consumer [testId=AS1-TC3, description=Txn amount; wallet -Bal, BA,CC,VISA Credit; Consumer - CC;No other preference set]]]>
              </value>
           </param>
         </params>
      </test-method>
  </test-results>
The above payload I have to read the status of test-method if it's "FAIL" then I need to get the "testId" value from that particular test method. The above payload I have 3 test methods only two had status Failed I need to fetch both Test id and assign to variable as like below

Expected Output:

Code:
 fetchResult = AS1-TC2,AS1-TC3
I just need to fetch the "testId" of failed test-methods and assign it to variable with comma separated using shell script.

I tried the below lines but It doesn't return the entire test-method tags

Code:
failedTC=`grep "test-method.*FAIL" results.xml | sed -e 's/^.*test-instance-name="(.*)-----.*/\1/'
Output:

Code:
 <test-method status="FAIL" name="flowVISA" duration-ms="843329" data-provider="533w43werwer" finished-at="2018-11-01T23:30:48Z">
I want to return entire <test-method> ... </test-method> for status="FAIL"
 
Old 11-13-2018, 11:09 AM   #2
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 22,815

Rep: Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317
Quote:
Originally Posted by SasikumarBharanikumar View Post
I have runtime XML file which has predefined format and Its being created in Jenkins workspace. I just want to parse the XML payload by using shell script.
Code:
<test-results>
     <test-method status="FAIL" name="flowVISA" duration-ms="843329" data-provider="533w43werwer" finished-at="2018-11-01T23:30:48Z">
       <params>
         <param index="0">
           <value>
           <![CDATA[ account_number:22988419 ]]>
           </value>
         </param>
         <param index="1">
           <value>
           <![CDATA[ txn_id:6wdadfsad2134330L ]]>
           </value>
         </param>
         <param index="2">
           <value>
           <![CDATA[ amount:1100 ]]>
           </value>
          </param>
          <param index="3">
            <value>
            <![CDATA[ currency:USD ]]>
            </value>
          </param>
          <param index="4">
            <value>
            <![CDATA[Id:11a09 ]]>
            </value>
          </param>
          <param index="5">
            <value>
            <![CDATA[Name:Consumer [testId=AS1-TC2, description=Txn amount; wallet -Bal, BA,CC,VISA Credit; Consumer - CC;No other preference set]]]>
            </value>
          </param>
        </params>
     </test-method>
     <test-method status="PASS" name="flowVISA" duration-ms="843329" data-provider="533w43werwer" finished-at="2018-11-01T23:30:48Z">
         <params>
           <param index="0">
              <value>
              <![CDATA[ account_number:22988419 ]]>
              </value>
           </param>
           <param index="1">
             <value>
             <![CDATA[ txn_id:6wdadfsad2134330L ]]>
             </value>
           </param>
        </params>
      </test-method>
      <test-method status="FAIL" name="flowVISA" duration-ms="843329" data-provider="533w43werwer" finished-at="2018-11-01T23:30:48Z">
        <params>
           <param index="0">
              <value>
              <![CDATA[ account_number:22988419 ]]>
              </value>
           </param>
           <param index="1">
              <value>
              <![CDATA[Name:Consumer [testId=AS1-TC3, description=Txn amount; wallet -Bal, BA,CC,VISA Credit; Consumer - CC;No other preference set]]]>
              </value>
           </param>
         </params>
      </test-method>
  </test-results>
The above payload I have to read the status of test-method if it's "FAIL" then I need to get the "testId" value from that particular test method. The above payload I have 3 test methods only two had status Failed I need to fetch both Test id and assign to variable as like below
Expected Output:
Code:
 fetchResult = AS1-TC2,AS1-TC3
I just need to fetch the "testId" of failed test-methods and assign it to variable with comma separated using shell script. I tried the below lines but It doesn't return the entire test-method tags
Code:
failedTC=`grep "test-method.*FAIL" results.xml | sed -e 's/^.*test-instance-name="(.*)-----.*/\1/'
Output:
Code:
 <test-method status="FAIL" name="flowVISA" duration-ms="843329" data-provider="533w43werwer" finished-at="2018-11-01T23:30:48Z">
I want to return entire <test-method> ... </test-method> for status="FAIL"
While you probably *CAN* do this with a shell-script, it's going to take more than a single line. If you **KNOW** that you'll always get a Pass/fail before the ID, you can simply grep for those things, shovel them into variables, and do a basic match, outputting what you want, which is quick and dirty. And prone to failure. There are XML parsing libraries for perl and python that let you do just this, very easily:
Code:
#!/usr/bin/perl
use strict;
use warnings;
use XML::LibXML;

my $dom = XML::LibXML->load_xml(location => $InputXML);
 foreach my $title ($dom->findnodes('//test-results')) {
# This splits apart the tags and values into variables. Do things with them.
        $METHOD = $title->findvalue('./test-method');
        $ID = $title->findvalue('./param/value');
....
<grep $METHOD for Fail...if so, output $ID>
Above for example only. Totally untested, but will get you started.
 
Old 11-13-2018, 11:24 AM   #3
SasikumarBharanikumar
LQ Newbie
 
Registered: Nov 2018
Posts: 2

Original Poster
Rep: Reputation: Disabled
Can you please give sample shell script to solve this problem.
 
Old 11-13-2018, 11:47 AM   #4
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 22,815

Rep: Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317
Quote:
Originally Posted by SasikumarBharanikumar View Post
Can you please give sample shell script to solve this problem.
No; we are not going to write your scripts for you, but we will be happy to HELP you. You were given suggestions for how to do this in bash, and provided an example of how to use perl to do it. It is now up to **YOU** to do your own work.

Read the "Question Guidelines" link in my posting signature. Post what you wrote/tried on your own, and tell us where you're stuck and we'll be glad to help.
 
1 members found this post helpful.
Old 11-13-2018, 11:57 AM   #5
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: Debian
Posts: 900

Rep: Reputation: 288Reputation: 288Reputation: 288
Quote:
Originally Posted by SasikumarBharanikumar View Post
Can you please give sample shell script to solve this problem.
I was about to give you one possible solution but I'm not so sure now...
So I will go like TB0ne, grep is not the good tool here because it's designed to work with lines so mainly with flat files and not with structured data like xml.
TB0ne suggested you to use Perl|Python but you can as well do it with xmllint or xmlstarlet. Considering the fact that xmlstarlet has a flatter learning curve and that your requirement is not too complicated, I would go with xmllint (I've just done your exercise myself so it works).
 
1 members found this post helpful.
Old 11-13-2018, 12:04 PM   #6
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 22,815

Rep: Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317Reputation: 6317
Quote:
Originally Posted by l0f4r0 View Post
I was about to give you one possible solution but I'm not so sure now...
So I will go like TB0ne, grep is not the good tool here because it's designed to work with lines so mainly with flat files and not with structured data like xml.
TB0ne suggested you to use Perl|Python but you can as well do it with xmllint or xmlstarlet. Considering the fact that xmlstarlet has a flatter learning curve and that your requirement is not too complicated, I would go with xmllint (I've just done your exercise myself so it works).
Indeed; both are good tools as well. I prefer perl, but that's just me. Aside from what I already gave the OP, it would probably take less than 10 more lines of code to get what they need.
 
1 members found this post helpful.
Old 11-13-2018, 12:32 PM   #7
tofino_surfer
Member
 
Registered: Aug 2007
Posts: 422

Rep: Reputation: 121Reputation: 121
Quote:
I have runtime XML file which has predefined format and Its being created in Jenkins workspace. I just want to parse the XML payload by using shell script.
If you want to parse XML you should use a more powerful language such as perl or python which have actual modules for parsing XML.

For perl:

https://metacpan.org/pod/XML::Parser

For python:

https://docs.python.org/2/library/xm...ementtree.html

Bash shell scripting will be comprised of primitive operations that work on a line by line basis and don't understand XML syntax.

Quote:
I tried the below lines but It doesn't return the entire test-method tags
I want to return entire <test-method> ... </test-method> for status="FAIL"
This would be simple with an XML parsing library. You would iterate over all test_method tags and then operate only on those with FAIL.
 
1 members found this post helpful.
Old 11-14-2018, 06:47 AM   #8
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: Debian
Posts: 900

Rep: Reputation: 288Reputation: 288Reputation: 288
Quote:
Originally Posted by TB0ne View Post
Aside from what I already gave the OP, it would probably take less than 10 more lines of code to get what they need.
Definitely less than 10 more lines to your existing 10
Actually I have a one-liner with no loop needed, like:
Code:
fetchResult=$(xmllint <...> results.xml | sed <...> | tr <...>)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
parse xml and extract the attribute value by search tag in shell boby.kumar Linux - Newbie 4 04-29-2016 02:08 PM
difference between value *value and value * value PoleStar Linux - Newbie 1 11-26-2010 03:37 PM
extract attribute value from xml using bash testac Programming 6 06-21-2010 05:35 AM
Difference between FTP payload Data and simply chopping a file into payload size ahm_irf Programming 1 11-07-2007 08:58 AM
need to extract payload from the tcp packet ahm_irf Programming 1 07-22-2007 06:31 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:39 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration