LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-22-2010, 12:43 AM   #1
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Rep: Reputation: 16
perl on line to take out select values


This is a Fix message - it is a type of protocol for transmitting financial data
each number followed by a = sign is a tag - each tag means something.
some tage are moe important than other.


8=FIX.4.1^9=0235^35=D^34=10^43=N^49=VENDOR^50=CUSTOME^ 56=BROKER^52=19980930- 09:25:58^1=XQCCFUND^11=10^21=1^55=EK^48=277461109^22= 1^54=1^38=10000^40=2^44=76.750000^59=0^10=165 18
8=FIX.4.1^9=0235^35=F^34=10^43=N^49=VENDOR^ 50=CUSTOMER^56=BROKER^52=19980930-09:25:58^ 1=XQCCFUND^11=10^21=1^55=EK^48=277461109^ 22=1^54=1^38=10000^40=2^44=76.750000^10=165 24

I have a perl on liner that takes out some of the tags so i can better see them.

cat sample | perl -nle "print /(39=[\w]/'
35=D
35=F


However, I need something that will take out multipe tags all at once ( instaed of hitting each line with the one liner and then cutting and pasting the values to an excel speadsheet).

I need something like this - something that takes out the same tags from each line:
35=D 43=N 56=Broker 54=1
35=F 43=N 56=Broker 54=1
 
Old 01-22-2010, 07:15 AM   #2
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 650Reputation: 650Reputation: 650Reputation: 650Reputation: 650Reputation: 650
This isn't perl - if your data stays in the format you posted this should do the job though:

Code:
cat sample.txt | cut -d^ --output-delimiter=" " -f3,5,8,16 | sed 's/[[:space:]]\{1,\}/ /g'
Output:
35=D 43=N 56=BROKER 54=1
35=F 43=N 56=BROKER 54=1

cheers

Last edited by kbp; 01-22-2010 at 07:20 AM. Reason: found a way to make it shorter
 
Old 01-24-2010, 12:40 PM   #3
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
FIX regex perl while loop

the first three tags of fix messages are always in the same order - but the rest arr not. Sed and awk will not work, because it goes by column, I need something that will go after the characters itself.
perl -nle 'print if /regex/' works great - for one regex at a time.
I guess I will try making a while loop - that reads in the whole line and splits out the regex one by one.
 
Old 01-24-2010, 01:37 PM   #4
bartonski
Member
 
Registered: Jul 2006
Location: Louisville, KY
Distribution: Fedora 12, Slackware, Debian, Ubuntu Karmic, FreeBSD 7.1
Posts: 443
Blog Entries: 1

Rep: Reputation: 47
You need two things to make this work:
  • An array to hold all of your matching back references
  • A more sophisticated regular expression which will capture all of the fields that you want to capture.

In this case, I'm using the regular expression alternation operator '|' to specify which fields to pull, and then I'm loading all of the matching backreferences into the array '@matches'

Code:
#! /usr/bin/perl -w
while(<>){
        if ( @matches = $_ =~ /((35|43|54|56)=[^^]+)/g){
                print "@matches\n";
        }
}
and ...
Code:
$ perl  /tmp/match.pl /tmp/sample
35=D 35 43=N 43 56=BROKER 56 54=1 54
35=F 35 43=N 43 56=BROKER 56 54=1 54
... it works!

I wrote this as a full perl script, I'll let you golf it down to a one liner.
 
Old 01-24-2010, 11:06 PM   #5
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
yeah that is a nice script - look at what i came up with (don't laugh)

while read line
do
echo $line | perl -nle 'print /(35=[A-Z0-9]{1,2})/'
echo $line | perl -nle 'print /(39=[A-Z0-9]{1,2})/'
sleep 3
printf "this line is over\n"
done < fixmessage3
35=8
39=E
this line is over
35=D
this line is over
35=F
this line is over

i was going to post this and ask the forum
for a way to print the values on one line - like this:
35=8 39=E
this line is over
35=D
this line is over
35=F
this line is over

instead of a list.

Anyhow - this perl script looks really good.

just for reference this is fixmessage3

8=FIX.4.2 9=67 35=8 49=PHLX 56=PERS 11=ATOMNOCCC9990900 52=20071123-05:30:00.000
20=3 150=E 39=E 55=MSFT 167=CS 54=1 38=15 40=2 44=15 58=PHLX EQUITY TESTING 59=
0 47=C 32=0 31=0 151=15 14=0 6=0 10=102
8=FIX.4.1 9=0235 35=D 34=10 43=N 49=VENDOR 50=CUSTOME 56=BROKER 52=19980930- 09
:25:58 1=XQCCFUND 11=10 21=1 55=EK 48=277461109 22= 1 54=1 38=10000 40=2 44=76.7
50000 59=0 10=165
8=FIX.4.1 9=0235 35=F 34=10 43=N 49=VENDOR 50=CUSTOMER 56=BROKER 52=19980930-09
:25:58 1=XQCCFUND 11=10 21=1 55=EK 48=277461109 22=1 54=1 38=10000 40=2 44=76.
750000 10=165
 
Old 01-24-2010, 11:37 PM   #6
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,356

Rep: Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367
Well, if you ever need to check all (or a lot) of fields, I'd use the split() fn
Code:
$var1 = "fixmsg from post #1";
@arr = split(/\^/, $var1);

# Use this for space separated fixmsg
@arr = split(/ /, $var1);
 
Old 01-26-2010, 10:33 PM   #7
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
split function

how would i integrate this split function into the perl program

$var1 = "fixmsg from post #1";
@arr = split(/\^/, $var1);

# Use this for space separated fixmsg
@arr = split(/ /, $var1);
 
Old 01-27-2010, 02:52 AM   #8
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 650Reputation: 650Reputation: 650Reputation: 650Reputation: 650Reputation: 650
The split line will break $var1 up into pieces divided on the '^' character, from there you can process the array something like:

foreach (@arr) {

if ( /^35=.*/ ) { print; }
if ( /^53=.*/ ) { print; }
...

}

cheers
 
Old 01-27-2010, 08:02 AM   #9
bartonski
Member
 
Registered: Jul 2006
Location: Louisville, KY
Distribution: Fedora 12, Slackware, Debian, Ubuntu Karmic, FreeBSD 7.1
Posts: 443
Blog Entries: 1

Rep: Reputation: 47
better to use alternation:

Code:
foreach(@arr){
    print if(/((35|43|54|56)=/);
}
print "\n";
 
Old 02-01-2010, 12:37 AM   #10
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
setting up cgi on ubuntu

this look svery good thank you
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Search values within multiple files line to line Chrizzieej Programming 5 09-26-2008 04:11 PM
Perl: getting %ENV values using su cmd chrism01 Programming 10 05-25-2007 11:11 AM
perl - passing values to subroutines sporty Programming 7 08-08-2006 09:13 AM
How To Put Values In Select Box From Database In Jsp nimishabhatia Programming 1 07-20-2005 01:16 PM
seperating values using perl pantera Programming 3 05-26-2004 12:31 AM


All times are GMT -5. The time now is 08:23 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration