LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-19-2012, 12:48 AM   #1
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Rep: Reputation: 16
perl while loop substitution


I have a log file which is very dense
You cant see them, but there is a 'CTRL-A' character before each "numeric tag='. The escape character for the CTRL-A is '0001'
Code:
casper@casper-SX280:~$ more dense_file  | head -1
>8=FIX.4.29=027835=834=11605249=NQPX56=S085H652=20120217-14:35:31.191128=S15BX257=6T0050=INET11=6T0010005924641=6T0010005924537=92392017=-11185520=0150=539=5109=MLCO55=NTAP54=538=20032=031=0.0151=20014=06=0.044=42.780040=247=A29=176=INET111=018=N59=558=Replaced10=085
casper@casper-SX280:~$
anyhow I can sed them out with 's/\x01/ /g' - look at those nice spaces in the dense_log !!!
Code:
casper@casper-SX280:~$ more dense_file  | sed -e 's/\x01/ /g' | head -1
> 8=FIX.4.2 9=0278 35=8 34=116052 49=NQPX 56=S085H6 52=20120217-14:35:31.191 128=S15BX2 57=6T00 50=INET 11=6T00100059246 41=6T00100059245 37=923920 17=-111855 20=0 150=5 39=5 109=MLCO 55=NTAP 54=5 38=200 32=0 31=0.0 151=200 14=0 6=0.0 44=42.7800 40=2 47=A 29=1 76=INET 111=0 18=N 59=5 58=Replaced 10=085

So I wanted to write something up in perl - this works
Code:
#!/usr/bin/perl
use strict ;
open (FILE , "dense_file") or die "cannot open up the file";
while (<FILE>) {
        s/\x01/ /g;
        print $_;
        sleep 1;
}
However when I assign a variable to the default varible $_ it stops working.

Code:
#!/usr/bin/perl
use strict ;
open (FILE , "dense_file") or die "cannot open up the file";
while (<FILE>) {
        my $line = $_;
        s/\x01/ /g;
        print $line;
        sleep 1;
}
 
Old 02-19-2012, 07:29 AM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,438

Rep: Reputation: 2842Reputation: 2842Reputation: 2842Reputation: 2842Reputation: 2842Reputation: 2842Reputation: 2842Reputation: 2842Reputation: 2842Reputation: 2842Reputation: 2842
I would think maybe it is an order thing, try switching the sed portion and the assignment.
 
Old 02-19-2012, 02:09 PM   #3
aluchko
Member
 
Registered: Mar 2004
Location: Edmonton, AB, Canada
Distribution: Fedora
Posts: 37

Rep: Reputation: 15
I agree with grail,

Code:
        my $line = $_;
        s/\x01/ /g;
        print $line;
You assign $line to default variable, then perform a substitution on the default variable, which reassigns the default variable to the result of the substitution, but leaving $line with the original value.

Switching the order to

Code:
        s/\x01/ /g;
        my $line = $_;
        print $line;
Will fix the problem
 
Old 02-19-2012, 07:07 PM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,295

Rep: Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358
As aluchko said, you're printing one var, but substituting the other one.
I would use
Code:
my $line;
while (defined( $line = <FILE>) )
{
        $line =~ s/\x01/ /g;
        print $line;
PS This is what I used to test
Code:
$var1="x\x01=dw";
print length($var1)." X${var1}X\n";

$var1 =~ s/\x01/ /;
print length($var1)." X${var1}X\n";

$var1="x\x01=dw";
$var1 =~ s/\x01//;
print length($var1)." X${var1}X\n";

Last edited by chrism01; 02-19-2012 at 07:08 PM.
 
Old 02-19-2012, 08:08 PM   #5
casperdaghost
Member
 
Registered: Aug 2009
Posts: 349

Original Poster
Rep: Reputation: 16
Yeah chrism01 I came to a similiar conculsion myself
The latest iteration of this script stops once it reaches a certain time frame
and substutes values.

Code:
#!/usr/bin/perl 
use strict ; 
open (FILE , "roo") or die "cannot open up the file"; 
while (<FILE>) { 
        my $line = $_; 
        if ($line !~ /52=20120217-14:35:31.399*/) {

                $line =~ s/\x01/ /g; 
                $line =~ s/INET/marmaduke/g; 
                $line =~ s/MLCO/peanut/g; 
                print $line; 
                #sleep 1; 
        }
}
notice the marmaduke and the peanut
I dont know why I have to use the matching operator =~ but it works.
Code:
casper@casper-SX280:~$ ./perl_parse3 | head -1
> 8=FIX.4.2 9=0278 35=8 34=116052 49=NQPX 56=S085H6 52=20120217-14:35:31.191 128=S15BX2 57=6T00 50=marmaduke 11=6T00100059246 41=6T00100059245 37=923920 17=-111855 20=0 150=5 39=5 109=peanut 55=NTAP 54=5 38=200 32=0 31=0.0 151=200 14=0 6=0.0 44=42.7800 40=2 47=A 29=1 76=marmaduke 111=0 18=N 59=5 58=Replaced 10=085
i really don't see too many examples of while loop perl substitutions - I thought that this is what perl is for?

Last edited by casperdaghost; 02-19-2012 at 08:10 PM.
 
Old 02-19-2012, 08:14 PM   #6
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,295

Rep: Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358Reputation: 2358
As per my example, no need to copy $_ to $line, just assign it in the while().
Also, '=~' is the 'match' regex operator in Perl; not-match is '!~', see
http://perldoc.perl.org/perlrequick.html
http://perldoc.perl.org/perlretut.html
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Perl and nested loop substitution Evgeniy Arbatov Linux - General 2 04-28-2010 05:19 AM
substitution in perl vinaytp Linux - Newbie 10 06-23-2009 12:24 AM
substitution question in perl sharky Programming 4 11-03-2008 07:10 PM
Perl substitution with arrays cramer Programming 2 08-14-2006 11:07 PM
Substitution in Perl rigel_kent Programming 4 06-02-2006 10:11 AM


All times are GMT -5. The time now is 12:17 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration