LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-16-2005, 05:00 PM   #1
vitoal18t
LQ Newbie
 
Registered: May 2005
Posts: 7

Rep: Reputation: 0
Perl - Reading Real Time streams


I need help with a script...


I have a program that generates a string of characters to the STDOUT.

I need to write a script to process that stream, the problem is that I don't get any input untill the character generating program quits..


Here is a program that generates characters:

#dtmfdecode | my_perl_script.pl


I tried using pipes, but perl script doesn't get any data unless dtmfdecode quits, but it is a real time process, where dtmfdecode will run in the background generating characters.


Please Help !
Thank you!
 
Old 05-16-2005, 06:00 PM   #2
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 6,552

Rep: Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872
Maybe flush stdin explicitly :
Code:
select((select(STDIN), $| = 1)[0]); # autoflush

while(<STDIN>) {
...
}
[edit]
...or maybe it is perl stdout that need to be flushed
in this case replace
select((select(STDIN), $| = 1)[0]);

with :
select((select(STDOUT), $| = 1)[0]);

Last edited by keefaz; 05-16-2005 at 06:03 PM.
 
Old 05-16-2005, 06:39 PM   #3
vitoal18t
LQ Newbie
 
Registered: May 2005
Posts: 7

Original Poster
Rep: Reputation: 0
Thanx for the code... I guess I have a different case and it didn't solve the problem .


I think what I need is to echo a character when I press it. So if I run the script and press "A" it will print "A" right away without me pressing ENTER. Can this be done?

This way when I pipe the output of my #dtmfdecoder (character generator) I will be able to process characters.


Thank you!
 
Old 05-16-2005, 07:03 PM   #4
vitoal18t
LQ Newbie
 
Registered: May 2005
Posts: 7

Original Poster
Rep: Reputation: 0
I apologize if I am not being clear about my problem.

Let me try again. Here is what I need to do:

Say I would have a program #Stock_Update, that listens to some Network server and continuesly prints out stock updates to the screen without any spaces or new lines.

Example OUTPUT from #Stock_Update:
ABC$1232WAL$34DEL$56IBM$54 and so on.......


Now I need to write a Perl Script that will parse incoming real-time stream of data and produce report, showing largest stock, and average......


I tried piping #Stock_Update | my_perl_script.pl,
but since #Stock_Update never quits and doesn't print newlines I can't receive that data in a Perl Script.


I am sure if #Stock_Update would print out 1000 characters and then QUIT I would be able to take that input and process it, no big deal.

I also tried using FIFO or named Pipes but since #Stock_Update constantly writes to the FIFO it blocks Perl from reading it......



NOW in my case I am NOT using #Stock_Update, but #dtmfdecoder that takes SOUND CARD input and decodes DTMF tones and prints out characters to the screen. I need Perl to parse it just like in the fictional #Stock_Update......


Please Help!
I am sure someone tackled a similar problem. Do I need to use pipes or what?
Thank you!
 
Old 05-16-2005, 08:10 PM   #5
puffinman
Member
 
Registered: Jan 2005
Location: Atlanta, GA
Distribution: Gentoo, Slackware
Posts: 217

Rep: Reputation: 31
The perl cookbook says to use a structure like this. You will have to invoke the stock program from the perl program that is reading its output though. Also, since the other program is piping data continuously, I guess you'd have undef the input record separator and grab each character as it comes. Then maybe sleep for a bit and try for more characters. It would be easier if the stock program had some kind of character to mark the end of a piece of data.

Code:
$pid = open $readme, "-|", "program", "arguments"                                          or die "Couldn't fork: $!\n";

# no input record separator
undef $/;

MAIN:
while (1) {

  while (<$readme>) {
    # ...
  }

  # normally you'd close
  #close $readme or die "Couldn't close: $!\n";

  # take a nap
  sleep 10;
}
 
Old 05-17-2005, 03:54 AM   #6
vitoal18t
LQ Newbie
 
Registered: May 2005
Posts: 7

Original Poster
Rep: Reputation: 0
Thank you for all your help, BUT I am still stuck:

I wrote a simple test program gen.pl

#!/usr/bin/perl

while(1){
$|=1; #Must USE FLUSH else it will never print to screen;
print "Hello Dude";
sleep 1;
}

That generates characters "Hello Dude" and goes forever.

Now here is my Read script:

#!/usr/bin/perl

$pid = open (README, "./gen.pl |") or die "Couldn't fork: $!\n";


undef $/;

select( (select(README), $|=1)[0]);

while(1) {

select( (select(README), $|=1)[0]);
undef $/;
$|=1;
while( <README>)
{print;}


sleep 2;


}


This abover program does nothing ....it doesn't print out what gen.pl generates, so no "Hello Dude" I guess if I can make this work I am
in bussiness.

I appreciate any help!

Thank you!
 
Old 05-17-2005, 05:05 AM   #7
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 6,552

Rep: Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872
In this case, I would use Term::ReadKey module :

Read script:
Code:
use Term::ReadKey;
use strict;

$| = 1;

my $prog;

open ($prog, "./gen.pl |") or die "Couldn't fork: $!\n";

ReadMode 4, $prog;
my $key = '';

while($prog) {
    $key = ReadKey(-1, $prog);
    print $key;
}

ReadMode 0;
close $prog;
 
Old 05-17-2005, 06:25 PM   #8
vitoal18t
LQ Newbie
 
Registered: May 2005
Posts: 7

Original Poster
Rep: Reputation: 0
Thanx! This is much better, but it only printed Hello Dude once.

However after I modified the script to:

use Term::ReadKey;

$|=1;

my $prog;

open($prog, "./gen.pl |") or die "Couldn't fork: $! \n";


while(1) {

ReadMode 'raw';
if( sysread($prog, $key, 1))
{
print $key;
}

}

close $prog;

It prints Hello DudeHello DUDEHello DUDEHello DUDE over and over again, so it is good, problem solved partially !


Now the input for this program was ./gen.pl, which is another perl script that sleeps for 1 sec.

However if I use actual program open($prog, "./dtmfdecoder |") or die "Couldn't fork: $! \n";
the program seems to constantly block the pipe since it always writes to STDOUT and I still get nothing.....

This is beginning to bother me, but I am not giving up. I am sure there is a way to do it in Perl.

Thank you!
 
Old 05-17-2005, 07:10 PM   #9
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 6,552

Rep: Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872
Maybe the dtmfdecoder output is stderr, try open both stdout and stderr like :

Code:
open($prog, "./dtmfdecoder 2>&1 |") or die "Couldn't fork: $! \n";
 
Old 05-17-2005, 08:05 PM   #10
vitoal18t
LQ Newbie
 
Registered: May 2005
Posts: 7

Original Poster
Rep: Reputation: 0
Wow it is finally working... it wasn't writting to STDERR, but thanx for the tip I'll keep that in mind next time..


The reason it didn't work was because the program that I have #dtmfdecode ocassionally starts and doesn't decode as
if it can't open a sound card or something, there are no error messages so it was really hard to debug.

So the problem was not with the script, but with the #dtmfdecode that sometimes fails to initialize properly, I'll have to look into that, but it works.

Perl can process a real-time character stream

I was also able to solve this problem by running #dtmfdecode > .buff on a seperate terminal


and then doing sysread(), like this

open (README, ".buff") or die "Couldn't open file: $!\n";

while(1) {
$|=1;

if( sysread(README, $key, 1))
{
print $key;

}

}





Thanx!


P.S. I also have a program #dtmfencode, I use it from the script using exec 'dtmfencode';

But script quits when the program is done, how can I go back to the script??? Should I fork instead of exec 'dtmfencode';


Thank you for all your help.
 
Old 05-18-2005, 06:59 AM   #11
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 6,552

Rep: Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872Reputation: 872
Try use system('dtmfencode') instead of exec()
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Real Player Integrated Video Streams in Firefox W0bbles Linux - Software 7 12-15-2005 07:17 AM
SuSE 9.3 Can't play real streams any more Riddick SUSE / openSUSE 2 06-21-2005 10:25 AM
reading idle time with perl/shell script daryl314 Linux - General 1 12-27-2004 01:11 PM
"Real time" Apache log filtering with Perl skelly Programming 1 07-01-2004 02:24 PM
mplayer and real streams georgtornqvist Linux - Software 1 08-06-2003 02:50 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 01:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration