reading input from telnet into php

MissEileen · 01-26-2009, 06:18 AM

hi all,
I need help with a little project:
I have a sensor whose data must be shown in a web page. up to now, the only way to acquire data is by using:

$ telnet xxx.xxx.xxx.xxx 10001

which results in an infinite loop of data written (1 line per second), with the following format:

30 lines of: HELLO "foo"
1 line of: GOODBYE "the data i care about (numbers)"

what i need is to read the numbers following the keyword GOODBYE. so, i first tried outputting the lines i need into a file, to let php do its work from there.

from the shell, if i do:

$ telnet xxx.xxx.xxx.xxx 10001 |grep GOODBYE

i get only the lines i need. if i do:

$ telnet xxx.xxx.xxx.xxx 10001 > file.txt

i get all the lines written into the file; but if i do

$ telnet xxx.xxx.xxx.xxx 10001 |grep GOODBYE >file.txt

nothing is written into the file!

so, my questions are:
- why isn't the last command working as i expect? or
- do you have any other ideas on how to approach this project?

thank you very much!

sydney-troz · 01-26-2009, 08:32 AM

Not sure why it wouldn't work off the top of my head, but what happens if you do

Code:

telnet xxx.xxx.xxx.xxx 10001 | grep GOODBYE | tee file.txt

tredegar · 01-26-2009, 08:39 AM

I don't know why it's not working. bash can be awkward sometimes

I'd try
telnet xxx.xxx.xxx.xxx 10001 > file1
grep GOODBYE file1 > file2
cat file2

MissEileen · 01-26-2009, 09:19 AM

correction:

Code:

$ telnet xxx 10001 |grep GOODBYE > file.txt

writes GOODBYE lines to file, but only after killing the telnet process (which is not useful); same happens

Code:

$ telnet xxx 10001 |grep GOODBYE |tee file.txt

any ideas how to fix this?

MissEileen · 01-26-2009, 09:23 AM

tredegar: thanks; i'd rather process "file1" with php directly (which is what I'm currently doing), but I really don't want the file growing unnecessarily big. I know I can rename/delete it on a periodic basis with a cronjob, but I'm looking for an "elegant" solution.

tredegar · 01-26-2009, 09:50 AM

Quote:

writes GOODBYE lines to file, but only after killing the telnet process (which is not useful)

Ah Ha! This is because your telnet to port 10001 is continually reading lines, and never finishes.
There must be a better way to get the data you need.
Maybe have the remote host write the sensor data to a file and then just
scp the file from the remote host?

What sort of process do you have running on port 10001 on the remote host?

Can you make it send the sensor data, and then terminate the connection?

MissEileen · 01-26-2009, 11:49 AM

I don't mean to be rude, but I already said in the first post that the telnet produces an infinite loop and it's the only way to acquire data. source machine is given as a black box; no chance for scp. thanks.

tredegar · 01-26-2009, 12:31 PM

Quote:

I don't mean to be rude, but I already said in the first post that the telnet produces an infinite loop and it's the only way to acquire data.

I don't mean to be rude either, but you omitted the part I have underlined

Maybe write a script along the lines of:

Code:

#!/bin/bash
# WARNING pseudocode follows
do
  read foo
  if $foo is what you are interested in then do_something_with_it
until you_are_bored

Call it like this:

Code:

telnet xxx.xxx.xxx.xxx 10001 | /path/to/script

MissEileen · 01-26-2009, 01:48 PM

Looks like a good idea, but I have almost no experience with bash. I don't know if the condition
"you_are_bored" might be a timestamp or a period of time, or a certain amount of lines in the
file, etc. So, it's gonna take a while for me to test, but I'll try it; thank you.

by the way, the exact phrase "the only way to acquire data is by using: $ telnet ..." is on my
first post indeed.

tredegar · 01-26-2009, 04:57 PM

Quote:

Looks like a good idea, but I have almost no experience with bash

I'm also still a beginner with bash, but not quite a "newbie".

The point I am making is that you are receiving an (it seems now, an endless) data-stream from your telnet command.

You can pipe (the "|" in my previous post) this data to a script that will read each line one at a time and assign each line to the variable $foo

You can then use bash scripting to deal with each line (as assigned to the variable $foo) in turn: If it is saying HELLO foo, then you can safely discard it (but why does it need to do this THIRTY times?)

If it is saying "GOODBYE "the data i care about (numbers)"" then you can filter this out ( maybe with grep and cut ?) and then process it as you see fit.

Quote:

"you_are_bored" might be a timestamp or a period of time, or a certain amount of lines in the file, etc.

Well, that was a kind of a joke. Sorry. You could define you_are_bored=false and it would continue forever, which is maybe what you need. It was pseudocode anyway. Cheer up!

Seriously, I am alerting one of LQ's bash-gurus to this thread (Tinkster, are you listening? HELP!), and I hope he'll be able to help you better than I can. He may be busy though.

Quote:

by the way, the exact phrase "the only way to acquire data is by using: $ telnet ..." is on my first post indeed.

Ok. I'm just trying to help you out. I'm busy too. My apologies if I mis-read.

MissEileen · 01-26-2009, 08:42 PM

[OFF TOPIC!]

hey tredegar,
I might not know a lot of bash scripting, but I do understand pseudocode; so when I said your idea looks good and I'm gonna try it, I really meant that (well, maybe next week, 'cause my work shift ends tomorrow); no need to further explain! Also, I did get the joke and appreciate your humor and help, 'cause in most forums people are jerks and treat you badly just for not knowing stuff. Actually, this is the only place where I have found nice, patient answers; so, thank you very much.

Just to satisfy your curiosity: the output strings aren't actually "HELLO foo" and "GOODBYE foo"; they're just 2 different strings followed by numbers. I changed them to explain easily. The HELLO lines contain current data for one sensor (every second) while the GOODBYE lines contain average values for all sensors every 30 secs, which are the data I need to work with. And the source data machine is a weather station with an IP, not a computer, hence the unfeasibility of any command other than telnet through port 10001.

***sorry if I broke any rules by going off topic ***

michaelk · 01-26-2009, 09:23 PM

Just to throw out something else to try would be the netcat command.

tredegar · 01-27-2009, 03:01 AM

Tinkster is indeed busy, but he recommends you take a look at the telnet man-page, searching for "flush", and also suggests you may need to use expect to get it working.

netcat looks a good tool too (thanks michaelk)

So, give it a go.

If / when you come back here with some code that is partly working, it would be useful if you gave us a real sample of the data stream you are trying to process (within [CODE] tags please, so we can see exactly how it is formatted)

Good luck.

humandynamo · 01-29-2009, 10:01 AM

I believe this is on topic as it is actually concerning netcat. I am currently having some struggles with the the processing of the <STDIN> from my executable.

netcat -v -v IPaddress port -e ./myprogram.pl

Within "myprogram.pl" I do the following to get the user input:
$sLine = "";
$sLine = <>;
print = "Received: $sLine\n\r";

I receive the warning "Use of uninitialized value in concatenation (.) at ./myprogram.pl"

I expected the program to wait till there was information to the <STDIN> before processing. Does anybody know why the program does not wait for user input?

tredegar · 01-29-2009, 12:31 PM

I hope michaelk can help you with this as I have not used netcat or written perl programs.
It might be a better idea to start a new thread "netcat and my perl program"