Having a tough time with K-shell pipeline

pjz · 03-21-2005, 11:57 AM

Hi-

I'm using Red Hat 9.0 and programming in C++ using the g++ compiler.

I'm going crazy trying to get a pipeline to work out. I have two programs
that constitute a larger model I'm working on. These two programs,
written in C++, are connected to each other using sockets. I've debugged
this system of programs, and if I run it manually, everything is fine. Since I
have a bunch of data points to collect, it's important for me to automate data
collection by trying to build a shell script that will loop though my test
space and run the program over and over at each point in the test space.

If anyone can tell me what socket ID is reliably available every single
time I call for it, then that would solve my problem. I'd just synchronize
on this specific socket in each of the two contributing programs of the
model and be done with this.

I've tried to do this, however, and it works best if I ask for the next available
socket ID from the system. This is okay as long as I manually supply the
socket ID to the program running the socket client. I'd like to feed the socket
ID automatically to the client through the shell, and that's where I'm running
into a problem. I've tracked it down to the below:

As a test, at the command line I type:

prog1 | perl1 | cat

prog1 contains the socket server code and produces a port number after the
socket invocation code using the following line of code

std:cout << "Socket has port # "<< ntoh(server_b.sin_port) << "\n" << std::flush

the perl script checks for the string "Socket" at the start, then strips off the port
number at the end, and prints the port number to standard output.

When I run the test line above, it locks. I need to control-C the terminal window
to get the prompt back.

On the other hand, when I run the following test:

cat xx | perl1 | cat

where xx contains a line that mocks up the output of prog1, the port number is
correctly printed and the prompt returns. In fact, if I run the test like this:

prog1 | perl1

the port number is produced just fine, too.

Folks, I've been beating on this for days, trying different filter choices such as awk,
trying all sorts of shell stuff. I can't seem to get this figured out. Help!!!! And if anyone
knows which socket will be available each and every time, I'll just use that one and
won't have to worry about passing the port number to the client program and that'll
be the end of this headache.

thanks and hope to hear from someone soon
pjz

chrism01 · 03-24-2005, 01:02 AM

I'm not quite clear here. You say you have 2 C++ progs, that talk well over a socket, then you mention 1 prog, 1 perl, 1 cat in your example.
Generally speaking you could either use a pipeline by just preinting to std out in prog1, then pipe into std in on prog2 eg
prog1|perl1
or set the progs to use a specific socket ie supply the fqdn and port num to both writer & reader progs.

pjz · 03-24-2005, 11:12 AM

Chris-

The problem I had was that when I tried to pick a specific socket, I'd occasionally not get the socket and have to retry. That was screwing me up when I tried to automate execution of the model, but I've figured out how to test for success when attempting to have the socket server open at a specific port number. When I fail, I just re-run until I get it. This way, I can code the client with the same port number and I don't have to pass the port number over to the client program.

I attempted to do as you said and print to stdout then read from stdin in a pipeline, but for some reason it stalled. I worked around it, so that's all academic now. Never got that to work for some reason. Don't know what was the deal there, still a mystery.

Thanks for replying
pjz

TheLinuxDuck · 03-24-2005, 11:49 AM

One thing that I noticed right away is, you're printing to the standard output from the C++ program, then using perl to strip/print the socket # which you then pipe through cat? Unless there's an inssue here of which I am not familiar, you can eliminate the perl step by having the C++ program only print the socket #. That will have the same effect as stripping off the other data with perl and printing it from there, and will save you some processing time, via the perl interpreter load time.

As for the connection between the two programs... do they require data to be passed back and forth between the two? Or is it a one way communication? It's it's only one way, then I think that you'd be better off printing the data to stdout, and then receiving it in your other program via stdin.

Consider the following example programs:

Code:

#include <iostream>

// play_out.cpp
//
int main(void)
{
  std::cout << "This is printed from play_out\n" << std::flush;
  return 0;
}

Code:

#include <iostream>

// play_in.cpp
//
int main(void)
{
  char play_input[256];

  std::cin.getline (play_input, 255);

  std::cout << "Received from stdin: " << play_input << "\n" << std::flush;
  return 0;
}

And when piping them:

Code:

~/cpp> ./play_out | ./play_in
Received from stdin: This is printed from play_out

I hope that I haven't over-simplified what you're trying to do. (=

pjz · 03-24-2005, 03:05 PM

TheLinuxDuck-

Unfortunately, I need a bunch of output from the first C++ program, so I use a tee and the perl filter. And, yes, the data exchange is bidirectional between the two C++ programs. I didn't try getline(), and that may make a difference.

Thanks to everyone for their time and consideration
pjz