LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 08-20-2015, 11:48 PM   #1
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Rep: Reputation: Disabled
Help with pipe server


I'm pretty new to linux. I've written a pipe server on Windows and have been researching pipes on linux. I thought I had a good enough understanding on how a pipe server would work and started coding one. Based on a result I just got I'm now wondering whether my understanding was wrong.

I've created a single fifo in a directory. My pipe server opens several file descriptors on this fifo, 128 to be exact, readonly and non-blocking. From some of the documentation I read I got the impression that a request to open for writing will be matched with a request to open for reading. My thinking then was that these 128 file descriptors would be matched with 128 open requests for writing.

I couldn't figure out how to get SIGIO working so I moved on to using select(). I added all 128 file descriptors to the read fd set and called select(). I then have a program that opens the fifo up for writing, just a single file descriptor, and writes some data to it. To my surprise the select in the pipe server returned but with all 128 file descriptors set. I was expecting just one file descriptor to be set. Can someone explain to me why it works this way or maybe how a pipe server should work in linux?

Thanks,
Nick

Last edited by nickdu; 08-21-2015 at 09:06 AM.
 
Old 08-21-2015, 03:35 AM   #2
fatmac
LQ Guru
 
Registered: Sep 2011
Location: Upper Hale, Surrey/Hants Border, UK
Distribution: Mainly Devuan, antiX, & Void, with Tiny Core, Fatdog, & BSD thrown in.
Posts: 5,484

Rep: Reputation: Disabled
Never heard of a pipe server, on unix like systems, anyone can create & use a pipe by using the | in a shell command.

E.G.
Code:
dmesg | grep DDR
will give
Code:
spdmem0 at iic0 addr 0x50: 2GB DDR3 SDRAM PC3-10600
spdmem1 at iic0 addr 0x52: 2GB DDR3 SDRAM PC3-10600
on my machine.
 
Old 08-21-2015, 05:56 AM   #3
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,862
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
Pipes in unix are uni-directional, so you cannot use them. Use unix-domain-sockets [local computer only] or TCP-sockets [network]. Read documentation of these: socket, bind, listen, accept, select/poll, read, write, close.
 
Old 08-21-2015, 06:53 AM   #4
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
Written long ago, but still correct I believe: Using PIPES For Interprocess Communications
 
Old 08-21-2015, 08:46 AM   #5
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by fatmac View Post
Never heard of a pipe server, on unix like systems, anyone can create & use a pipe by using the | in a shell command.
Sorry, I assumed it was somewhat of a more well known term. A pipe server is a server which provides some functionality and uses named pipes as an IPC mechanism.

The pipes you're referring to are unnamed pipes which the shell uses to connect two or more programs which know nothing about one another.

Thanks,
Nick
 
Old 08-21-2015, 08:48 AM   #6
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by NevemTeve View Post
Pipes in unix are uni-directional, so you cannot use them. Use unix-domain-sockets [local computer only] or TCP-sockets [network]. Read documentation of these: socket, bind, listen, accept, select/poll, read, write, close.
I'm only using them unidirectional. The server only reads out of one end and the clients only write to one end. I believe the linux implementation of pipes ends up using sockets. On Windows the implementation uses shared memory.

Thanks,
Nick

Last edited by nickdu; 08-21-2015 at 09:08 AM.
 
Old 08-21-2015, 09:03 AM   #7
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by rtmistler View Post
Written long ago, but still correct I believe: Using PIPES For Interprocess Communications
Yes, looks valid. However it seems geared towards two processes communicating with one another, which of course is a use case for what I'm trying to accomplish. However, the part I seem to be having trouble with is the server handling multiple concurrent connections. My impression was that I could issue multiple open() calls up front. For instance, if I want to handle 128 concurrent connections I would open the 128 file descriptors up front, all on the same fifo of course. Which is what I did. However, when a single client opens the fifo up and writes into it, the select() indicates all file descriptors are ready for reading. Two possible reasons came to mind:

1. Some docs indicate that any pipe which the client hasn't opened yet will be signaled because it will be in an EOF state so to speak. So I was thinking that maybe one file descriptor has data to read and the others will all indicate EOF. This doesn't sound reasonable as the select() didn't return until the client wrote into the fifo.

2. I can't open multiple file descriptors on the fifo up front. I can only open one file descriptor at a time waiting for a client to connect. Once a client connects I can open another file descriptor waiting for the next client connection. Basically, I can have only 1 outstanding unconnected file descriptor on a fifo. I'm hoping this is not the case.

Thanks,
Nick
 
Old 08-21-2015, 09:18 AM   #8
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
Quote:
Originally Posted by nickdu View Post
Yes, looks valid. However it seems geared towards two processes communicating with one another, which of course is a use case for what I'm trying to accomplish. However, the part I seem to be having trouble with is the server handling multiple concurrent connections. My impression was that I could issue multiple open() calls up front. For instance, if I want to handle 128 concurrent connections I would open the 128 file descriptors up front, all on the same fifo of course. Which is what I did. However, when a single client opens the fifo up and writes into it, the select() indicates all file descriptors are ready for reading. Two possible reasons came to mind:

1. Some docs indicate that any pipe which the client hasn't opened yet will be signaled because it will be in an EOF state so to speak. So I was thinking that maybe one file descriptor has data to read and the others will all indicate EOF. This doesn't sound reasonable as the select() didn't return until the client wrote into the fifo.

2. I can't open multiple file descriptors on the fifo up front. I can only open one file descriptor at a time waiting for a client to connect. Once a client connects I can open another file descriptor waiting for the next client connection. Basically, I can have only 1 outstanding unconnected file descriptor on a fifo. I'm hoping this is not the case.

Thanks,
Nick
I've used this technique to establish a central process and had about 5 or 6 child processes performing bi-directional pipe communications between each of them. See the references on the daemon as well as how to employ the select() call to check for information.

It's just a lot of code, hence why I created a set of helper functions to perform those tasks.

I wouldn't bother NOT having the PIPE solution architect so that you could not communicate in both directions even if you don't need it. You may find that you do need it. Having one extra descriptor open doesn't matter, at least at the scale I've chosen and even with a few hundred, I think the limit is more like 65535 where you'd have to start worrying about system limits and changing those.

Just be organized and look for places where you can abstract to functions for reuse of functionality.

The specific architecture I ended up with was:
  • Daemon spawns all processes and sets up the resources and monitors all children, also re-establishes a failed child after logging this information. The benefit here is the daemon knows all the descriptors necessary to perform this re-establishment.
  • One of the processes is actually the central master process where it talks from that one to many other processes, similar to what you're saying. They all use select() to check for new data.

Last edited by rtmistler; 08-21-2015 at 09:21 AM.
 
Old 08-21-2015, 10:02 AM   #9
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by rtmistler View Post
I've used this technique to establish a central process and had about 5 or 6 child processes performing bi-directional pipe communications between each of them. See the references on the daemon as well as how to employ the select() call to check for information.
I have looked over the code and I think what I'm trying to accomplish varies quite a bit from what you're doing in the code. Your solution makes use of unnamed pipes between, I'm guessing, your parent process and its child processes. Those pipes are static, so to speak. Once created, the code at one side continues to write into the write fd and reads from the read fd.

The named pipe server I'm trying to create will be continually opening new pipes between it and clients. They communicate for a short while, always in one direction, and then end their conversation, closing both ends of the pipe.

Here is the named pipe server code I have at the moment:

pipeserver.c

command to compile: gcc -pthread -std=gnu99 pipeserver.c -o pipeserver

Code:
#include <stdio.h>
#include <errno.h>
#include <signal.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/select.h>
#include <fcntl.h>
#include <string.h>
#include <unistd.h>
#include <pthread.h>

#define DimensionOf(rg) (sizeof(rg) / sizeof(rg[0]))

void* handleio(void* state)
{
  printf("handleio thread starting\n");
  int fds[128];
  
  for (int i = 0; i < DimensionOf(fds); ++i)
    {
    fds[i] = -1;
    }
  
  // Open the pipe instances.

  fd_set rfds;
  FD_ZERO(&rfds);
  for (int i = 0; i < DimensionOf(fds); ++i)
    {
      fds[i] = open((char*) state, O_RDONLY | O_NONBLOCK);
    if (fds[i] == -1)
      {
	printf("Failed opening pipe s, errno = %d\n", (char*) state, errno);
      goto done;
      }
    FD_SET(fds[i], &rfds);
    }

  while (1)
    {
      int ready = select(fds[DimensionOf(fds) - 1] + 1, &rfds, NULL,
	     NULL, NULL);
      if (ready == -1)
	{
	  printf("select() failed, error = %d\n", errno);
	  goto done;
	}
      printf("select return %d\n", ready);
      for (int i = 0; i < DimensionOf(fds); ++i)
	{
	if (FD_ISSET(fds[i], &rfds) != 0)
	  {
	  printf("fd %d is set", fds[i]);
	  }
	}
      break;
    }
done:
  printf("handleio thread returning\n");
}

void main(int argc, char* argv[])
{
  if (argc != 2)
    {
    printf("Expecting fifo path.\n");
    goto done;
    }

  // Block SIGINT.
  
  sigset_t sigset;
  sigemptyset(&sigset);
  sigaddset(&sigset, SIGINT);
  if (pthread_sigmask(SIG_BLOCK, &sigset, NULL) != 0)
    {
    printf("Failed setting sigmask\n");
    goto done;
    }

  // Start the thread which will handle the IO on the pipe instances.
  
  pthread_t iothread;
  if (pthread_create(&iothread, NULL, handleio, argv[1]) != 0)
    {
    printf("Failed creating IO thread, errno = %d\n", errno);
    goto done;
    }
  
  sigemptyset(&sigset);
  sigaddset(&sigset, SIGINT);
  int sig;
  int result = sigwait(&sigset, &sig);
  if (result == 0)
    printf("sigwait got signal: %d\n", sig);

done:
  ;
}
As I mentioned, the problem I'm having is that as soon as a client client opens a file descriptor up on the fifo and write into it, the select() returns with all 128 file descriptors set in the read fd set. I was expecting only a single file descriptor to be set in the read fd set. Note that the handleio thread ends as soon as the select returns. This is just this way while I debug the issue.

Here is the client code:

writepipe.c

command to compile: gcc writepipe.c -o writepipe

Code:
#include "stdio.h"
#include "errno.h"
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

void main(int argc, char* argv[])
{
  int fd;

  if (argc != 2)
    printf("Expecting fifo path.\n");
  else
    {
    fd = open(argv[1], O_WRONLY | O_NONBLOCK | O_ASYNC);
    if (fd == -1)
        {
	printf("Failed opening pipe %s, errno = %d\n", argv[1], errno);
	}
    else
	{
	char message[] = "This is a test";
	printf("Successfully opened pipe, fd = %d\n", fd);
	int count = sizeof(message) / sizeof(message[0]);
	int total = 0;
	while (total != count)
	  {
	    int written = write(fd, &message[total], count - total);
	    if (written == -1)
	      {
		printf("Failed writing to pipe, errno = %d\n", errno);
		break;
	      }
	    total += written;
	  }
	close(fd);
	}
    }
}
In order for the code to work you'll need to create the fifo using mkfifo and then pass the fifo to each of the programs.

Thanks,
Nick

Last edited by nickdu; 08-21-2015 at 10:05 AM.
 
Old 08-21-2015, 10:11 AM   #10
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Original Poster
Rep: Reputation: Disabled
Interestingly, when I removed the O_ASYNC on my client, writepipe.c, the results changed. I'm still getting multiple file descriptors as ready for reading, but not all 128. Here is the output:

Quote:
[nick@squiddy namedpipe]$ ./pipeserver businessEvents
handleio thread starting
select return 10
fd 121 is setfd 122 is setfd 123 is setfd 124 is setfd 125 is setfd 126 is setfd 127 is setfd 128 is setfd 129 is setfd 130 is sethandleio thread returning
with O_ASYNC set on the client, writepipe.c, I get:

Quote:
[nick@squiddy namedpipe]$ ./pipeserver businessEvents
handleio thread starting
select return 128
fd 3 is setfd 4 is setfd 5 is setfd 6 is setfd 7 is setfd 8 is setfd 9 is setfd 10 is setfd 11 is setfd 12 is setfd 13 is setfd 14 is setfd 15 is setfd 16 is setfd 17 is setfd 18 is setfd 19 is setfd 20 is setfd 21 is setfd 22 is setfd 23 is setfd 24 is setfd 25 is setfd 26 is setfd 27 is setfd 28 is setfd 29 is setfd 30 is setfd 31 is setfd 32 is setfd 33 is setfd 34 is setfd 35 is setfd 36 is setfd 37 is setfd 38 is setfd 39 is setfd 40 is setfd 41 is setfd 42 is setfd 43 is setfd 44 is setfd 45 is setfd 46 is setfd 47 is setfd 48 is setfd 49 is setfd 50 is setfd 51 is setfd 52 is setfd 53 is setfd 54 is setfd 55 is setfd 56 is setfd 57 is setfd 58 is setfd 59 is setfd 60 is setfd 61 is setfd 62 is setfd 63 is setfd 64 is setfd 65 is setfd 66 is setfd 67 is setfd 68 is setfd 69 is setfd 70 is setfd 71 is setfd 72 is setfd 73 is setfd 74 is setfd 75 is setfd 76 is setfd 77 is setfd 78 is setfd 79 is setfd 80 is setfd 81 is setfd 82 is setfd 83 is setfd 84 is setfd 85 is setfd 86 is setfd 87 is setfd 88 is setfd 89 is setfd 90 is setfd 91 is setfd 92 is setfd 93 is setfd 94 is setfd 95 is setfd 96 is setfd 97 is setfd 98 is setfd 99 is setfd 100 is setfd 101 is setfd 102 is setfd 103 is setfd 104 is setfd 105 is setfd 106 is setfd 107 is setfd 108 is setfd 109 is setfd 110 is setfd 111 is setfd 112 is setfd 113 is setfd 114 is setfd 115 is setfd 116 is setfd 117 is setfd 118 is setfd 119 is setfd 120 is setfd 121 is setfd 122 is setfd 123 is setfd 124 is setfd 125 is setfd 126 is setfd 127 is setfd 128 is setfd 129 is setfd 130 is sethandleio thread returning
Thanks,
Nick
 
Old 08-21-2015, 10:19 AM   #11
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by nickdu View Post
Interestingly, when I removed the O_ASYNC on my client, writepipe.c, the results changed. I'm still getting multiple file descriptors as ready for reading, but not all 128. Here is the output:


with O_ASYNC set on the client, writepipe.c, I get:


Thanks,
Nick
Ignore my last reply. It seems it varies, all but that one time I've been getting all 128 file descriptors set in the read set.

Thanks,
Nick

Last edited by nickdu; 08-21-2015 at 10:20 AM.
 
Old 08-21-2015, 11:29 AM   #12
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,862
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
There is no point in opening a pipe more than once. I don't think it makes sense in Windows, but certainly doesn't make sense in Unix.
 
Old 08-21-2015, 11:33 AM   #13
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,862
Blog Entries: 1

Rep: Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869Reputation: 1869
@OP: please read this: https://en.wikipedia.org/wiki/Named_pipe
especially the difference between Windows and Unix
 
Old 08-21-2015, 11:55 AM   #14
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by NevemTeve View Post
There is no point in opening a pipe more than once. I don't think it makes sense in Windows, but certainly doesn't make sense in Unix.
It certainly makes sense in Windows, that's what the instances parameter is to CreateNamedPipe. And then you issue multiple async ConnectNamedPipe() calls so that multiple clients can connect concurrently.

I would think the same applies to linux, othewise named pipes would seem somewhat useless.

Thanks,
Nick
 
Old 08-21-2015, 11:59 AM   #15
nickdu
Member
 
Registered: Aug 2015
Posts: 41

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by NevemTeve View Post
@OP: please read this: https://en.wikipedia.org/wiki/Named_pipe
especially the difference between Windows and Unix
I have read many articles already on named pipes on linux and the differences between linux named pipes and windows named pipes. Not sure if I read the specific article you mention above. Two of the differences pointed out in some of the articles I read were:

1. On windows named pipes are implemented using shared memory, on linux they are implemented using sockets.

2. On windows a single named pipe can be bidirectional, where the client on each end can can both read and write into the pipe. On linux named pipes are unidirectional. If you want a client to both read and write to a named pipe you need to open two different pipes.

Thanks,
Nick
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to ignore Pipe in Pipe delimited file? rohit_shinez Programming 29 08-13-2013 11:53 PM
[SOLVED] How to handle a broken pipe exception (SIGPIPE) in FIFO pipe? zyroot998 Programming 5 03-03-2011 08:10 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:02 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration