[C] stdout and stdin replace by pipes and execve the child

chudzielec · 01-24-2008, 02:45 PM

I've managed to create pipe between two processes. I can make parent's stout child's stdin but I cannot send EOF to child.

Here is the code
Edit: looks like rafb.net have forgotten about my code, so I've used wklej.org (but tabs are gone)
http://wklej.org/id/ca1f2b983b

My target is to make progress bar for tar application without knowing what is inside the archive file. I only know its name and size. The solution is to read tar archive byte after byte and send read values as stdin to tar. In simply way it can be done by that way:

cat archive.tar | tar xz

By replacing cat with application that writes down how much bytes was read (writing to tar stdin actually blocks until tar unpacks sent data) I can make a progress bar. The reason why I'm not doing it using bash is that I'm writing library that shouldn't depend on shell or create and execute unnecessary script files.

nadroj · 01-24-2008, 11:25 PM

not sure if this helps, but this is from man execve (which is the 'back-end' to exec*):

Quote:

...
execve() does not return on success, and the text, data, bss, and stack
of the calling process are overwritten by that of the program loaded.
The program invoked inherits the calling process's PID, and any open
file descriptors that are not set to close on exec. ...

maybe when you exec, it uses actually stdin and not the file descriptor you used dup2 with?

have you tried using named pipes (see man fifo)? the advantage of FIFOs is that any programs on the same host can communicate, that is, without needing a parent-child relationship. i would guess that a call to exec* is destroying this link.

osor · 01-26-2008, 10:37 AM

I don’t know why, but your hyperlink is broken, and we can’t see the code.

Usually, to send the child an EOF, you would close the “write-end” of the pipe (and the next time the child tries to read from it, it will get EOF). If you close the “read-end” of the pipe first, then the parent is send SIGPIPE.

Also, the write() function should only block when the pipe is full (on linux a full pipe means 64kB pending for the “read-end”).

Btw nadroj, the exec function should keep the file descriptor open (that is the point).

chudzielec · 01-26-2008, 04:21 PM

@osor: new hyperlink but with broken indent http://wklej.org/id/ca1f2b983b

I disabled the buffering in my code.

@nadjor: I'm pretty sure that file descriptor are kept (at least the ones that describes stdin, stdout and stderr). Try to execute my code without exec line or try to replace exec with /bin/echo ( `echo "hello" | echo` prints nothing ). cat application displays what it gets so it looks good. I will check named pipes and report any progress. Thanks for advice.

osor · 01-26-2008, 05:20 PM

Quote:

Originally Posted by chudzielec

@osor: new hyperlink but with broken indent http://wklej.org/id/ca1f2b983b

I disabled the buffering in my code.

I see the problem:

Code:

if( pid == 0 ) {
// I am child

close(fd[1]);

// connect pipe output to stdin
dup2( fd[0], 0 );

close(fd[0]);

// loading image of cat application
execl("/bin/cat", "cat", NULL);
 
// because of exec* will never gets here and below
// so this is the end of child code

} else {

The line in red must be inserted for your problem to go away. The reason is that the right before the exec, the child has these file descriptors open:

Code:

0  — a copy of the read end of the pipe
1  — the terminal (normal stdout)
2  — stderr (probably also the terminal)
3* — read end of the pipe
4* — write end of the pipe

*I base this on the fact that fd[0] should be 3 and fd[1] should be 4

So when you do the exec, the stdin and stdout work as expected. But when you close the write end of the pipe in the parent, you forget that there is a copy of it still open in the child. So when the child does a read(), then call will block, waiting for the child to write to the pipe (which is never going to happen). So the correct thing to do is to close the write end of the pipe as soon as you get into the child process. Basically, the reading the read end of a pipe will return EOF when and only when all copies of the write end of the pipe are closed. In your case, you had left one copy open.

As for the line in blue, it is not strictly necessary, but it is good style to close a file descriptor you don’t need. You can also insert a similar line after the other call to dup2 (on line 33). In a real-world situation, you would check that the return values of each of those system calls (including pipe(), fork(), and dup2()) is not -1 before continuing.

osor · 01-26-2008, 05:31 PM

Btw, have you seen the program bar?

chudzielec · 01-27-2008, 05:52 AM

@osor: Thanks a lot! Works great. About that bar application, I'm writing a library so progress will be returned as callback.