Running external program using C and read the result output

Kunsheng · 11-14-2009, 03:04 PM

I am running some external executable inside C and getting the output from the executable using pipe like below:

1. create pipe

2. fork a child process to execute the program

3. parent reads from pipe

I know in Perl there is something pretty easy like:

my $result = qx (command);

Scalar $result will contain all std output from the command, just wondering if C has the same thing ? A pipe reads is working but seems too much work ?

ta0kira · 11-14-2009, 03:38 PM

Before anyone suggests it, don't use popen. You'll need to fork at the very least so that you can replace standard out for the command to be executed. Look at the manpages for pipe, dup2, fork, system, and execv. I haven't told you how to set it up because I'm not quite sure the context; e.g. is it a one-time thing, do you need continued communication with the command, does your program need to do something else in the meantime, does your program need to wait until the other command finishes, does your program need to kill the command when it exits, does the command need terminal access, is it merely a command name and arguments or does it have shell operators, does it need to run under the same user or a different one? I'm sure I've forgotten something.
Kevin Barry

Hko · 11-15-2009, 03:05 PM

Quote:

Originally Posted by ta0kira

Before anyone suggests it, don't use popen.

Why's that?
It would be the easiest way to do it, and would be closest to the perl example the OP mentioned.

smeezekitty · 11-15-2009, 04:41 PM

what about

Code:

#include <stdlib.h>
#include <stdio.h>
int main(){
char c;
system("whoami > $temp$.###");
FILE *f = fopen("$temp$.###", "r");
while(!feof(f)){c = fgetc(f); printf("%c", c);}
fclose(f);
remove ("$temp%.###");
}

Kunsheng · 11-15-2009, 04:44 PM

I got confused about that too... was pipe a mostly used way (if not only) for that ? I guess it might also be the low-level behaviors behind that perl 'qw' command ?

Quote:

Originally Posted by Hko

Why's that?
It would be the easiest way to do it, and would be closest to the perl example the OP mentioned.

ta0kira · 11-15-2009, 05:02 PM

Quote:

Originally Posted by Hko

Why's that?
It would be the easiest way to do it, and would be closest to the perl example the OP mentioned.

I'm not dead-set against it, but you don't have control over the process when you use it (especially the pipe direction you didn't choose,) and I've seen implementations that didn't account for already having a lot of files open.
Kevin Barry

ta0kira · 11-15-2009, 05:06 PM

Quote:

Originally Posted by Kunsheng

I got confused about that too... was pipe a mostly used way (if not only) for that ? I guess it might also be the low-level behaviors behind that perl 'qw' command ?

pipe is usually used to create a descriptor pair before forking. After forking, dup2 is often used to replace standard input or output with the pipe, followed by system or execv, allowing the parent to communicate with the child using its standard input and output.
Kevin Barry

SaintDanBert · 11-15-2009, 06:40 PM

Quote:

Originally Posted by smeezekitty

what about

Code:

#include <stdlib.h>
#include <stdio.h>
int main(){
char c;
system("whoami > $temp$.###");
FILE *f = fopen("$temp$.###", "r");
while(!feof(f)){c = fgetc(f); printf("%c", c);}
fclose(f);
remove ("$temp%.###");
}

The system( ) function does not return until there is an exit from the
command string. This is why folks have mentioned fork() and execv() --
both of which will return to the caller before the command exits.

Also, Just because the command passed to system(...) does a write does not mean that a file is visible to the subsequent fopen(...). As I recall, you need open+write+close before a following fopen will see a file that exists. An alternative requires that there is enough written to force a buffer flush, or an explicit call to flush() or similar in the code.

It has been a long time since I coded at this detail level, so I'm sure there are things forgotten. I hope these comments give you helpful things to think about.

~~~ 0;-Dan

paulsm4 · 11-15-2009, 08:22 PM

Hi, Kunsheng -

The called "popen()/fgets ()/pclose()" are probably the ideal solution to your problem.

No tool is right for every job - but I'm frankly not aware of any issues or problems of any kind with "popen()". Certainly none that you wouldn't ALSO face if you did the "fork()/exec()/pipe()/etc" all yourself.

IMHO .. PSM

Kunsheng · 11-16-2009, 01:26 PM

Thanks, folks. I checked that a little bit, popen is invoking a new shell to run the command, which is beaten by fork+pipe functions in performance.

It is something similar to difference between system() and exec family.

bigearsbilly · 11-16-2009, 02:17 PM

funnily enough I've been playing with this today,
a function helper that runs like execlp(3)
after which you can read the output from stdin.
(not tidied up yet)

Code:

#include <stdarg.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <limits.h>
#include <pthread.h>


pid_t helper(char * prog, ...) {

    char * args[ARG_MAX];
    int n = 0;
    va_list ap;
    char * p = prog;
    int filedes[2];
    pid_t pid;


    args[n++] = (char *) prog;
    va_start(ap, prog);
    while (p) {

	p = va_arg(ap, char *);
	args[n++] = p;
	//printf("got:%s\n", p);
    }
    va_end(ap);

    /* args done */

    if (-1 == pipe(filedes)) {
	perror("pipe failed");
	return 0;
    }

    switch (pid = fork()) {

    case -1:
	perror("fork failed");
	pid = 0;
	break;

    case 0:   /* child */
	close(1);
	dup(filedes[1]);
	close(filedes[1]);
	execvp(prog, args);
	perror(prog);
	break;

    default: /* parent */
	close(0);
	dup(filedes[0]);
	close(filedes[0]);
	close(filedes[1]);
	break;
	

    }

    return pid;


}




#define SZ 4096
int main (int argc, char ** argv) {


    char  buffer[SZ];
    pid_t pid;
    pthread_t thr[1];


    pid = helper ("cksum", "1", "2", "3", (char *) 0 );
    printf("started %d\n", pid);
    while(fgets(buffer, SZ, stdin) ) {

	printf(":%s", buffer);
    }


   return 0;
}

smeezekitty · 11-16-2009, 07:23 PM

Quote:

Originally Posted by SaintDanBert

The system( ) function does not return until there is an exit from the
command string. This is why folks have mentioned fork() and execv() --
both of which will return to the caller before the command exits.

Also, Just because the command passed to system(...) does a write does not mean that a file is visible to the subsequent fopen(...). As I recall, you need open+write+close before a following fopen will see a file that exists. An alternative requires that there is enough written to force a buffer flush, or an explicit call to flush() or similar in the code.

It has been a long time since I coded at this detail level, so I'm sure there are things forgotten. I hope these comments give you helpful things to think about.

~~~ 0;-Dan

But maybe its better to wait for the child to finish before your read the output?

theNbomr · 11-17-2009, 11:08 AM

Quote:

Originally Posted by smeezekitty

But maybe its better to wait for the child to finish before your read the output?

Better depends on the context. It sounds like the OP wants his parent process to interact with the child. That is the whole point of having popen(). The downside of popen() versus pipe()+fork()+exec() is that popen() launches a shell, which may be useless.
A couple of websites provide useful samples and tutorials for contrast:

UNIX System Calls and Subroutines using C
Beej's Guide to UNIX IPC

--- rod.

ta0kira · 11-17-2009, 12:03 PM

Quote:

Originally Posted by smeezekitty

But maybe its better to wait for the child to finish before your read the output?

If too much output is sent without reading, the next write will block. That might make the processes deadlock waiting for each other.
Kevin Barry

tuxdev · 11-17-2009, 12:13 PM

Also pipes are a lot faster than possibly touching the disk, and you don't have an annoying race condition to deal with.