Perl child processes become zombie on a multi-core processor

kdelover · 04-11-2011, 02:51 PM

Hi,

I am learning about forking and IPC in perl.I have written a simple script which has to find required patterns from a bunch of files ( where each file is around 2 GB each,which contain the output of seq 1 10000000000000) on an 8 core machine.

I am current forking 6 child processes which run simultaneously on 6 cores of the processor & have to search for the required pattern in 6 different files & inform the parent process when a pattern is found using a PIPE.

The problem is,when a child process is done reading a text file looking for a pattern,it is becoming a zombie process.It exits cleanly when i put a $SIG{CHLD} = "IGNORE"; in the script.

Can any one tell me whats going on & how do i improve the communication between child and parent processes?Sorry for such a long question.Thanks.

Code:

#!/bin/perl
use strict;
use warnings;

my $pid;
my $result;
my $pat='28012';
my $patx='99';
my $paty='992';
my $patz='999';

#Pipe for reading and writing
pipe (READ,WRITE);

for my $x (1..6) {
    my $file="/local/a/tmp/test/file$x.txt";
    
    $pid=fork();

    if ($pid == 0 ) {

        #child process
        my $upid=$$;
        open(FH,'<',"$file");
        while (my $lol=<FH>) {
            my $time=localtime();
            $lol=~s/\n//;
            print WRITE ("found pattern-$lol in $file by process $upid at $time\n") if ($lol=~m/^($pat|$patx|$paty|$patz)$/);
        }
        close (FH);
        exit (0);
    }
    else {

        #parent process
        print "PARENT = $$,Forked Child = $pid\n";
        waitpid ($pid,1);

    } 
}
    # Parent keeps looping expecting a message from child process
    while (my $msg=<READ>) {
        print "$msg";
    }

wje_lq · 04-11-2011, 04:03 PM

You're doing a waitpid() within the loop, so it's done for each child process before starting the next one. There's nothing wrong with this, but it will keep the second child process from starting before the first one is done. I don't think you want this. Instead, you want to do six fork()s; the parent will keep an array of child process numbers for the eventual waitpid(), and will want to wait for data from all processes simultaneously.
Use a separate pipe for each child process, created just before the fork(). Rather than use READ and WRITE, you can have two arrays with six elements each, naming each one something like $read_side[x-1] or $write_side[x-1]. I use "-1" here, because you should ordinarily number array elements starting at zero, not one.
Very important: after the fork(), the parent should close the writing side, and the child should close the reading side.
Since you have a separate pipe for each child process, you're going to have to do select() or something to coordinate all that input.
When you've reached end of file on all six inputs in the parent, you can do waitpid() for each of the children. But if your parent process is going to exit almost immediately, there's no reason for this; the child processes will at that point completely dezombify out of existence.
There's no reason for the parent process to assign $file.

kdelover · 04-12-2011, 02:23 AM

Hi wje,

Quote:

There's nothing wrong with this, but it will keep the second child process from starting before the first one is done. I don't think you want this

I am doing waitpid($pid,1),so i have 6 processes running at a time & each process takes approx an hour to complete.If i put waitpid($pid,0),thats when 2nd process starts only after the 1st complete.

I'll do what ever you said in your post,thanks a lot for the input. But my Q was ,why child processes turned into a zombie process,once they are done? Is that because they were not able to update their exit status to the parent,as parent is stuck in a while loop ?

wje_lq · 04-12-2011, 04:01 AM

Quote:

Originally Posted by kdelover

I am doing waitpid($pid,1),so i have 6 processes running at a time

Oops. I didn't catch the "1". Sorry.

What that does is test, once, now, whether the process is complete. Since the processes last about an hour each, that call basically has no effect, because you're testing just when the processes have started.

What that call would do, if the process were actually done, is to harvest the completion status of the process. For example, if it died due to a signal (example: SIGTERM), what was the signal? If it exited normally, what was the exit code? And so on. Until you actually do that, you're going to have a zombie process. Zombie processes don't do anything; they just stick around until you can do the wait() or waitpid().

When the parent process gets end of file on a particular pipe, that would be a good time to do a waitpid(process number,0). Otherwise, you'll have zombie processes hanging around until the parent exits. When the parent exits, any unharvested zombie children get passed to the parent's parent, which in most cases means process 1 gets them almost immediately and waitpid()s them out of existence.