Cron job zombie

dpriz · 04-26-2016, 09:06 AM

Hi all,

I'm new to the forum and looking for some help. This script, running as a cron job is creating a zombie process when it executes the startup/shutdown commands. Can someone help me to understand why?

Code:

#!/bin/bash

#first see if the processes are running
if  pgrep "java" > /dev/null
then
    echo "INFORMATION: Java container is running" >> ./container_util.log
    exit 0
#if not running restart the container
else
    echo "INFORMATION: Java container is not running" >> ./container_util.log
#Get the container name
    container=$(ls /path/to/file | grep nd_)
    echo $container
    echo "INFORMATION: retrieved the container name, $container" #>>.container_\
util.log
#Shutdown the container
    comm="/path/to/ShutdownScript.sh $container"
    echo "INFORMATION: Command being sent $comm"  #>> ./container_util.log
    eval $comm
    sleep 3
#Startup the container
    comm="/path/to/StartupScript.sh $container -bg"                    echo   eval $comm
    echo "INFORMATION: Command being sent $comm"  #>> ./container_util.log
#confirm the container is running
    sleep 3
fi

#Final check whther the container is running
if pgrep "java" #>>./container_util.log
then
    echo "SUCCESS: Java container is running" #>> ./container_util.log
    exit 0
else
 echo "ERROR: Restart failed!"
    exit 1
fi

I get the following the logs:

Sun Apr 24 01:30:02 EDT 2016 INFORMATION: Shutdown command being sent
Sun Apr 24 01:30:05 EDT 2016 INFORMATION: Startup command being sent
Sun Apr 24 01:30:08 EDT 2016 SUCCESS: Java container is running
Sun Apr 24 02:00:01 EDT 2016 INFORMATION: Java container is running
Sun Apr 24 02:30:01 EDT 2016 INFORMATION: Java container is running
Sun Apr 24 03:00:02 EDT 2016 INFORMATION: Java container is running

Right after the success, it seems to create a zombie process, and I can't understand why this is. Any help the group can offer would be awesome.

Thanks,
Dave

keefaz · 04-27-2016, 04:26 AM

Where is the zombie process? It seems your script runs properly.

X-LFS-2010 · 05-02-2016, 01:28 PM

yes, show your "ps ax", show us the zombie

let's assume if cron script causes error you might get strange effects (though that may be wrong)

i beleive your script can "exit 1" without hitting the exit 1

if any statement is false, exit is 1

so if and if is UNTRUE, and control flow hits bottom: your exit 1.

your script can exit 1 without ever hitting your intended exit 1, i think

using "if" questionbly to plow through the script you should have "exit 0" at the end of the script, so that only your "exit 1" statement can "exit 1"

you'll frequently see in other sh scripts "exit 0" or "return 0" at the end, for the same reason

X-LFS-2010 · 05-02-2016, 01:30 PM

another possible (i'm unfamiliar with your container thing)

is if you are starting a 2nd container which doesnt like running when one is already running

i can't tell from where i sit if that might be occuring

X-LFS-2010 · 05-02-2016, 01:32 PM

comm="/path/to/StartupScript.sh $container -bg"

-bg i assume is background

i'm unsure you understand how cron deals with PID children and & (background); (how cron deals with children and how to "really detatch a child" from cron). yes use of & improperly can cause zombies using cron to show in ps(1) listing. see cron manual and google issues about cron backgrounding and PID and issues like "setsid(1)".

fiddle with that

sundialsvcs · 05-03-2016, 07:43 AM

It might not be a good idea to trust such matters to "simple cron," because this daemon merely responds to a timer ... and does not know what it has done in the past.

Let's say that you launch a "background" activity every five minutes, that takes ten minutes to complete. Every five minutes, a new instance is added, and there are two instances active, but cron doesn't know this. Furthermore, if any of the instances "run long," e.g. because of interference caused by the fact that there are two-or-more instances of itself running simultaneously

(which never happened in "testing" [sic] on the developer's super-fast machine ...), the instances can start to pile-up.

You might need to use a more-sophisticated daemon: a true batch-execution monitor or distributed processing monitor (e.g. OpenStack, or any one of several batch-job monitoring systems), which are aware of what's going on at all times, and which can schedule work according to an established "work-flow" graph.

cron is a ubiquitous daemon, but it is also a simple one which dates from the 1970's. (Which, [koff, koff ...] is not such a bad thing, really.)

TenTenths · 05-03-2016, 08:02 AM

Off Topic: "Cron Job Zombie" would make an awesome band name!