LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 08-28-2009, 11:48 AM   #1
sdeng
LQ Newbie
 
Registered: Aug 2009
Posts: 6

Rep: Reputation: 0
Question defunct process caused by cron job


Hi, Experts:
In my crontab, I have the following line:
05 7-16 * * 1-5 /home/scripts/check_mirror_server

The check_mirror_server script is the following:
What the script does --> ps -ef to check mirroring process, expect 4 processes. If less the 4, check which one is death and restart it (I only put one restart here)
#!/bin/bash
#
cd /home/scripts
lc=$(ps -ef | grep -cE '[0-9]{2}:[0-9]{2}:[0-9]{2} ./mirror o server2')
if ((lc != 4));
then
mail admin@mycompany.com -s "Redundent Server Has Problem, count $lc" < <(ps -ef | grep "mirror o server2")
ps -ef | grep -qE '[0-9]{2}:[0-9]{2}:[0-9]{2} ./mirror o server2 7300'
if [ $? = 1 ]
then
nohup /AppHome/rc8mirout
mail admin@mycompany.com -s "Redundent Server has been restarted" < /home/scripts/MirrorProMsg
fi
fi

The script itself works fine. But it created a defunct process after it restart the mirroring process
#ps -ef | grep 22979
root 22979 2307 0 07:05 ? 00:00:00 crond
root 22981 22979 0 07:05 ? 00:00:00 [check_mirror_se] <defunct>

Would anybody tell me where I did wrong in my script to cause this defunct process?
Thank you very much!

Last edited by sdeng; 08-28-2009 at 12:50 PM. Reason: add icon for question
 
Old 08-29-2009, 05:04 AM   #2
avalonit
Member
 
Registered: Jun 2008
Posts: 81

Rep: Reputation: 19
Does this happen only under cron or when you execute manually also?
 
Old 08-29-2009, 06:04 AM   #3
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Assuming you have tested from the command line then three things come to mind.

Firstly the environment set up by cron to run your script is not the same as the login environment. You can determine whether this is causing the problem by changing the seccond line of your script to
Code:
#!/bin/bash -l
That "l" is a letter. It tells bash to simulate the login process.

Secondly, if the script writes anything to stdout or stderr (these will normally appear on the terminal when running interactively) then that will break a cron job.

Thirdly the < < is weird in
Code:
mail admin@mycompany.com -s "Redundent Server Has Problem, count $lc" < <(ps -ef | grep "mirror o server2")
Here's how to mail output from ps -ef | grep "mirror o server2" without redirection
Code:
mail admin@mycompany.com -s "Redundent Server Has Problem, count $lc $(ps -ef | grep 'mirror o server2' 2>&1)"
Here the double quotes around mirror o server2 have been replaced with single quotes for simplicity and 2>&1 has been added to redirect any stderr (which would break the cron job) to stdout so it is mailed along with the stdout from ps -ef | grep 'mirror o server2'
 
Old 09-01-2009, 11:13 AM   #4
sdeng
LQ Newbie
 
Registered: Aug 2009
Posts: 6

Original Poster
Rep: Reputation: 0
The problem only happen under cron job.

I follow Catkin's suggestion:
1) add -l to this line ==> #!/bin/bash -l
2) I don't think I have any stdout or stderr from the script except this ==> "nohup /AppHome/rc8mirout", from vendor and I have no idea if it will have any stdout or stderr

3)The following line:
mail admin@mycompany.com -s "Redundent Server Has Problem, count $lc" < <(ps -ef | grep "mirror o server2")
The -s "Redundent Server Has Problem, count $lc" is supposed to go to subject line. The weird redirect < <(ps -ef | grep "mirror o server2") is suppsed go to the email body.

I could be wrong, but the following line seems everything go to the subject line.
mail admin@mycompany.com -s "Redundent Server Has Problem, count $lc $(ps -ef | grep 'mirror o server2' 2>&1)"

But I eliminated the weird redirect for now.

Finally, I add the >/dev/null 2>&1 to the cron job line
05 7-16 * * 1-5 /home/scripts/check_mirror_server >/dev/null 2>&1

Now, I run under test mood (comment out the "nohup /home/fsi/rc8mirout" and change the condition), and it seems working (no defunct process was created).

Thank you for all your help and your suggestions.
Thank you again.

Last edited by sdeng; 09-01-2009 at 11:17 AM.
 
Old 09-01-2009, 11:40 AM   #5
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Quote:
Originally Posted by sdeng View Post
I could be wrong, but the following line seems everything go to the subject line.
mail admin@mycompany.com -s "Redundent Server Has Problem, count $lc $(ps -ef | grep 'mirror o server2' 2>&1)"
Of course it does! The mail command wants the body of the mail from stdin. Sorry for my error. How did you solve it? If I'd realised I might have suggested
Code:
mail admin@mycompany.com -s "Redundent Server Has Problem, count $lc" <<EOF
$(ps -ef | grep 'mirror o server2' 2>&1)"
EOF
 
Old 09-02-2009, 11:06 AM   #6
sdeng
LQ Newbie
 
Registered: Aug 2009
Posts: 6

Original Poster
Rep: Reputation: 0
Catkin:
For testing purpose, I eliminated the weird redirect completely in case it cause problem first. After I tested it and made sure the cron job didn't create any defunct process, I put it back as it was, and it worked.

But I think your code is cleaner and look nicer. I will use your code instead. I didn't know where I got the idea of the weird redirect.

Thank you very much!
 
Old 09-02-2009, 11:26 AM   #7
sdeng
LQ Newbie
 
Registered: Aug 2009
Posts: 6

Original Poster
Rep: Reputation: 0
Catkin:

I tried your code, it gave me syntax error: unexpected end of file.
========
mail admin@mycompany.com -s "Redundant Server Has Problem, count $lc" <<EOF
$(ps -ef | grep 'mirror o server2' 2>&1)"
EOF
========
First, I think the double quotes are mis-matched. But I am not sure if I need to remove one or add one. I tried to add a double quote right after the first EOF, but got the same syntax error.

Any suggestion?
Thank you very much!
 
Old 09-02-2009, 12:25 PM   #8
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Quote:
Originally Posted by sdeng View Post
Catkin:

I tried your code, it gave me syntax error: unexpected end of file.
========
mail admin@mycompany.com -s "Redundant Server Has Problem, count $lc" <<EOF
$(ps -ef | grep 'mirror o server2' 2>&1)"
EOF
========
First, I think the double quotes are mis-matched. But I am not sure if I need to remove one or add one. I tried to add a double quote right after the first EOF, but got the same syntax error.

Any suggestion?
Thank you very much!
The double quotes are mis-matched. Add a double quote in front of $(

Intriguing that the "weird" redirect (< space <) works!
 
Old 09-02-2009, 02:57 PM   #9
sdeng
LQ Newbie
 
Registered: Aug 2009
Posts: 6

Original Poster
Rep: Reputation: 0
Catkin:
You really got me curious. The code you suggest is not working. I simplified it to the following:

#!/bin/bash -x
#
mail admin@mycompany.com -s "Redundant Server Has Problem, count $lc"
<<EOF
"$(ps -ef | grep 'mirror o server2' 2>&1)"
EOF

When I run it from the command line (the script called check_mirror), it stop after the mail command. I have to Ctrl/C couple of times, then the "ps -ef" appeared and finished.

#./check_mirror
+ cd /home/deng/scripts
+ mail admin@mycompany.com -s 'Redundant Server Has Problem, count '


(Interrupt -- one more to kill letter)
++ ps -ef
++ grep 'mirror o server2'

Any suggestions?
Thank you very much!
 
Old 09-02-2009, 03:08 PM   #10
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
I'm sorry you are having such a hard time and thank you for your patience (especially with my mistakes!).

You need this format, so the "here document" is fed into the mail command as if you had entered it from the keyboard
Code:
mail admin@mycompany.com -s "Redundant Server Has Problem, count $lc" <<EOF
 
Old 09-02-2009, 03:31 PM   #11
sdeng
LQ Newbie
 
Registered: Aug 2009
Posts: 6

Original Poster
Rep: Reputation: 0
After I stared the screen output for a while, I suddenly recognized the mail command is waiting for a input. As soon as I put the <<EOF in the same line with the mail command (just like what you suggested), the simplified script works. I am going to put it to the real script for testing.
Thank you very much!
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
adding a perl script to cron.daily / cron.d to setup a cron job CrontabNewBIE Linux - Software 6 01-14-2008 08:16 AM
Cron/Sendmail Defunct Processes Schiz0 Ubuntu 2 12-02-2007 07:43 PM
defunct processes in cron bujecas Debian 1 02-07-2006 01:59 PM
cron job <defunct> error? existo Slackware 7 01-21-2004 03:39 PM
cron job to restart process if dead gborrageiro Linux - General 3 09-25-2002 11:55 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 02:49 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration