LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Red Hat
User Name
Password
Red Hat This forum is for the discussion of Red Hat Linux.

Notices

Reply
 
Search this Thread
Old 08-20-2010, 01:12 PM   #1
dafydd2277
Member
 
Registered: Mar 2010
Posts: 58

Rep: Reputation: 5
Init tries to respawn a running process


Good Morning,

This is on RHEL 5.4.

First clue: Three lines in /var/log/messages, Every 5 Minutes:

Code:
[Daemon name redacted]: Daemon started
last message repeated 9 times
init: Id "mp" respawning too fast: disabled for 5 minutes
Second clue: Run multiple ps commands. The daemon is running.

Code:
# ps -ef | grep [Redacted] | grep -v grep
root    21519     1  0 Jun08 ?  00:02:48 [Redacted]
Third clue: Re run the ps commands before and after one of these message triplets in /var/log/messages. The PID of the daemon does not change! Therefore, the process is not, in fact, being stopped and restarted.

Fourth clue: The relevant line in /etc/inittab:
Code:
mp:12345:respawn:/usr/sbin/[SubDir]/[Daemon]
Fifth clue: The execution line in the inittab entry is identical to the path and daemon name in the ps output's CMD field.

Problem: Why is init trying to respawn a running process? how do I tell it to stop making the inappropriate attempt, without removing the ability to restart the process when it ~does~ die?!

Thanks!
dafydd

PS: Occasionally, this daemon does die, which means we have to wait up to five minutes for it to respawn. In testing, the respawn is successful after the five minute wait, although I still get the "last message repeated 9 times" output with no decrease in the number of repeats. I have reports that the respawn fails, in that the daemon is never restarted. I've yet to recreate that problem.
 
Old 08-20-2010, 01:42 PM   #2
eSelix
Senior Member
 
Registered: Oct 2009
Location: Wroclaw, Poland
Distribution: Arch, Kubuntu
Posts: 1,195

Rep: Reputation: 301Reputation: 301Reputation: 301Reputation: 301
If pid hasn't changed then daemon wasn't restarted. You have time of working 00:02:48. Is this some standard daemon or yours? Maybe it fork or run another process and then quit, leaving child process (real daemon) working.
 
1 members found this post helpful.
Old 08-20-2010, 03:16 PM   #3
dafydd2277
Member
 
Registered: Mar 2010
Posts: 58

Original Poster
Rep: Reputation: 5
Hi, eSelix,

Turns out I got that same reply from one of the developers about half an hour before I saw this.

They've got a work around. I think I'll keep my "Workaround? Not a fix?" to myself.

dafydd
 
Old 08-10-2011, 10:41 AM   #4
dafydd2277
Member
 
Registered: Mar 2010
Posts: 58

Original Poster
Rep: Reputation: 5
Cool

I'm going back over my old questions, and realized I never posted a "What did it turn out to be?" reply.

What it turned out to be is that the process being spawned by init launches just long enough to fork a copy of itself and then die.

So, init doesn't see the PID of the process it launched, and tries to launch another. The new one starts just long enough to realize the self-named child is still running, and then kills itself. So, init tries again. Lather, rinse, repeat. So, every five minutes, init goes through this process about 10 times in a second, and throttles itself for another five minutes.

The workaround from a year ago was to keep the parent process running but empty. I believe subsequent versions since then have done away with the fork.

Cheers!
dafydd
 
  


Reply

Tags
init, linux


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
INIT"x"respawn too fast:disabled 5 minutes steveprakash Linux - Desktop 1 07-23-2009 02:23 AM
Is there any differences between ::respawn:-/bin/sh and ::respawn:/bin/sh? rainballdh Linux - Newbie 1 11-06-2008 10:17 AM
Shell Script : Kill a running process when another process starts ashmew2 Linux - General 3 08-20-2008 03:47 AM
JFFS2 [VFS mount root /dev/mtdblock2 ok] - kernel panic on running init process sud_vijay Linux - Kernel 0 03-05-2008 10:05 AM
How to automatically respawn a process? lucasbl3 Linux - General 2 06-11-2004 03:29 PM


All times are GMT -5. The time now is 01:03 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration