LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Reply
 
Search this Thread
Old 03-18-2009, 08:59 AM   #1
mdarwin
LQ Newbie
 
Registered: Mar 2009
Posts: 5

Rep: Reputation: 0
kill -9 not working


Hi guys,

I'm a newbie to Linux (but not POSIX), and I have the following problem:

There seem to be 32 instances of the "df" command running which have been hanging for nearly a month in some cases.
[root@ussd-apps2 root]# ps -ef |grep " df" |wc -l
33

I tried killing them all:
for proc in `ps -ef |grep " df" |grep -v grep |awk '{print $2}'`; do echo "killing ${proc}"; kill ${proc}; done

But it simply didn't work. No error message, and the processes are all still there.

I tried killing them individually:
root 14800 14397 0 14:49 pts/11 00:00:00 df -kh
root 14896 14397 0 15:10 pts/11 00:00:00 grep df
[root@ussd-apps2 root]# kill -9 14800
[root@ussd-apps2 root]# ps -ef |grep 14800
root 14800 14397 0 14:49 pts/11 00:00:00 df -kh
root 14898 14397 0 15:11 pts/11 00:00:00 grep 14800


I'm probably just being thick - I'm sure it's something simple I'm missing.

Can anyone help?

More info in case you need it:

[root@ussd-apps2 root]# uname -a
Linux ussd-apps2 2.4.21-9.ELsmp #1 SMP Thu Jan 8 17:08:56 EST 2004 i686 i686 i386 GNU/Linux

Matt
 
Old 03-18-2009, 09:45 AM   #2
kpraveen455
Member
 
Registered: Feb 2009
Location: Hyderabad
Distribution: fedora
Posts: 33

Rep: Reputation: 17
Hi,

I don't understand why you are following such a lengthy process to kill your applications

You can use 'pkill' command to kill all your `df' applications

pkill -09 <application name>

Example: pkill -09 df

or Else kill all the processes with command "killall" (see more for man page)
killall <signal name> <process name>

Example: killall -s KILL df
 
Old 03-18-2009, 10:07 AM   #3
mdarwin
LQ Newbie
 
Registered: Mar 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hi there kpraveen,

Thanks for your help.

Neither of the commands seem to work:

[root@ussd-apps2 root]# pkill -9 df
[root@ussd-apps2 root]# killall -s KILL df
[root@ussd-apps2 root]# ps -ef |grep df |head
root 18413 18311 0 Feb23 ? 00:00:00 df -kh
root 18803 18802 0 Feb24 ? 00:00:00 df -h
cvs 19647 1 0 Feb24 ? 00:00:00 df -kh
root 20611 20610 0 Feb25 ? 00:00:00 df -h
root 21246 21245 0 Feb26 ? 00:00:00 df -h
root 21969 21968 0 Feb27 ? 00:00:00 df -h

Any thoughts?

Matt
 
Old 03-18-2009, 10:27 AM   #4
openSauce
Member
 
Registered: Oct 2007
Distribution: Fedora, openSUSE
Posts: 252

Rep: Reputation: 39
Could they be zombie processes?

http://www.linux-mag.com/id/5707
Quote:
Even an “uncatchable” signal may not terminate a process. For instance, a zombie is the remnant of a process that’s waiting to exit. (A zombie has Z in the ps STAT column.) A process that’s being traced can also be unkillable.
Unfortunately that article doesn't tell you what you can do about zombies, if anything.
 
Old 03-18-2009, 10:40 AM   #5
bitpicker
Member
 
Registered: Jul 2003
Location: Germany
Distribution: Xubuntu, Ubuntu
Posts: 416
Blog Entries: 14

Rep: Reputation: 35
root 18413 18311 0 Feb23 ? 00:00:00 df -kh
root 18803 18802 0 Feb24 ? 00:00:00 df -h
cvs 19647 1 0 Feb24 ? 00:00:00 df -kh
root 20611 20610 0 Feb25 ? 00:00:00 df -h
root 21246 21245 0 Feb26 ? 00:00:00 df -h
root 21969 21968 0 Feb27 ? 00:00:00 df -h


You should find out what those parent processes are (like 21968 in the case of the last one). These processes are not waiting for their spawned df processes.

Maybe it helps to visualize what is going on if you install and use the program htop and use it to display the process structure as a tree. It will update the situation regularly and you can see what those processes spawning all those instances of df are. Normally df is too fast to even learn its process number, it just prints its results then exits. I suppose there is a badly written script or something which spawns these zombie processes.

Robin
 
Old 03-18-2009, 10:46 AM   #6
mdarwin
LQ Newbie
 
Registered: Mar 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Ok so now I have a bit more info:

Code:
[root@ussd-apps2 root]# ps -axl |grep " df" |head
F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME COMMAND
0     0 18413 18311  15   0  3580  508 end    D    ?          0:00 df -kh
0     0 18803 18802  25   0  3772  456 end    D    ?          0:00 df -h
0   506 19647     1  15   0  3580  508 end    D    ?          0:00 df -kh
0     0 20611 20610  25   0  3792  456 end    D    ?          0:00 df -h
0     0 21246 21245  25   0  3780  456 end    D    ?          0:00 df -h
0     0 21969 21968  25   0  3772  456 end    D    ?          0:00 df -h
0     0 25745 25744  25   0  3796  456 end    D    ?          0:00 df -h
0     0 26112 26111  25   0  3772  456 end    D    ?          0:00 df -h
0     0  1036  1035  25   0  3784  456 end    D    ?          0:00 df -h
0     0  3502  3501  25   0  3784  456 end    D    ?          0:00 df -h

From the same site (http://www.slackbook.org/html/process-control-ps.html)
Quote:
D stands for a process that has entered an uninterruptible sleep. Often, these processes refuse to die even when passed a SIGKILL.
Ok so what now?

I was thinking that it may have something to do with nfs mounts. When I run the df command I do get some output - it hangs when it gets to the nfs mount.


[root@ussd-apps2 root]# df -kh &
[1] 15292
[root@ussd-apps2 root]# Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 99G 64G 31G 68% /
/dev/cciss/c0d0p1 97M 15M 77M 17% /boot
none 881M 0 881M 0% /dev/shm


So maybe I could try turning off the mountd or something?

Last edited by mdarwin; 03-18-2009 at 10:51 AM. Reason: text layout not right
 
Old 03-18-2009, 10:49 AM   #7
mdarwin
LQ Newbie
 
Registered: Mar 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hi bitpicker,

There is no script spawning these processes - it's curious people like me who want to find out what the disk usage is and run df -kh. So even if I can kill these processes I'll still create a new one every time I run df -kh. I suppose I should try and find out the reason the df command is hanging - as per my last post, I think it's something to do with nfs....
 
Old 03-18-2009, 10:56 AM   #8
bitpicker
Member
 
Registered: Jul 2003
Location: Germany
Distribution: Xubuntu, Ubuntu
Posts: 416
Blog Entries: 14

Rep: Reputation: 35
Have you started all those df instances yourself or is there an automatism? All those ppids look as if they got created just to spawn a df instance. There must be a reason why df cannot read the nfs mount, so there's something wrong there.

Robin
 
Old 03-18-2009, 10:57 AM   #9
bitpicker
Member
 
Registered: Jul 2003
Location: Germany
Distribution: Xubuntu, Ubuntu
Posts: 416
Blog Entries: 14

Rep: Reputation: 35
Never mind the previous post, I wrote it while you were replying.

Robin
 
Old 03-18-2009, 11:06 AM   #10
mdarwin
LQ Newbie
 
Registered: Mar 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Ok I found the problem and fixed it:
Code:
[root@ussd-apps2 root]# grep nfs /etc/fstab 
10.113.97.24:/var/SP    /nfs             nfs    rsize=8192,wsize=8192,timeo=14,intr
[root@ussd-apps2 root]# umount -f /nfs
umount2: Device or resource busy
umount: /nfs: device is busy
[root@ussd-apps2 root]#
[root@ussd-apps2 root]# ps -axl |grep " df" |wc -l
      1
Looks like the nfs mount was hanging.

I found some useful info here:
http://linuxgazette.net/issue83/tag/6.html:
Quote:
If the NFS server (or the network connection thereto) becomes unavailable all processes that try to access any part of that share will be set into D state. (Use intr or soft mount options on NFS to avoid all that).
Thanks for your help and note the solution for future reference!
 
Old 03-18-2009, 12:38 PM   #11
AlucardZero
Senior Member
 
Registered: May 2006
Location: USA
Distribution: Debian
Posts: 4,653

Rep: Reputation: 536Reputation: 536Reputation: 536Reputation: 536Reputation: 536Reputation: 536
Yep, a little more info, state "D" is uninterruptible sleep. In that state a program is off in a driver call and cannot be interrupted even by kill -9. Fixing or removing the NFS mount, or mounting with different options, solves that as you saw.
 
  


Reply

Tags
kill, pid


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Kill a hung task when kill -9 doesn't help 10110111 Linux - General 4 04-02-2009 12:10 PM
Kill a program not working HarryBoy Linux - Newbie 3 12-09-2008 09:23 AM
kill -9 the unkillable process... isn't working ender42 Ubuntu 1 02-18-2006 02:33 AM
how to use kill to kill a batch of processes with same name? dr_zayus69 Linux - Software 2 09-03-2005 07:35 PM
kill/killall not working (?) didi156 Linux - Software 2 05-21-2004 12:47 AM


All times are GMT -5. The time now is 04:44 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration