LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 12-29-2016, 01:05 PM   #1
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939
I am ROOT, by Gawd, and I say "killall -9 httpd" ... and they DON'T!


The title says it all.

Yes, I used "-s" to see in advance exactly what PIDs would be affected. And yes, I even directed the signal directly at a few of them.

Not only did they "not die," but when I asked the operating system (note: "OS/X Lion, not Linux ...") to "reboot," it didn't.

I've gotta know why.

And, I've gotta know that "Linux can't possibly(!) do this."

Last edited by sundialsvcs; 12-29-2016 at 01:07 PM.
 
Old 12-29-2016, 01:21 PM   #2
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694
What does the 'ps aux' list look like? Do you happen to be running NFS on the box?

Last edited by szboardstretcher; 12-29-2016 at 01:23 PM.
 
Old 12-29-2016, 03:01 PM   #3
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659

Original Poster
Blog Entries: 4

Rep: Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939
No, this is a solitary box running OS/X Lion (BSD ...), with no NFS in sight.

What I observed is that, with only 7.5GB of 12.0GB physical RAM allocated, and with less than 2% CPU utilization, more than fifty HTTPD slave processes could not be killed with "-SIGKILL."

Furthermore, when we issued the "shutdown" command, apparently it never did so. (Apparently it never completely shut down, although it had teminated all SSH and/or VNC access by then.)

Gotta get to the bottom of this. Fast ...
 
Old 12-29-2016, 03:37 PM   #4
linux4evr5581
Member
 
Registered: Sep 2016
Location: USA
Posts: 275

Rep: Reputation: Disabled
Maybe you can try kill -9 -kill pid#.. I recently has something similar happen in a live cd where I couldnt delete a file in my home directory as root... Maybe it has something to do with an another program keeping an inode link open but, just a wild guess.. You could try ps -ax f to get a forest view of how daemons relate to eachother

Last edited by linux4evr5581; 12-29-2016 at 03:47 PM.
 
Old 12-30-2016, 01:52 AM   #5
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694
I was going to look at your 'ps aux' output and see whether you had some Uninteruptable Sleep/Zombie Processes running which will ignore kill -9. But I now realize that this is OS/X, not Linux (confused myself because of the "Linux can't possibly do this" statement). I am unfamiliar with Apple and I'm unsure whether they have UiS/Z processes or not and I'm unable to test.
 
Old 12-30-2016, 06:18 AM   #6
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 21 MATE
Posts: 8,048
Blog Entries: 5

Rep: Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925
Wait, just so I can understand, you're running OS/X, asking for help on a Linux forum (Linux - Server), and say "Linux can't possibly(!) do this."

Is there something I'm missing here?
 
Old 12-30-2016, 07:46 AM   #7
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659

Original Poster
Blog Entries: 4

Rep: Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939
Quote:
Originally Posted by hydrurga View Post
Wait, just so I can understand, you're running OS/X, asking for help on a Linux forum (Linux - Server), and say "Linux can't possibly(!) do this."

Is there something I'm missing here?
... because I reasonably expect that the two systems are probably similar enough that a Linux forum might have some insight. The "ps" flags certainly are. I'll check carefully to see if there are any zombies or unusual wait-codes for these processes, but basically I am flummoxed by encountering any Unix/Linux environment where "kill -9" was shrugged-off. I have never seen similar behavior, well, anywhere.

Kill by specific PID also had no effect. And, I mean, c'mon ... these are just perfectly ordinary Apache worker processes running "httpd" under a nobody-like uid/gid, with no blue body-suit or red cape in sight.

Last edited by sundialsvcs; 12-30-2016 at 07:49 AM.
 
Old 12-30-2016, 07:58 AM   #8
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 21 MATE
Posts: 8,048
Blog Entries: 5

Rep: Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925
Is there any possibility that the processes were terminated but very quickly respawned by whatever master process controls them?
 
Old 12-30-2016, 08:47 AM   #9
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694
It's entirely possible that kill -9 not work. A process has to 'accept' the kill. If it is unable to, because it is in Uninteruptable Sleep for example, then it will not honor the kill -9 signal. This happens often on NFS, to the point where they actually added a new process state code called something like 'nfs unavailable' to deal with it.

There is some explanation in the linux world if you just google for 'linux kill 9 not working'
 
Old 12-30-2016, 09:38 AM   #10
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 21 MATE
Posts: 8,048
Blog Entries: 5

Rep: Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925
How about the possibility that you're running out of mbufs, as referred to in one of the answers here: http://superuser.com/questions/37391...ll-screen-game
 
Old 01-01-2017, 06:26 AM   #11
linux4evr5581
Member
 
Registered: Sep 2016
Location: USA
Posts: 275

Rep: Reputation: Disabled
I just happen to come accross this:


In Scenario 1, we demonstrate an instance in which a process cannot be killed.

Scenario 1: Troubleshooting a Process That Does Not Respond to kill
A user begins rewinding a tape but realizes that the wrong tape is in the drive. The user tries to kill the job but must wait for the process to finish.

Why?

The mt command has made an ioctl call to the SCSI tape driver (st) and must wait for the driver to release the process back to user space so that use signals will be handled.

# mt -f /dev/st0 rewind
# ps -emo state,pid,ppid,pri,size,stime,time,comm,wchan | grep mt
D 9225 8916 24 112 20:46 00:00:00 mt wait_for_completion

[root@atlorca2 root]# kill -9 9225
[root@atlorca2 root]# echo $? # This produces the return code for the previous command. 0 = success
0
[root@atlorca2 root]# ps -elf | grep 9225
0 D root 9225 8916 0 24 0 - 112 wait_f 20:46 pts/1 00:00:00 mt -f /dev/st0

The mt command has entered a wait channel, and after the code returns from the driver, the signal will be processed.

Let's check the pending signals:

cat ../9225/status
Name: mt
State: D (disk sleep)
Tgid: 9225
Pid: 9225
PPid: 8916
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
FDSize: 256
Groups: 0 1 2 3 4 6 10
VmSize: 2800 kB
VmLck: 0 kB
VmRSS: 640 kB
VmData: 96 kB
VmStk: 16 kB
VmExe: 32 kB
VmLib: 2560 kB
SigPnd: 0000000000000100 <-- SigPnd is a bit mask which indicates the value of the pending signal. Each byte
accounts for 4 bits. In this case, the pending signal has a value of 9, so the first bit on the 3rd byte is set. This
algorithm is detailed in linux/fs/proc/array.c under the render_sigset_t() function. The following table
illustrates this function.

Signal : 1 2 3 4 . 5 6 7 8 . 9 10 11 12 . 13 14 15 16
bit value : 1 2 4 8 . 1 2 4 8 . 1 2 4 8 . 1 2 4 8

kill -3 yields bit mask 0000000000000004
kill -9 yields bit mask 0000000000000100

ShdPnd: 0000000000000100
SigBlk: 0000000000000000
SigIgn: 0000000000000000
SigCgt: 0000000000000000
CapInh: 0000000000000000
CapPrm: 00000000fffffeff
CapEff: 00000000fffffeff
 
Old 01-02-2017, 08:17 AM   #12
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659

Original Poster
Blog Entries: 4

Rep: Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939
Quote:
Originally Posted by hydrurga View Post
Is there any possibility that the processes were terminated but very quickly respawned by whatever master process controls them?
Good thought, but no: the PID does not change and there are no logs.
 
Old 01-16-2017, 05:37 PM   #13
ericson007
Member
 
Registered: Sep 2004
Location: Japan
Distribution: CentOS 7.1
Posts: 735

Rep: Reputation: 154Reputation: 154
I cannot contribute but in pure sundialsvcs style, your thread heading had me fold double. Well thought out. Lol.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Files owned with AD user don't get deleted by root. "Operation not permitted" error. minlinux Linux - Server 0 12-04-2015 11:45 PM
[SOLVED] Difference between installing Apache httpd from "source" and "yum" Iyyappan Linux - Server 9 03-24-2012 06:07 AM
I can't install VMware Tools because Setup is unable to find the "killall" program. corone Linux - General 1 07-22-2009 10:18 AM
Standard commands give "-bash: open: command not found" even in "su -" and "su root" mibo12 Linux - General 4 11-11-2007 10:18 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 08:33 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration