LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-04-2016, 08:46 AM   #1
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 536

Rep: Reputation: Disabled
httpd dead but subsys locked


Hi, I upgraded PHP on my Centos 6 box, but now Apache is not running. I think one or more process is locked, but I don't know what to do about it. Please advise. Thank you
Code:
[root@vps run]# service httpd status
httpd dead but subsys locked
[root@vps run]# ls -l /var/run/*.pid
-rw-r--r-- 1 root root 6 Sep  4 07:33 /var/run/crond.pid
-rw-r--r-- 1 root root 5 Sep  4 07:32 /var/run/messagebus.pid
-rw-r--r-- 1 root root 5 Sep  4 07:32 /var/run/sshd.pid
-rw------- 1 root root 5 Sep  4 07:32 /var/run/syslogd.pid
[root@vps run]# ls -l /var/lock/subsys/*
-rw-r--r-- 1 root root 0 Sep  4 07:33 /var/lock/subsys/crond
-rw-r--r-- 1 root root 0 Sep  4 07:36 /var/lock/subsys/httpd
-rw-r--r-- 1 root root 0 Sep  4 07:32 /var/lock/subsys/iptables
-rw-r--r-- 1 root root 0 Sep  4 07:33 /var/lock/subsys/local
-rw-r--r-- 1 root root 0 Sep  4 07:32 /var/lock/subsys/messagebus
-rw-r--r-- 1 root root 0 Sep  4 07:32 /var/lock/subsys/mysqld
-rw-r--r-- 1 root root 0 Sep  4 07:32 /var/lock/subsys/network
-rw-r--r-- 1 root root 0 Sep  4 07:32 /var/lock/subsys/redis
-rw------- 1 root root 0 Sep  4 07:32 /var/lock/subsys/rsyslog
-rw-r--r-- 1 root root 0 Sep  4 07:33 /var/lock/subsys/sidekiq
-rw-r--r-- 1 root root 0 Sep  4 07:32 /var/lock/subsys/sshd
-rw-r--r-- 1 root root 0 Sep  4 07:33 /var/lock/subsys/unicorn
-rw-r--r-- 1 root root 0 Sep  4 07:32 /var/lock/subsys/vsftpd
-rw-r--r-- 1 root root 0 Sep  4 07:33 /var/lock/subsys/webmin
[root@vps run]# ls -l /var/run/*
---------- 1 root  root      0 Sep  4 07:33 /var/run/cron.reboot
-rw-r--r-- 1 root  root      6 Sep  4 07:33 /var/run/crond.pid
-rw-r--r-- 1 root  root      5 Sep  4 07:32 /var/run/messagebus.pid
-rw-r--r-- 1 root  root      5 Sep  4 07:32 /var/run/sshd.pid
-rw------- 1 root  root      5 Sep  4 07:32 /var/run/syslogd.pid
-rw-rw-r-- 1 root  utmp   1152 Sep  4 07:33 /var/run/utmp

/var/run/ConsoleKit:
total 0

/var/run/console:
total 0

/var/run/dbus:
total 0
srwxrwxrwx 1 root root 0 Sep  4 07:32 system_bus_socket

/var/run/faillock:
total 0

/var/run/httpd:
total 0

/var/run/mysqld:
total 4
-rw-rw---- 1 mysql mysql 6 Sep  4 07:32 mysqld.pid

/var/run/named:
total 0

/var/run/net-snmp:
total 0

/var/run/netreport:
total 0

/var/run/nscd:
total 0

/var/run/plymouth:
total 0

/var/run/portreserve:
total 0

/var/run/redis:
total 4
-rw-r--r-- 1 redis redis 5 Sep  4 07:32 redis.pid

/var/run/saslauthd:
total 0

/var/run/screen:
total 0

/var/run/sepermit:
total 0

/var/run/setrans:
total 0

/var/run/winbindd:
total 0
[root@vps run]# ls -l /var/run
total 92
drwxr-xr-x 2 root  root   4096 Nov 10  2010 ConsoleKit
drwxr-xr-x 2 root  root   4096 Aug 18  2015 console
---------- 1 root  root      0 Sep  4 07:33 cron.reboot
-rw-r--r-- 1 root  root      6 Sep  4 07:33 crond.pid
drwxr-xr-x 2 root  root   4096 Sep  4 07:32 dbus
drwxr-xr-x 2 root  root   4096 Aug 18  2015 faillock
drwx--x--- 2 root  apache 4096 Sep  4 07:32 httpd
-rw-r--r-- 1 root  root      5 Sep  4 07:32 messagebus.pid
drwxr-xr-x 2 mysql mysql  4096 Sep  4 07:32 mysqld
drwxr-xr-x 2 named named  4096 Mar 16 08:22 named
drwxr-xr-x 2 root  root   4096 Aug 17  2015 net-snmp
drwxrwxr-x 2 root  root   4096 May  3 14:21 netreport
drwxr-xr-x 2 root  root   4096 Feb 16  2016 nscd
drwxr-xr-x 2 root  root   4096 Aug 11  2014 plymouth
drwxr-xr-x 2 root  root   4096 Apr  3  2012 portreserve
drwxr-xr-x 2 redis root   4096 Sep  4 07:32 redis
drwxr-xr-x 2 root  root   4096 Mar 25  2015 saslauthd
drwxrwxr-x 2 root  screen 4096 Jul 23  2015 screen
drwxr-xr-x 2 root  root   4096 Aug 18  2015 sepermit
drwxr-xr-x 2 root  root   4096 Oct 15  2014 setrans
-rw-r--r-- 1 root  root      5 Sep  4 07:32 sshd.pid
-rw------- 1 root  root      5 Sep  4 07:32 syslogd.pid
-rw-rw-r-- 1 root  utmp   1152 Sep  4 07:33 utmp
drwxr-xr-x 2 root  root   4096 Apr 12 13:08 winbindd
[root@vps run]#
#

Last edited by NotionCommotion; 09-04-2016 at 08:48 AM.
 
Old 09-04-2016, 09:30 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,331
Blog Entries: 55

Rep: Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529
Quote:
Originally Posted by NotionCommotion View Post
Hi, I upgraded PHP on my Centos 6 box, but now Apache is not running. I think one or more process is locked, but I don't know what to do about it.
- check what packages got upgraded and if configs where merged or not,
- run apachectl and validate your configuration,
- check Apaches error log.
 
Old 09-04-2016, 09:47 AM   #3
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 536

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
- check what packages got upgraded and if configs where merged or not,
- run apachectl and validate your configuration,
- check Apaches error log.
Thanks unSpawn,

I went through the config files, and think they are all good. apachectl checks out.

Should have checked the error log before crying wolf. Looks like it might not be a locked process as I first thought, but a memory issue. Agree? Does the memory reports below shed any light?

EDIT. By the way, it is a virtual server. I did the same upgrade on a physical machine, and don't have any issues.

Thank you

Code:
[root@vps httpd]# apachectl configtest
Syntax OK
[root@vps httpd]# tail error_log
[Sun Sep 04 07:36:17 2016] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Sep 04 07:36:17 2016] [warn] RSA server certificate wildcard CommonName (CN) `*.example.com' does NOT match server name!?
[Sun Sep 04 07:36:17 2016] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Sun Sep 04 07:36:17 2016] [notice] Digest: generating secret for digest authentication ...
[Sun Sep 04 07:36:17 2016] [notice] Digest: done
Sun Sep  4 07:36:17 2016 (13844): Fatal Error Unable to allocate shared memory segment of 134217728 bytes: mmap: No space left on device (28)
[root@vps httpd]#
Code:
[root@vps httpd]# ipcs -lm

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 32768
max total shared memory (kbytes) = 8388608
min seg size (bytes) = 1

[root@vps httpd]# ipcs -ma

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0x00000000 0          root       600        1
0x00000000 32769      apache     600        1
0x00000000 65538      apache     600        1
0x00000000 98307      apache     600        1
0x00000000 131076     apache     600        1

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

[root@vps httpd]# cat /proc/user_beancounters
Version: 2.5
       uid  resource           held    maxheld    barrier      limit    failcnt
      108:  kmemsize        5892421   12492266   33554432   33554432        409
            lockedpages           0        511       2048       2048          0
            privvmpages      193397     348949 1000000000 1000000000          0
            shmpages           1954      25341      32768      32768         42
            dummy                 0          0          0          0          0
            numproc              86        121        400        400          0
            physpages         63867     215288          0 2147483647          0
            vmguarpages           0          0      33792 2147483647          0
            oomguarpages      63867     215289      26112 2147483647          0
            numtcpsock           15         57        360        360          0
            numflock              4         15        188        206          0
            numpty                1          1         16         16          0
            numsiginfo            0         13        256        256          0
            tcpsndbuf        246848    1770944    4194304    4194304       2325
            tcprcvbuf        245760    1949952    4194304    4194304          0
            othersockbuf      30656     176256    3145728    3145728         26
            dgramrcvbuf           0      16768     262144     262144          0
            numothersock         33         92        360        360          0
            dcachesize       289680     534040    3409920    3624960          0
            numfile            1815       2834       9312       9312          0
            dummy                 0          0          0          0          0
            dummy                 0          0          0          0          0
            dummy                 0          0          0          0          0
            numiptent            25         25        128        128          0
[root@vps httpd]#

Last edited by NotionCommotion; 09-04-2016 at 09:49 AM.
 
Old 09-04-2016, 10:18 AM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,331
Blog Entries: 55

Rep: Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529
Quote:
Originally Posted by NotionCommotion View Post
Looks like it might not be a locked process as I first thought, but a memory issue.
If the VM has enough RAM see if you can up the "kernel.shmmax" sysctl?
 
Old 09-04-2016, 10:25 AM   #5
NotionCommotion
Member
 
Registered: Aug 2012
Posts: 536

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
If the VM has enough RAM see if you can up the "kernel.shmmax" sysctl?
I don't understand. Please elaborate.
 
Old 09-04-2016, 03:37 PM   #6
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,331
Blog Entries: 55

Rep: Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529
Quote:
Originally Posted by NotionCommotion View Post
I don't understand. Please elaborate.
As root try 'sysctl -w kernel.shmmax=$[134217728+(134217728/2)]'? (If that works you also need to fixate in /etc/sysctl.conf.) The other way could be to check and lower memory_limit in php.ini. IIRC that could affect zend / opcache so you have to assess if it's worth it.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
httpd dead but subsys locked paleogryph Linux - Software 3 05-18-2016 11:37 AM
httpd dead but subsys locked NotionCommotion Linux - Newbie 6 05-16-2016 03:57 PM
HTTPD dead but subsys locked jsaravana87 Linux - Server 1 02-02-2012 10:54 AM
httpd dead but subsys locked x2000koh Linux - General 5 03-21-2005 06:26 PM
service httpd status, results in httpd dead but subsys locked squadja Red Hat 2 09-11-2004 11:31 PM


All times are GMT -5. The time now is 08:12 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration