For the past few weeks, I have been getting intermittent segfaults in httpd.
Afterwards I experience a massive amount of blocked processes in vmstat.
This makes the website become extremely unresponsive and I am forced to restart Apache. Occasionally the restart fails with an "unable to bind to port" error. Searching through the processlist shows an httpd process which is stuck and not shutdown. I manually have to kill this process to get Apache to restart back up.
I am currently running:
Fedora Core 3
Apache 2.0.53
MySQL 4.1.15
PHP 5.02
This is a fairly high traffic site with about 500,000 pageviews/day.
These are the segfaults which show in /var/log/messages. I don't get any errors in my apache weblogs that correspond to the segfaults. Here's an example:
Code:
Nov 30 03:06:48 www kernel: httpd[31811]: segfault at 00007fffff05e988 rip 0000555555574d2f rsp 00007fffff05e950 error 6
Nov 30 03:06:57 www kernel: httpd[31948]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:06:57 www kernel: httpd[31748]: segfault at 00007fffff05edc4 rip 00002aaaab43ef3e rsp 00007fffff05ec50 error 6
Nov 30 03:06:58 www kernel: httpd[31883]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:00 www kernel: httpd[31911]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:01 www kernel: httpd[32245]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:03 www kernel: httpd[31910]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:05 www kernel: httpd[31888]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:07 www kernel: httpd[31889]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:10 www kernel: httpd[31901]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:12 www kernel: httpd[31903]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:14 www kernel: httpd[31905]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:16 www kernel: httpd[31907]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:19 www kernel: httpd[31915]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Nov 30 03:07:21 www kernel: httpd[31918]: segfault at 00007fffff05efdc rip 00002aaaaabc832a rsp 00007fffff05ef60 error 6
Sorry, it's not much to go on, but any pointers would help me out immensely.
