LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 11-08-2009, 11:49 AM   #1
ventas
LQ Newbie
 
Registered: Apr 2009
Posts: 8

Rep: Reputation: 0
Apache segmentation fault


Hi,
my webserver coredumps on high load, my guess is that the epoll.max_user_instances is too low, but I cannot increase it, can you please help..

root@www4:~# echo "fs.epoll.max_user_instances=4092" >> /etc/sysctl.conf
root@www4:~# sysctl -p
error: "fs.epoll.max_user_instances" is an unknown key


root@www4:~# uname -a
Linux www4 2.6.28-16-server #55-Ubuntu SMP Tue Oct 20 20:37:10 UTC 2009 x86_64 GNU/Linux


Thanks!!
 
Old 11-08-2009, 02:55 PM   #2
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
No need to guess. Let's see some relevant sections from your error log.
 
Old 11-08-2009, 04:14 PM   #3
ventas
LQ Newbie
 
Registered: Apr 2009
Posts: 8

Original Poster
Rep: Reputation: 0
# gdb /usr/local/apache2/bin/httpd core.25582
GNU gdb Fedora (6.8-27.el5)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...


warning: Can't read pathname for load map: Input/output error.
Reading symbols from /usr/lib64/libz.so.1...done.
Loaded symbols for /usr/lib64/libz.so.1
Reading symbols from /lib64/libm.so.6...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /var/www/html/usr/local/apache2/lib/libaprutil-1.so.0...done.
Loaded symbols for /usr/local/apache2/lib/libaprutil-1.so.0
Reading symbols from /var/www/html/usr/local/apache2/lib/libexpat.so.0...done.
Loaded symbols for /usr/local/apache2/lib/libexpat.so.0
Reading symbols from /var/www/html/usr/local/apache2/lib/libapr-1.so.0...done.
Loaded symbols for /usr/local/apache2/lib/libapr-1.so.0
Reading symbols from /lib64/libuuid.so.1...done.
Loaded symbols for /lib64/libuuid.so.1
Reading symbols from /lib64/librt.so.1...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libcrypt.so.1...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libpthread.so.0...done.
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libnss_files.so.2...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /var/www/html/usr/local/apache2/modules/mod_proxy.so...done.
Loaded symbols for /usr/local/apache2/modules/mod_proxy.so
Reading symbols from /var/www/html/usr/local/apache2/modules/mod_proxy_http.so...done.
Loaded symbols for /usr/local/apache2/modules/mod_proxy_http.so
Reading symbols from /var/www/html/usr/local/apache2/modules/mod_ssl.so...done.
Loaded symbols for /usr/local/apache2/modules/mod_ssl.so
Reading symbols from /lib64/libssl.so.6...done.
Loaded symbols for /lib64/libssl.so.6
Reading symbols from /lib64/libcrypto.so.6...done.
Loaded symbols for /lib64/libcrypto.so.6
Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib64/libgssapi_krb5.so.2
Reading symbols from /usr/lib64/libkrb5.so.3...done.
Loaded symbols for /usr/lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /usr/lib64/libk5crypto.so.3...done.
Loaded symbols for /usr/lib64/libk5crypto.so.3
Reading symbols from /usr/lib64/libkrb5support.so.0...done.
Loaded symbols for /usr/lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libresolv.so.2...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libselinux.so.1...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libsepol.so.1...done.
Loaded symbols for /lib64/libsepol.so.1
Reading symbols from /var/www/html/usr/local/apache2/modules/mod_info.so...done.
Loaded symbols for /usr/local/apache2/modules/mod_info.so
Reading symbols from /var/www/html/usr/local/apache2/modules/mod_chroot.so...done.
Loaded symbols for /usr/local/apache2/modules/mod_chroot.so
Reading symbols from /var/www/html/usr/local/apache2/modules/libphp5.so...done.
Loaded symbols for /usr/local/apache2/modules/libphp5.so
Reading symbols from /usr/lib64/mysql/libmysqlclient.so.15...done.
Loaded symbols for /usr/lib64/mysql/libmysqlclient.so.15
Reading symbols from /usr/lib64/libmcrypt.so.4...done.
Loaded symbols for /usr/lib64/libmcrypt.so.4
Reading symbols from /usr/lib64/libltdl.so.3...done.
Loaded symbols for /usr/lib64/libltdl.so.3
Reading symbols from /usr/lib64/libfreetype.so.6...done.
Loaded symbols for /usr/lib64/libfreetype.so.6
Reading symbols from /usr/lib64/libX11.so.6...done.
Loaded symbols for /usr/lib64/libX11.so.6
Reading symbols from /usr/lib64/libXpm.so.4...done.
Loaded symbols for /usr/lib64/libXpm.so.4
Reading symbols from /usr/lib64/libpng12.so.0...done.
Loaded symbols for /usr/lib64/libpng12.so.0
Reading symbols from /usr/lib64/libjpeg.so.62...done.
Loaded symbols for /usr/lib64/libjpeg.so.62
Reading symbols from /usr/lib64/libcurl.so.3...done.
Loaded symbols for /usr/lib64/libcurl.so.3
Reading symbols from /lib64/libnsl.so.1...done.
Loaded symbols for /lib64/libnsl.so.1
Reading symbols from /usr/lib64/libxml2.so.2...done.
Loaded symbols for /usr/lib64/libxml2.so.2
Reading symbols from /usr/lib64/libidn.so.11...done.
Loaded symbols for /usr/lib64/libidn.so.11
Reading symbols from /usr/lib64/libXau.so.6...done.
Loaded symbols for /usr/lib64/libXau.so.6
Reading symbols from /usr/lib64/libXdmcp.so.6...done.
Loaded symbols for /usr/lib64/libXdmcp.so.6
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal 11, Segmentation fault.
[New process 25582]
#0 apr_pollset_add (pollset=0x0, descriptor=0x7fff51ab15a0) at poll/unix/epoll.c:150
150 if (pollset->flags & APR_POLLSET_NOCOPY) {
(gdb)
 
Old 11-08-2009, 04:36 PM   #4
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
That's a core dump analysis.

Instead, check your main Apache config (not sure what it's called in Ubuntu-land, but on most systems this is httpd.conf). Look for a directive like:
Code:
ServerRoot "/etc/httpd"
And another directive like:
Code:
ErrorLog logs/error_log
Then combine the two, forming (in this case) /etc/httpd/logs/error_log.

We need to see snippets from that log -- hopefully that will provide clues. Please use code tags so it's more readable.
 
Old 11-08-2009, 04:57 PM   #5
ventas
LQ Newbie
 
Registered: Apr 2009
Posts: 8

Original Poster
Rep: Reputation: 0
here are the error_logs

[Sun Oct 08 05:46:54 2009] [notice] child pid 31117 exit signal Segmentation fault (11)
[Sun Oct 08 05:47:07 2009] [notice] child pid 31255 exit signal Segmentation fault (11)
[Sun Oct 08 05:47:57 2009] [notice] child pid 30396 exit signal Segmentation fault (11)
[Sun Oct 08 05:47:59 2009] [notice] child pid 31512 exit signal Segmentation fault (11)
[Sun Oct 08 05:52:09 2009] [notice] child pid 31569 exit signal Segmentation fault (11)
[Sun Oct 08 05:52:20 2009] [notice] child pid 31550 exit signal Segmentation fault (11)
[Sun Oct 08 05:56:35 2009] [notice] child pid 31680 exit signal Segmentation fault (11)
[Sun Oct 08 05:57:38 2009] [notice] child pid 31748 exit signal Segmentation fault (11)
[Sun Oct 08 06:45:16 2009] [notice] child pid 31767 exit signal Segmentation fault (11)
[Sun Oct 08 07:36:24 2009] [notice] child pid 31699 exit signal Segmentation fault (11)
[Sun Oct 08 07:59:28 2009] [notice] child pid 31442 exit signal Segmentation fault (11)
[Sun Oct 08 08:00:35 2009] [notice] child pid 31493 exit signal Segmentation fault (11)
[Sun Oct 08 08:01:05 2009] [notice] child pid 31531 exit signal Segmentation fault (11)
[Sun Oct 08 08:08:57 2009] [notice] child pid 32644 exit signal Segmentation fault (11)
[Sun Oct 08 09:13:29 2009] [notice] child pid 1125 exit signal Segmentation fault (11)
[Sun Oct 08 09:36:59 2009] [notice] child pid 1851 exit signal Segmentation fault (11)

my apache version is 2.2.11


Thanks
 
Old 11-08-2009, 05:11 PM   #6
ventas
LQ Newbie
 
Registered: Apr 2009
Posts: 8

Original Poster
Rep: Reputation: 0
I just upgraded to 2.2.14 but did not help....

[Mon Nov 09 00:09:16 2009] [notice] child pid 26092 exit signal Segmentation fault (11)
[Mon Nov 09 00:09:16 2009] [notice] child pid 26170 exit signal Segmentation fault (11)
[Mon Nov 09 00:09:25 2009] [notice] child pid 26089 exit signal Segmentation fault (11)
[Mon Nov 09 00:09:30 2009] [notice] child pid 26102 exit signal Segmentation fault (11)
[Mon Nov 09 00:09:34 2009] [notice] child pid 26088 exit signal Segmentation fault (11)
[Mon Nov 09 00:09:34 2009] [notice] child pid 26128 exit signal Segmentation fault (11)
[Mon Nov 09 00:09:34 2009] [notice] child pid 26278 exit signal Segmentation fault (11)
[Mon Nov 09 00:09:35 2009] [notice] child pid 26169 exit signal Segmentation fault (11)
[Mon Nov 09 00:09:46 2009] [notice] child pid 26349 exit signal Segmentation fault (11)
[Mon Nov 09 00:10:01 2009] [notice] child pid 26369 exit signal Segmentation fault (11)
[Mon Nov 09 00:10:32 2009] [notice] child pid 26463 exit signal Segmentation fault (11)
[Mon Nov 09 00:10:36 2009] [notice] child pid 26283 exit signal Segmentation fault (11)
[Mon Nov 09 00:10:47 2009] [notice] child pid 26539 exit signal Segmentation fault (11)
[Mon Nov 09 00:10:48 2009] [notice] child pid 26167 exit signal Segmentation fault (11)
[Mon Nov 09 00:10:48 2009] [notice] child pid 26482 exit signal Segmentation fault (11)
 
Old 11-08-2009, 07:12 PM   #7
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
What sort of web applications are you running?

Also, let's see the output of: # httpd -M

Maybe one of your modules is configured badly and/or misbehaving.
 
Old 11-09-2009, 03:11 AM   #8
ventas
LQ Newbie
 
Registered: Apr 2009
Posts: 8

Original Poster
Rep: Reputation: 0
root@www4:~/httpd-2.2.14# httpd -M
Loaded Modules:
core_module (static)
authn_file_module (static)
authn_default_module (static)
authz_host_module (static)
authz_groupfile_module (static)
authz_user_module (static)
authz_default_module (static)
auth_basic_module (static)
include_module (static)
filter_module (static)
log_config_module (static)
env_module (static)
setenvif_module (static)
version_module (static)
mpm_worker_module (static)
http_module (static)
mime_module (static)
status_module (static)
autoindex_module (static)
asis_module (static)
cgid_module (static)
negotiation_module (static)
dir_module (static)
actions_module (static)
userdir_module (static)
alias_module (static)
so_module (static)
php5_module (shared)
Syntax OK
 
Old 11-09-2009, 01:48 PM   #9
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
That's a lot of static modules...

So you're running a php webapp, or what? Any details on that?
 
Old 11-09-2009, 04:14 PM   #10
ventas
LQ Newbie
 
Registered: Apr 2009
Posts: 8

Original Poster
Rep: Reputation: 0
yes, it's a PHP5 website.
 
Old 11-09-2009, 05:37 PM   #11
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
Try a google query for: "mod_php exit signal Segmentation fault"

Not sure what else to suggest until you give a lot more details about your app and what it's doing.
 
Old 11-09-2009, 08:01 PM   #12
sleddog
Member
 
Registered: Jan 2002
Location: Labrador, Canada
Distribution: CentOS, Debian
Posts: 182

Rep: Reputation: 35
Any system changes that coincide with the problem?

Did you upgrade PHP recently, do you run a PHP accelerator (like eaccelerator, xcache)...?
 
Old 11-10-2009, 03:14 AM   #13
ventas
LQ Newbie
 
Registered: Apr 2009
Posts: 8

Original Poster
Rep: Reputation: 0
I'm runnning php 5.2.10 with eaccelerator
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Apache Segmentation Fault Mark007 Linux - Newbie 1 03-04-2008 06:34 PM
apache segmentation fault kloppster Linux - Software 6 07-19-2004 01:56 PM
HELP PLEASE! Apache Segmentation Fault gabriele_101 Linux - Software 4 08-01-2003 12:56 AM
Apache CGI Segmentation Fault Jon- Linux - Software 1 07-15-2003 09:16 AM
Apache Segmentation Fault.... morbo Linux - General 2 12-14-2001 03:14 PM


All times are GMT -5. The time now is 11:41 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration