LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 06-24-2009, 03:38 AM   #1
Pawang
LQ Newbie
 
Registered: Jun 2009
Posts: 11

Rep: Reputation: 0
*** glibc detected *** /usr/sbin/httpd: malloc(): memory corruption problem -


Hi All,

I am having a problem in my Apache Server (RHEL5).Where apache server stops responding suddenly and when restarted httpd it starts working.It happened 3 times in last two days.I am not able to figure out the problem.

Please help me to resolve the same.

Please find the error message found in /var/log/httpd/error_log:

[Tue Jun 23 10:48:08 2009] [notice] child pid 21869 exit signal Segmentation fault (11)
*** glibc detected *** /usr/sbin/httpd: malloc(): memory corruption (fast): 0xb6a00807 ***
======= Backtrace: =========
/lib/libc.so.6[0x396803]
/lib/libc.so.6(__libc_malloc+0x7e)[0x39712e]
/lib/libc.so.6(__strdup+0x30)[0x39bb80]
/etc/httpd/modules/libphp5.so[0x5dee13b]
/etc/httpd/modules/libphp5.so(zend_error+0x3db)[0x5e30ccb]
/etc/httpd/modules/libphp5.so[0x5e82c1d]
/etc/httpd/modules/libphp5.so(execute+0x188)[0x5e4ec98]
/etc/httpd/modules/libphp5.so[0x5e56348]
/etc/httpd/modules/libphp5.so(execute+0x188)[0x5e4ec98]
/etc/httpd/modules/libphp5.so(zend_execute_scripts+0x2f0)[0x5e2f1b0]
/etc/httpd/modules/libphp5.so(php_execute_script+0x255)[0x5deea55]
/etc/httpd/modules/libphp5.so[0x5eb20a5]
/usr/sbin/httpd(ap_run_handler+0x6d)[0x7c18fd]
/usr/sbin/httpd(ap_invoke_handler+0x68)[0x7c52a8]
/usr/sbin/httpd(ap_process_request+0x1ae)[0x7d132e]
/usr/sbin/httpd[0x7ce11f]
/usr/sbin/httpd(ap_run_process_connection+0x6d)[0x7c97bd]
/usr/sbin/httpd(ap_process_connection+0x4c)[0x7c98bc]
/usr/sbin/httpd[0x7d58e2]/usr/sbin/httpd(ap_mpm_run+0x913)[0x7d65f3]
/usr/sbin/httpd(main+0x8b7)[0x7ad127]
/lib/libc.so.6(__libc_start_main+0xdc)[0x344dec]
/usr/sbin/httpd[0x7ac221]
======= Memory map: ========
00110000-00135000 r-xp 00000000 fd:00 3987560 /lib/libm-2.5.so
00135000-00136000 r-xp 00024000 fd:00 3987560 /lib/libm-2.5.so
00136000-00137000 rwxp 00025000 fd:00 3987560 /lib/libm-2.5.so
00137000-0013a000 r-xp 00000000 fd:00 3987563 /lib/libuuid.so.1.2
0013a000-0013b000 rwxp 00002000 fd:00 3987563 /lib/libuuid.so.1.2
0013b000-0014a000 r-xp 00000000 fd:00 3987570 /lib/libresolv-2.5.so
0014a000-0014b000 r-xp 0000e000 fd:00 3987570 /lib/libresolv-2.5.so
0014b000-0014c000 rwxp 0000f000 fd:00 3987570 /lib/libresolv-2.5.so
0014c000-0014e000 rwxp 0014c000 00:00 0
0014e000-00161000 r-xp 00000000 fd:00 3987568 /lib/libnsl-2.5.so
00161000-00162000 r-xp 00012000 fd:00 3987568 /lib/libnsl-2.5.so
00162000-00163000 rwxp 00013000 fd:00 3987568 /lib/libnsl-2.5.so
00163000-00165000 rwxp 00163000 00:00 0
00165000-00167000 r-xp 00000000 fd:00 3987573 /lib/libcom_err.so.2.1
00167000-00168000 rwxp 00001000 fd:00 3987573 /lib/libcom_err.so.2.1
00168000-0018d000 r-xp 00000000 fd:00 289292 /usr/lib/libk5crypto.so.3.1
0018d000-0018e000 rwxp 00025000 fd:00 289292 /usr/lib/libk5crypto.so.3.1
0018e000-00190000 r-xp 00000000 fd:00 3987569 /lib/libkeyutils-1.2.so
00190000-00191000 rwxp 00001000 fd:00 3987569 /lib/libkeyutils-1.2.so
00191000-00192000 r-xp 00000000 fd:00 1145950 /usr/lib/httpd/modules/mod_authn_file.so

002e8000-002ea000 rwxp 00000000 fd:00 1145946 /usr/lib/httpd/modules/mod_authn_anon.so
002ea000-002eb000 r-xp 00000000 fd:00 1145948 /usr/lib/httpd/modules/mod_authn_dbm.so
002eb000-002ed000 rwxp 00000000 fd:00 1145948 /usr/lib/httpd/modules/mod_authn_dbm.so
002ed000-002ee000 r-xp 00000000 fd:00 1145949 /usr/lib/httpd/modules01119000-01245000 r-xp 00000000 fd:00 272499 /usr/lib/libxml2.so.2.6.26
01245000-0124a000 rwxp 0012b000 fd:00 272499 /usr/lib/libxml2.so.2.6.26
0124a000-0124b000 rwxp 0124a000 00:00 0
0124b000-0133e000 r-xp 00000000 fd:00 276222 /usr/lib/libpython2.4.so.1.0
0133e000-01360000 rwxp 000f3000 fd:00 276222 /usr/lib/libpython2.4.so.1.0
01360000-01363000 rwxp 01360000 00:00 0
0172a000-01767000 r-xp 00000000 fd:00 277707 /usr/lib/libodbcpsql.so.2.0.0
01767000-01769000 rwxp 0003c000 fd:00 277707 /usr/lib/libodbcpsql.so.2.0.0
01769000-0179b000 rwxp 01769000 00:00 0
019ef000-01aa8000 r-xp 00000000 fd:00 272298 /usr/lib/libaspell.so.15.1.3
01aa8000-01aac000 rwxp 000b8000 fd:00 272298 /usr/lib/libaspell.so.15.1.3
01aac000-01ab0000 rwxp 01aac000 00:00 0
025f5000-0271f000 r-xp 00000000 fd:00 360990 /usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
0271f000-02724000 rwxp 0012a000 fd:00 360990 /usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
02724000-02726000 rwxp 02724000 00:00 0
02868000-02948000 r-xp 00000000 fd:00 271433 /usr/lib/libstdc++.so.6.0.8
02948000-0294c000 r-xp 000df000 fd:00 271433 /usr/lib/libstdc++.so.6.0.8
0294c000-0294d000 rwxp 000e3000 fd:00 271433 /usr/lib/libstdc++.so.6.0.8
0294d000-02953000 rwxp 0294d000 00:00 0
043a0000-0440a000 r-xp 00000000 fd:00 279412 /usr/lib/libodbc.so.1.0.0
0440a000-0440f000 rwxp 00069000 fd:00 279412 /usr/lib/libodbc.so.1.0.0
05c6f000-05f2f000 r-xp 00000000 fd:00 1146516 /usr/lib/httpd/modules/libphp5.so
05f2f000-05f89000 rwxp 002bf000 fd:00 1146516 /usr/lib/httpd/modules/libphp5.so
05f89000-05f90000 rwxp 05f89000 00:00 0
08775000-08c26000 rw-p 08775000 00:00 0
b6500000-b65fe000 rw-p b6500000 00:00 0b6f82000-b7000000 rw-s 00000000 00:08 55763563 /dev/zero (deleted)
b7000000-b7100000 rw-p b7000000 00:00 0
b7142000-b71c4000 rw-p b7142000 00:00 0
b71c4000-b71c5000 ---p b71c4000 00:00 0
b71c5000-b7bc5000 rw-p b71c5000 00:00 0
b7bcd000-b7f8b000 rw-s 00000000 fd:00 428025 /tmp/.NSPR-AFM-21861-88c8f78.0 (deleted)
b7f8b000-b7fa4000 rw-s 00000000 00:08 55763527 /dev/zero (deleted)
b7fa4000-b7fac000 rw-p b7fa4000 00:00 0
bfb0b000-bfb20000 rw-p bfb0b000 00:00 0 [stack]
[Tue Jun 23 10:48:13 2009] [notice] child pid 21918 exit signal Aborted (6)

b65fe000-b6600000 ---p b65fe000 00:00 0
b6700000-b67ff000 rw-p b6700000 00:00 0
b67ff000-b6800000 ---p b67ff000 00:00 0

00f03000-00f0d000 rwxp 00f03000 00:00 0
00f2f000-00f5b000 r-xp 00000000 fd:00 289294 /usr/lib/libgssapi_krb5.so.2.2

Thanks in advance

Regards,
Pawanganesh
 
Old 06-24-2009, 06:06 AM   #2
Agrouf
Senior Member
 
Registered: Sep 2005
Location: France
Distribution: LFS
Posts: 1,591

Rep: Reputation: 79
It has something to do with php.
look at access_log to see which page was last accessed.
 
Old 06-27-2009, 03:23 AM   #3
Pawang
LQ Newbie
 
Registered: Jun 2009
Posts: 11

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Agrouf View Post
It has something to do with php.
look at access_log to see which page was last accessed.
Thank for ur reply.We have found the problem.The problem is 15 days back to tweak mysql perfomance we have made changes in my.cnf to allocate more memory to mysql and restarted mysql.we didnt restart httpd that time.

When we restarted httpd last week the problem started.There is memory allocation for apache.

when we restored with old mysql configuration my.cnf the problem stopped happening.

Server IBM x3400 with 2gb memory.30- 40 concurrent users accessing the server(Hospital Management System Application).

database details:

456K marthasbloodbankv2
2.2G marthasfacontrollerv3
111M marthasfav
19M marthaspayrollv31
161M marthaspharmacyv2
241M marthaspharmacyv22008
948K marthasstoresv2
37M marthasultrasoundv2
863M marthasv2
3.2M marthasxrayv2

Present my.cnf
-------------
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1

# ganesh begin

port = 3306
key_buffer_size=32M
max_allowed_packet=10M
#table_cache=127
max_connections=310
max_user_connections=300
wait_timeout=120
flush_time=300
query_cache_type = 1
query_cache_size = 16M
innodb_data_home_dir =
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_file_size=20M
innodb_additional_mem_pool_size=10M
innodb_buffer_pool_size=64M
innodb_lock_wait_timeout=120

# ganesh end

[mysql.server]
user=mysql
basedir=/var/lib

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

tried my.cnf
-----------

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1

# ganesh begin

port = 3306
key_buffer_size=64M
max_allowed_packet=10M
table_cache=256
thread_cache_size=16
max_connections=310
max_user_connections=300
wait_timeout=600
sort_buffer_size=4M
join_buffer_size =4M
read_buffer_size=4M
read_rnd_buffer_size = 16M
myisam_sort_buffer_size = 64M
flush_time=300
long_query_time=15
tmp_table_size=32M
query_cache_type = 1
query_cache_limit= 2M
query_cache_size = 32M
innodb_data_home_dir =
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_file_size=20M
innodb_additional_mem_pool_size=10M
innodb_buffer_pool_size=64M
innodb_lock_wait_timeout=120

# ganesh end

[mysql.server]
user=mysql
basedir=/var/lib

[mysqld_safe]
log-error=/var/log/mysqld.log

Can you give me some suggestion to increase the perfomance of the server.

Thanks and regards,
Pawan Ganesh
 
  


Reply

Tags
apache, server


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
/usr/sbin/httpd: symbol lookup error: /usr/sbin/httpd: undefined symbol: apr_atomic_x dasoberdick Slackware 16 10-11-2014 03:36 AM
*** glibc detected *** malloc(): memory corruption arvind.ayyangar Programming 2 11-21-2006 12:59 AM
glibc detected *** malloc(): memory corruption: abefroman Linux - Software 2 04-12-2006 01:12 PM
Starting httpd: /usr/sbin/httpd: symbol lookup error: /usr/local/lib/libaprutil-0.so. bijuhpd Linux - Newbie 1 10-30-2005 06:07 PM
glibc detected ; memory corruption pingu Linux - Software 3 10-12-2005 02:29 AM


All times are GMT -5. The time now is 06:26 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration