LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise
User Name
Password
Linux - Enterprise This forum is for all items relating to using Linux in the Enterprise.

Notices


Reply
  Search this Thread
Old 01-18-2007, 03:12 AM   #1
bkak
LQ Newbie
 
Registered: Jan 2007
Posts: 1

Rep: Reputation: 0
RHEL3 server going in hang state


Problem definition:
We are facing server hang problem from past 3 months. We have analyzed all our services that we are executing, and the server logs in /var/log/ but couldn’t find the solution. We are manually rebooting the server to recover it from hung state.

Action taken:
We have analyzed all the system logs and application logs in all our servers but we haven’t found any fixed pattern of messages in system logs. We are taking memory dump by top command for every 15 minutes and we found sufficient memory left before server going into hang state.

System configuration:
Red Hat Enterprise Linux ES release 3 (Taroon)
Kernel: 2.4.21-40.EL
Postgres: 7.3.8-2
Redhat Cluster Manager: 1.2.28
RAM: 2GB
Server: HP ML 370 G3, DL 760 G2

Please let me know the scenario’s in which server gets into hung state and what we need to check for rectifying the server hang problem.

Thank you in advance
 
Old 01-18-2007, 06:52 AM   #2
Lenard
Senior Member
 
Registered: Dec 2005
Location: Indiana
Distribution: RHEL/CentOS/SL 5 i386 and x86_64 pata for IDE in use
Posts: 4,790

Rep: Reputation: 57
Update the systems, for example; https://rhn.redhat.com/errata/RHSA-2006-0710.html
 
Old 01-18-2007, 09:10 AM   #3
aarontoth
LQ Newbie
 
Registered: Jan 2007
Posts: 3

Rep: Reputation: 0
Cool RHEL3 server going in hang state

I think another good idea to do is setup a crash script. Make it run every 10 seconds or whatever you think is appropriate. Report all system status' i.e. df, top, netstat, connections, ps... etc. have the system send out the alerts via mail. This should help a bit more than just looking at the logs.

AA
 
Old 01-25-2007, 06:41 PM   #4
nwilkens
LQ Newbie
 
Registered: Jan 2007
Location: Michigan
Posts: 1

Rep: Reputation: 0
Diskdump or netdump

Setup the diskdump-utils or netdump package to capture the system crash (if thats what happening). This will help you narrow down the problem.

Also, as suggested earlier and system update may also help.
 
Old 01-31-2007, 02:41 PM   #5
nifran
LQ Newbie
 
Registered: Jan 2007
Location: Indianapolis, IN
Distribution: RHEL *, Fedora *, Gentoo
Posts: 7

Rep: Reputation: 0
Install the sysstat package so that you'll collect data on performance.

Default on the installation collects memory usage, cpu usage, disk io, swap usage, and a number of other statistics every 10 minutes. You can change this down to a 1 minute interval if needed in /etc/cron.d/sysstat.

After the server crashes, you can run:
sar -r # gets memory information
sar # gets CPU information (like in top)
sar -q # load average and run que sizes
sar -n DEV # network interface statistics
sar -b # io rates

Those should give you a very good picture of what your server was doing when it hung, as well as any trend leading up to it.


Other than that, we've experienced a lot of the same problems with some of our machines. It turned out that the running kernel wasn't certified for the processors that we were running on, and updating the kernel fixed our issues. Take a look at the release notes for the newer kernels to see if they have added support for your server, or processors.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
rhel3 U5 hang with no error sutti Linux - Enterprise 8 12-19-2006 05:37 AM
rhel3 U5 hang with no error sutti Red Hat 1 12-17-2006 09:32 PM
Need urgen Help to install Qmail Server on RHEL3 ES vishal_titre Linux - Software 1 09-09-2006 03:54 PM
Linux sockets hang in FIN_WAIT1 state pavan Linux - Networking 2 06-19-2005 10:13 AM
X server not start on i810 in RHEL3 chetan3492 Red Hat 1 03-19-2005 03:04 PM

LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise

All times are GMT -5. The time now is 01:26 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration