LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices



Reply
 
Search this Thread
Old 11-12-2005, 12:34 PM   #1
Louie55
Member
 
Registered: Oct 2003
Location: Nebraska
Distribution: SuSE 9.0, Redhat 9.0
Posts: 41

Rep: Reputation: 15
Apache Crashes at 12:15 AM Every Time!


Hello,

I am having a problem with a RedHat 9.0 Webserver that I am running with Apache 2. Once in awhile (not every day), about every week or 2, apache will crash. The funny part is, every time it crashes, the first "Status Warning" e-mail message is sent to me at 12:15 AM. It is always exactly 12:15 AM.

The "Status Warning" message gives me this as why Apache could not be restarted:

Code:
Nov 12 00:11:00 bulldog httpd: httpd shutdown succeeded
Nov 12 00:11:00 bulldog httpd: (98)Address already in use: make_sock: could not bind to address 0.0.0.0:443
Nov 12 00:11:00 bulldog httpd: no listening sockets available, shutting down
Nov 12 00:11:00 bulldog httpd: Unable to open logs
Nov 12 00:11:00 bulldog httpd: httpd startup failed
Nov 12 00:11:00 bulldog httpd: httpd shutdown failed
Nov 12 00:11:00 bulldog httpd: (98)Address already in use: make_sock: could not bind to address 0.0.0.0:443
Nov 12 00:11:00 bulldog httpd: no listening sockets available, shutting down
Nov 12 00:11:00 bulldog httpd: Unable to open logs
Nov 12 00:11:00 bulldog httpd: httpd startup failed
Nov 12 00:15:00 bulldog httpd: httpd shutdown failed
Nov 12 00:15:01 bulldog httpd: (98)Address already in use: make_sock: could not bind to address 0.0.0.0:443
Nov 12 00:15:01 bulldog httpd: no listening sockets available, shutting down
Nov 12 00:15:01 bulldog httpd: Unable to open logs
Nov 12 00:15:01 bulldog httpd: httpd startup failed
It is always fixed by somebody else, not me, so I can't tell you exactly what the state of the machine is when it happens. They usually fix it by rebooting the server.

I have checked root crontab and nothing is set to run around 12:15 AM, so I can't pinpoint it. Any more ideas??

Thanks.

Louie
 
Old 11-12-2005, 06:08 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,679
Blog Entries: 54

Rep: Reputation: 2954Reputation: 2954Reputation: 2954Reputation: 2954Reputation: 2954Reputation: 2954Reputation: 2954Reputation: 2954Reputation: 2954Reputation: 2954Reputation: 2954
Seems there is already something listening on the HTTPS port when it tries to restart. Find out what it is.
Next to that check if there's any app that tries to restart httpd if it fails.
 
Old 11-12-2005, 08:20 PM   #3
farslayer
Guru
 
Registered: Oct 2005
Location: Willoughby, Ohio
Distribution: linuxdebian
Posts: 7,232
Blog Entries: 5

Rep: Reputation: 189Reputation: 189
Next time the server crashes, Try using netstat to determine what has the port open for listening.

netstat -alp | grep :https
or
netstat -alp | grep :443

I don't have a web server running on this box, but Here's what the output looks like for IPP (Internet priting protocol..) notice since I used the -p flag it lists the process name that has the port open ? in this case it's the cups printing daemon. you should see a similar listig for port 443 or https when you use this command.
Quote:
debian:~# netstat -alp | grep :ipp
tcp 0 0 *:ipp *:* LISTEN 9793/cupsd
udp 0 0 *:ipp *:* 9793/cupsd

Last edited by farslayer; 11-13-2005 at 05:02 PM.
 
Old 11-12-2005, 09:58 PM   #4
KimVette
Senior Member
 
Registered: Dec 2004
Location: Lee, NH
Distribution: OpenSUSE, CentOS, RHEL
Posts: 1,794

Rep: Reputation: 46
If the time is the same for every single occurence there are several possibilities:

- your apache build or one of your web apps has a time-sensitive bug
- a cron job is messing things up (check crontab, /etc/cron.*/, etc)
- Someone else's box infected with worms or a trojan horse is hitting your site with malformed or malicious requests (check the logs, such things would be obvious)

There are other possibilities but check for those first.
 
Old 11-13-2005, 08:43 PM   #5
Louie55
Member
 
Registered: Oct 2003
Location: Nebraska
Distribution: SuSE 9.0, Redhat 9.0
Posts: 41

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by unSpawn
Seems there is already something listening on the HTTPS port when it tries to restart. Find out what it is.
Next to that check if there's any app that tries to restart httpd if it fails.
I am about 99% sure that it is Apache that has port 443 bound up. What is was doing last time I SSH'ed into it when it was crashed, there were apache processes still running (many of them, no childs, only parent processes). They would not die in the regular way however. Lets say that one of the processes had a process ID of 873 just for example. If you do "kill -HUP 873", it has no effect, if you do "kill 873", that has no effect either. If you do "killall httpd" or "killall apache", that also has no effect as it comes back saying it can't find any. However, if I do "kill -KILL 873", that finally kills it. I am not sure what the -KILL does, I just found it on the internet somewhere. After all apache processes were finally killed, then apache came back up within 1 minute.

Now, to the question about what apps try to restart apache if it is down, there are 2 of them on the server. The first one is the DirectAdmin Web Hosting Control Panel Software, and the other is a system monitoring program (I can't remember the name). Both of these monitor services on the machine and attempt to restart them when they find one down. But, since apache is still running in a semi-crashed state, it can not be restarted because the crashed form of apache is still bound to the ports.

I had not thought of some other machine sending some kind of requests to crash it, I will check the logs.

Louie
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
evolution crashes every time I start it JimBass Linux - Newbie 4 07-22-2008 10:53 PM
system crashes time to time Artik Debian 2 12-02-2005 08:19 AM
USB2 crashes big time xushi Slackware 10 08-21-2005 03:45 PM
Apollon crashes every time Celettu Linux - Software 0 07-14-2004 01:06 PM
Mouse crashes in X after some time sfbence Mandriva 3 03-02-2004 06:10 AM


All times are GMT -5. The time now is 06:50 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration