Nagios 3.0.6 Internal Server Error when accessing options on the web interface

fryzer · 04-24-2009, 02:16 PM

Problem statement:

Hello, I recently installed Nagios 3.0.6 with no complications at all under my server and I can access the web interface just fine, I configured Nagios to only run over the https and it works just fine, I can log in and I see the welcome page without any issues but when I click any of the left side options I get the "Internal Server Error" message.

Server Settings
The server is a:

Red Hat Enterprise Linux ES release 4 (Nahant Update 7)
Using Kernel 2.6.9-34.0.1.ELsmp
2GB RAM
Plenty Hard Drive space available (more than 100GB)
The server hosts multiple domains through Plesk Virtual Host technology and configurations
The Plesk version is 8.6.0
Nagios Version 3.0.6 as stated before
We use Mod Security
We use APF with IPtables

Nagios Installation journey
I installed Nagios as I said without any real errors, everything configured and compiled just fine and I followed the instructions from the quick start guide for Fedora contained here :
http://nagios.sourceforge.net/docs/3...rt-fedora.html

I followed every step very carefully and I ended up with what looks a like a working Nagios, I changed the e-mail address to mine in the appropriate configuration file as mentioned by such guide and as part of the process too, I wanted to implement some security measures for which I went ahead and choose the Enhanced CGI Security and Authentication option based on SSL followig the instructions from this webpage:

http://nagios.sourceforge.net/docs/3_0/cgisecurity.html

and I guess I'll be adding extra security using the IP Subnet Lockdown technique too but after this has been solved, Ok, sorry about that little comment detour, continuing with the guide steps after having implemented that security measure I proceeded to Install the plugins and everything went well there too and finally I tested the Nagios configuration files to see if there were any errors before I started the process and this is the result of the following command (mentioned in that guide too):

Command-->

Code:

/usr/local/nagios/bin/nagios -v /usr/local/nagios etc/nagios.cfg

Results

Code:

Nagios 3.0.6
Copyright (c) 1999-2008 Ethan Galstad (http://www.nagios.org)
Last Modified: 12-01-2008
License: GPL

Reading configuration data...

Running pre-flight check on configuration data...

Checking services...
        Checked 8 services.
Checking hosts...
        Checked 1 hosts.
Checking host groups...
        Checked 1 host groups.
Checking service groups...
        Checked 0 service groups.
Checking contacts...
        Checked 1 contacts.
Checking contact groups...
        Checked 1 contact groups.
Checking service escalations...
        Checked 0 service escalations.
Checking service dependencies...
        Checked 0 service dependencies.
Checking host escalations...
        Checked 0 host escalations.
Checking host dependencies...
        Checked 0 host dependencies.
Checking commands...
        Checked 24 commands.
Checking time periods...
        Checked 5 time periods.
Checking for circular paths between hosts...
Checking for circular host and service dependencies...
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors:   0

Things look okay - No serious problems were detected during the pre-flight check

Now, after looking at that and thinking that everything was correct I checked the SElinux configuration to avoid the so feared Internal Server Error from the guide and based on the command they suggest you to use which is:

Code:

getenforce

I got the following result:

Code:

Disabled

So SElinux is disabled, I knew that already but wanted to double check anyway, now, since SElinux is disabled I omitted the rest of the steps in that section regarding the SELinux and continued to the next step which pretty much involved login and enjoying of Nagios so I went to my https://mydomain.com/nagios and I got prompted with a user/pass window, I typed the ones I defined and they worked just fine and I could log, then I tried the options on the left side and none worked, instead I keep getting the so feared "Internal Server Error"

The Debugging process
When I got to this part my first instinct was to check the Nagios Log at:

Code:

/usr/local/nagios/var/nagios.log

...and there I could only see nothing but actual messages from nagios working as it is expected about alerts and warnings and services being Ok etc..

After that I went to check my httpd error log at:

Code:

/var/log/httpd/error_log

...and there I found this AHA!!!!:

Code:

[error] [client 127.0.0.1] Directory index forbidden by rule: /var/www/html/

So based on that I checked the ownership permissions on the Nagios directory and they are like this (the installation procedure left them like that)

Code:

drwxrwxr-x   8 nagios nagios 4.0K Apr 23 17:53 nagios

After checking that I tried changing the group to nagcmd by issuing a:

Code:

chown -R nagios:nagcmd nagios

but nothing changed (I must add that I know that I have to restart the httpd server every time a configuration file is changed ok and I did it, although for this change it was not necessary) but still same error, so I reverted the permissions change by restoring a previously created tgz file and then I tried checking the Nagios configuration file at:

Code:

/etc/httpd/conf.d/nagios.conf

and I have this:

Code:

ScriptAlias /nagios/cgi-bin "/usr/local/nagios/sbin"

<Directory "/usr/local/nagios/sbin">
   SSLRequireSSL
   Options ExecCGI
   AllowOverride None
   Order allow,deny
   Allow from all
#  Order deny,allow
#  Deny from all
#  Allow from 127.0.0.1
   AuthName "Nagios Access"
   AuthType Basic
   AuthUserFile /usr/local/nagios/etc/htpasswd.users
   Require valid-user
</Directory>

Alias /nagios "/usr/local/nagios/share"

<Directory "/usr/local/nagios/share">
   SSLRequireSSL
   Options None
   AllowOverride None
   Order allow,deny
   Allow from all
#  Order deny,allow
#  Deny from all
#  Allow from 127.0.0.1
   AuthName "Nagios Access"
   AuthType Basic
   AuthUserFile /usr/local/nagios/etc/htpasswd.users
   Require valid-user
</Directory>

So I thought, probably if I change the Options None to Options Indexes on the share area of that file it could work, but no, same error, then I thought probably this rule prevents the execution of any application (I must say I'm not an expert on the apache server ok) outside the /var/www directory so I placed all the application inside the that path having a new nagios set of directories at:

Code:

/var/www/nagios

with a Symlink on the original location and the application kept working as before but still the same error so I reverted those changes.

Then after reading and reading in forums, mailing lists and stuff I found that for one person this was solved by simply creating an empty index.html file inside:

Code:

/var/www/html/

so I did and tried again(remember that every time I attempted something new was based on the original settings so I didn't add more possible factors to the problem), I created index.html inside the /html dir and then I checked the options and nothing, still same error.

Conclusion and request for help
Guys, Ive been trying to make this work and I've been reading a lot through the Internet and nothing, I know it could be related to the permissions, Mod Security or Something else, but I haven't gotten to that level of debugging capacity and knowledge so I need a hand.

Any help, comment or hint is really appreciated.

Sorry about the thread being this long but I always tend to think that the more specific and the more info you provide, the easier and faster you can get a solution or help from someone else.

I guess the only left to say is Thanks in Advance for having taken the time to read this long thread and the time to reply to it.

unSpawn · 04-24-2009, 06:04 PM

Congratulations on the verbose OP, excellent. Your /etc/httpd/conf.d/nagios.conf isn't the problem but the DAC perms of your documentroot might. What does 'ls -ald /var/www/html' return?

fryzer · 04-27-2009, 03:31 PM

Hi I'm back, thanks for the reply and here is the result from that command you requested:

Code:

drwxr-xr-x  3 root root 4096 Apr 24 13:18 /var/www/html

...by the way, after researching a little bit more about this issue I found that this could be related to the suEXEC mod and if I go to my suexec_log file I found these type of entries after attempting to use the options on the left side on the nagios web interface:

Code:

[2009-04-27 15:26:09]: uid: (10001/www8_eonconnect) gid: (10001/10001) cmd: status.cgi
[2009-04-27 15:26:09]: command not in docroot (/usr/local/nagios/sbin/status.cgi)
[2009-04-27 15:27:46]: uid: (10001/www8_eonconnect) gid: (10001/10001) cmd: extinfo.cgi
[2009-04-27 15:27:46]: command not in docroot (/usr/local/nagios/sbin/extinfo.cgi)

...and looking into the FAQ from Nagios I found some suggestions to solve issues related to this based on the following text:

Quote:

Title: Problems installing the Nagios CGIs with Apache 2.0's suexec module
FAQ ID: F0118
Submitted By: David Torrey Jr.
Last Updated: 12/01/2002

Description:

Attempting to install the Nagios CGIs on an Apache 2.0 webserver with the suexec module may cause problems.

Solution:

The Nagios install defaults to creating files that are group writable, while suexec refuses to run things that are group writable. Fixing the CGI permissions manually (by removing group write permissions) post-install made the problem go away.

Apache's suexec also defaults to ignoring UIDs below 100, so you may have to create a "nagios" users with a UID above 100.

Keywords: apache suexec

Unfortunately after following those steps it didn't solve the Internal Server Error issue, I still need help so any inputs are greatly appreciated.

Thanks

fryzer · 04-27-2009, 03:45 PM

Hello, it's me again, in case it is needed please find some more info below:

The Nagios user id is:
10139

...and the permissions on the nagios directory structure are as attached on the .txt file.

mkiler · 04-28-2009, 06:13 AM

Hi
If you still have problem with nagios try this link:

http://debian.linux.pl/viewtopic.php...ghlight=nagios

It's my how to, in polish but maybe can help you.
I test's it in debian etch, lenny, rhel4.

fryzer · 04-28-2009, 05:09 PM

Thank you for the link to the guide, I'll try to follow the steps and even translate the page since a good friend of mine is polish so I'll come back with you if it helps in some way.

Unfortunately up to this moment I'm still in the same place and need help.

unSpawn · 04-28-2009, 05:50 PM

That link doesn't address suexec problems in any way?.. One option could be to recompile Apache2 with a "–with-suexec-docroot=" set to the root of your Nagios installation (I didn't try this). Another option could be to run without suexec (which shouldn't be done if the host runs publicly accessable services without user authentication). Another option could be to configure a VirtualHost for it as shown here.

banneris · 04-30-2009, 04:20 AM

Might be odd but...just to check, do you have PHP installed?

fryzer · 05-01-2009, 01:40 PM

Thanks guys I've tried everything and nothing has worked not even trying to set it up within a virtual host, I mean I could make the left menus to present info when installing it within a virtualhost but the plugins refused to work and so the application could monitor nothing.

Answering to the previous posted question, yeah, we have php installed.

nedohranen · 09-21-2009, 06:00 AM

Hi fryzer,

I had the same problem as you do. After spending a lot of time reading and trying anything, the step that solved the problem was issuing

Code:

chcon -R  -t httpd_sys_content_t /usr/local/nagios

You should have passed through this, if you followed the quick install guide in the nagios documentation.
Anyway, I was very surpised of this solution, because I thought I was disabled the firewall (since I was asked to do so before completing the CentOS 5.3 installation). In fact the firewall was running in "enforcing mode", so I had to tunt it off manualy by editing

Code:

/etc/selinux/config

and make it look like this

Code:

SELINUX=disabled

I hope I helped, if you're still in this trouble!

centosboy · 09-21-2009, 07:21 AM

Quote:

Originally Posted by fryzer

Thanks guys I've tried everything and nothing has worked not even trying to set it up within a virtual host, I mean I could make the left menus to present info when installing it within a virtualhost but the plugins refused to work and so the application could monitor nothing.

Answering to the previous posted question, yeah, we have php installed.

hmmm....so the nagios install is working because you can log in all ok.
The problem starts with the cgi links.
What you have to do is check that you have given the user you log in as permissions to operate the relevant cgi's.

locate the cgi.cfg file.

quick check on this file will tell you who is auth to do what within the web interface?

Code:

grep authori cgi.cfg  | grep -v '#'

authorized_for_system_information=nagiosadmin
authorized_for_configuration_information=nagiosadmini
authorized_for_system_commands=nagiosadmin
authorized_for_all_services=nagiosadmin
authorized_for_all_hosts=nagiosadmin
authorized_for_all_service_commands=nagiosadmin
authorized_for_all_host_commands=nagiosadmin

if yours looks like this, then you wont be able to execute any links after successful login.

also check mod_security logs as mod_security can sometimes filter cgi access.

fryzer · 09-22-2009, 10:04 AM

Hello everyone, thanks for the latest replies but to be honest I lost interest on this a while ago, it was so complicated and the information was so scarce and vague that I decided to not implement it.

I really appreciate the help from all of you that replied but as of right now we can forget about this thread.

Thanks again.

Zenaku · 10-07-2009, 03:52 PM

Quote:

Originally Posted by nedohranen

Hi fryzer,

I had the same problem as you do. After spending a lot of time reading and trying anything, the step that solved the problem was issuing

Code:

chcon -R  -t httpd_sys_content_t /usr/local/nagios

You should have passed through this, if you followed the quick install guide in the nagios documentation.
Anyway, I was very surpised of this solution, because I thought I was disabled the firewall (since I was asked to do so before completing the CentOS 5.3 installation). In fact the firewall was running in "enforcing mode", so I had to tunt it off manualy by editing

Code:

/etc/selinux/config

and make it look like this

Code:

SELINUX=disabled

I hope I helped, if you're still in this trouble!

DUDE!!! thanks a lot for the help. I put my SELINUX in "permissive", but it works anyway.
I really apretiate that help. I was look for days some answer about this problem.

Jenish_uk · 10-04-2010, 10:20 AM

Try this, it should work.

#su - (note space)
#setenforce 0
#service nagios restart

Now go back to your browser and check the links on the left menu

Cheers,
Jenish
Oxford

Matir · 11-11-2010, 10:13 AM

Please do not resurrect old threads. If you have a new issue, please feel free to start a new thread on the subject.