Tighten-up Security for purely a Web server (no email needed)

GwenE · 06-11-2009, 06:05 PM

I have a 'cloud hosted' Linux server -- RHEL 5.1 64 bit, with Apache 2.2. I like to SSH into this box to transfer files, and do some small commands like starting my Eclipse Standalone Online Help demonstration (accessible via port 8082).

I have no need to use email on this system... and in fact I am quite clueless as to the details of the email servers or services provided.

Recently I found out from the Cloud Hosting Team:
"We are getting complaints regarding unsolicited Email, or spam coming from your server(s)."

The actual admin of the receiver of one of these spams said:
"One of our clients forwarded us what appears to be a bot form submission that originated from one of your IP addresses <ip address>."

Where the <ip address> is my virutual (cloud hosted) linux server.

As I say, I'm rather ignorant on these matters. I'm just hoping there is an easy way to tighten up security given that I do not need any email services. I only want people to be able to view (through a browser) my Eclipse help available at: <ip address>:8082/help/index.jsp

Here are the contents of my /etc/sysconfig/iptables file:

Code:

# Firewall configuration written by system-config-securitylevel
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p 50 -j ACCEPT
-A RH-Firewall-1-INPUT -p 51 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8081 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8082 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp --dport 5353 -d 224.0.0.251 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT

And here are parts of my /etc/httpd/conf/httpd.conf file that I have edited since I instantiated the cloud-hosted virtual server. Note that I commented out the proxy stuff today when I heard about the spam -- but I don't know how to detect if that helped or not:
(Note that in following <ip address> is my ip address.)

Code:

#  Gwens notes adding in the recommended line from Eclipse doc for proxy pass
#    Gwen's new notes on 6/11/09 -- commenting these out as I have a warning of a spam bot sending from
#     my address
#ProxyPass /demo/ http://<ip address>:8082/help/
#ProxyPassReverse /demo/ http://<ip address>:8082/help/
.
.
.
#   Gwen is commenting out per Eclipse doc on Making Infocenter available on the web
#AddDefaultCharset UTF-8
.
.
.
#     More Gwen's notes: I am commenting this back out on 6/11/09 as I am apparently having stuff sent by 
#     a bot (spam??) 
#
#                    
<IfModule mod_proxy.c>
#ProxyRequests On
#

Any suggestions would be greatly appreciated. Thank you.

anomie · 06-11-2009, 06:29 PM

Have you taken a look at your httpd access logs / error logs yet? How about the daily logwatch reports? (Check root's mail if you haven't already.)

Where would you expect your customer base be accessing your web application from? Anywhere in the world? Or just a couple known networks?

GwenE · 06-11-2009, 11:46 PM

Thank you for the quick reply. The httpd access log is huge, and there are several iterations (access_log.1, access_log.2... over the last month).

The current access log is over 630 meg and the previous one has a date stamp of June 7th (only four days ago).

I have a feeling I'm a bit in over my head...
Here is just a very little bit of the access log:

Code:

61.139.105.162 - - [07/Jun/2009:21:50:37 -0700] "GET http://ad.scanmedios.com/rw?title=&qs=iframe3%3FAAAAAPKoBwBPwh8AO%2EgJAAIAAAAAAP8AAAABFgIBAAP17QsAl3oJANA7DgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANYnBA0AAAAAAAAAAAAAAgAGddhBAAAAAAAAAAAAAAIACW3AbQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnsE%2Eprh7dQZVn6Qa3Te9L0QKf%2EUS2c38PJBOSgAAAAA%3D%2C%2Chttp%253a%252f%252fwww%2Eflashgamehome%2Ecom%252findex%2Ehtml HTTP/1.1" 200 557 "http%3A%2F%2Fwww.flashgamehome.com%2Findex.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
61.139.105.166 - - [07/Jun/2009:21:50:38 -0700] "GET http://ad.yieldmanager.com/iframe3?AAAAAEWaCAB6xyQAGQELAAIAAAAAAP8AAAABFgIBAAMV5AwAbakMACyxDwAAAAAAAAAAAAAAAAAAAAAAAAAAADMzMzMzM9M.MzMzMzMz0z8AAAAAAADgPwAAAAAAAOA.AAAAAAAA4D8AAAAAAADgPwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAALTDiu7x7dQYWOlTR37jXnKcgHjTEsvZU0tIBxgAAAAA=,,http%3a%2f%2fwww.flashopping.net%2findex.html HTTP/1.1" 302 - "http%3A%2F%2Fwww.flashopping.net%2Findex.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 2.0.40"
74.52.177.210 - - [07/Jun/2009:21:50:38 -0700] "GET http://www.isisrecovery.com/service_treatment_plan.php HTTP/1.1" 200 5861 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
174.120.24.242 - - [07/Jun/2009:21:50:36 -0700] "GET http://www.virginnigeria.com/eagleflier/ HTTP/1.1" 200 17792 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
67.159.44.143 - - [07/Jun/2009:21:50:35 -0700] "POST http://game.sun116.com/vdgame/msglist.aspx?act=add&ClipID=21084 HTTP/1.1" 302 150 "http://game.sun116.com/vdgame/msglist.aspx?clipid=21084&currpage=510" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
174.133.15.50 - - [07/Jun/2009:21:50:37 -0700] "GET http://www.pique.at/project/countries.html HTTP/1.1" 200 7325 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
174.133.15.50 - - [07/Jun/2009:21:50:37 -0700] "GET http://www.hepmed.com/f-info.htm HTTP/1.1" 200 51019 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
74.52.177.210 - - [07/Jun/2009:21:50:38 -0700] "GET http://www.isisrecovery.com/service_mental_health.php HTTP/1.1" 200 5691 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
174.120.24.242 - - [07/Jun/2009:21:50:38 -0700] "GET http://www.mycalabasas.com/html/main/classifieds_display/msgID/5006990/index.html HTTP/1.1" 200 20053 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
67.159.45.10 - - [07/Jun/2009:21:50:38 -0700] "GET http://www.cnet.com/4360-5_7-6563606.html?key=samsung_cellphones&ttag=cnetfd.aisledir-samsung-cellphones HTTP/1.1" 301 320 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
74.52.177.210 - - [07/Jun/2009:21:50:38 -0700] "GET http://take-shape-share.fenc.org.uk/ HTTP/1.1" 302 150 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
174.133.15.50 - - [07/Jun/2009:21:50:37 -0700] "GET http://www.thetrailmaster.com/content/trail-sign-language HTTP/1.1" 200 21394 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"

I'm going to try to hone in on the exact time they said the spam was sent. Anything I should specifically look for?

As far as traffic I expect on the site-- I don't expect any yet as I haven't sent the URL to anyone (though I know that doesn't mean anything with crawlers out there). I'm simply working on a demonstration of my technical writing capabilities with an Eclipse Standalone help system (running in /usr/bin/info_online2 ) which is acccessed via port 8082. (And that is not even ready for prime time.)

I don't believe I have logwatch setup (as there is nothing in the \etc\logwatch\conf\logfiles\ folder).

I'm going to go wrestle with the large access log.

Meanwhile I can at least open the httpd error log. Here is a sampling of that log below.

Code:

[Thu Jun 11 12:05:46 2009] [error] [client 174.120.24.242] proxy: error reading status line from remote server www.jungangsijang.co.kr
[Thu Jun 11 12:05:46 2009] [error] [client 174.120.24.242] proxy: Error reading from remote server returned by http://www.jungangsijang.co.kr/gallery/del_comment.php?no=53&menu_id=18&c_no=405560&start=0
[Thu Jun 11 12:06:15 2009] [error] [client 67.159.45.10] proxy: error reading status line from remote server www.lanbook.com
[Thu Jun 11 12:06:15 2009] [error] [client 67.159.45.10] proxy: Error reading from remote server returned by http://www.lanbook.com/publish/news.php
[Thu Jun 11 12:07:40 2009] [error] [client 67.159.45.10] proxy: error reading status line from remote server www.mil.be
[Thu Jun 11 12:07:40 2009] [error] [client 67.159.45.10] proxy: Error reading from remote server returned by http://www.mil.be/def/index.asp?LAN=nl
[Thu Jun 11 12:07:57 2009] [error] proxy: client 174.120.24.226 given Content-Length did not match number of body bytes read
[Thu Jun 11 12:07:57 2009] [error] (70014)End of file found: proxy: pass request body failed to 222.92.117.45:80 (dict.hjenglish.com) from 174.120.24.226 ()
[Thu Jun 11 12:07:59 2009] [error] [client ::1] Directory index forbidden by Options directive: /var/www/html/
[Thu Jun 11 12:08:02 2009] [error] [client ::1] Directory index forbidden by Options directive: /var/www/html/
[Thu Jun 11 12:08:07 2009] [error] [client 74.52.177.210] proxy: error reading status line from remote server slashdot.org
[Thu Jun 11 12:08:07 2009] [error] [client 74.52.177.210] proxy: Error reading from remote server returned by http://slashdot.org/bookmark.pl?url=http%3A%2F%2Fwww.slideboom.com%2Fpresentations%2F9677%2FDigital-Printing-Company&title=Digital%20Printing%20Company
[Thu Jun 11 12:08:18 2009] [error] [client 174.120.24.242] proxy: error reading status line from remote server www.kroniquent.com
[Thu Jun 11 12:08:18 2009] [error] [client 174.120.24.242] proxy: Error reading from remote server returned by http://www.kroniquent.com/blog/ginette.html
[Thu Jun 11 12:10:29 2009] [notice] caught SIGTERM, shutting down
[Thu Jun 11 12:10:30 2009] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Jun 11 12:10:30 2009] [notice] Digest: generating secret for digest authentication ...
[Thu Jun 11 12:10:30 2009] [notice] Digest: done
[Thu Jun 11 12:10:30 2009] [notice] mod_python: Creating 4 session mutexes based on 256 max processes and 0 max threads.
[Thu Jun 11 12:10:30 2009] [error] [client 174.120.24.242] File does not exist: /var/www/html/about_4.html
[Thu Jun 11 12:10:30 2009] [notice] Apache/2.2.3 (Red Hat) configured -- resuming normal operations
[Thu Jun 11 12:10:31 2009] [error] [client 74.55.95.250] File does not exist: /var/www/html/emailthispage.aspx, referer: http://www.treatmentactiongroup.org/emailthispage.aspx
[Thu Jun 11 12:10:32 2009] [error] [client 67.159.44.143] File does not exist: /var/www/html/phynea
[Thu Jun 11 12:10:32 2009] [error] [client 74.55.95.250] File does not exist: /var/www/html/Subscribe.aspx, referer: http://www.yourhealth.net.au/Subscribe.aspx?id=118
[Thu Jun 11 12:10:34 2009] [error] [client 67.159.45.10] File does not exist: /var/www/html/iotw
[Thu Jun 11 12:10:34 2009] [error] [client 71.56.95.140] File does not exist: /var/www/html/servlet
[Thu Jun 11 12:10:35 2009] [error] [client 67.159.45.10] Directory index forbidden by Options directive: /var/www/html/

From what I can tell the numerous 'File does not exist:' lines starting at Jun 11 12:10:30 coincide with my restarting of apache after commenting out the proxy stuff within the httpd.conf file. Hopefully that helped.... but from the sheer immensity of this, I doubt that it helped much.

I'm thinking I should shut down my server... or better yet limit the viewing of my Eclipse help demo to just my home machine (if possible) until I know what to do. (Especially since my demo isn't ready to view anyway.)
Thanks again for any thoughts.

grepmasterd · 06-12-2009, 12:20 AM

can you just remove your MTA (mail server) software, or at least configure it for local (system) delivery only?

As a side issue, ACCEPT policies on iptables firewalls are not recommended. you should consider changing:

Code:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]

to:

Code:

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]

you won't likely see any difference in behavior in general, but it's good practice to do this.

GwenE · 06-12-2009, 01:36 AM

Thank you. I changed the iptables ACCEPT policies to DROP policies as you recommended. And I restarted the firewall.

And I think removing my MTA (mail server) software would be good-- I just need to look into how to do that

Meanwhile I made sure sendmail wasn't running (I don't think it was) and I stopped apache server for now.

I also learned how to get LogWatch going and ran it for analysis of the last day. Here are parts that look interesting (though there is lots more stuff as it is 22 meg)

Note that I took out the actual host name from the log below.

Code:

################### Logwatch 7.3 (03/24/06) #################### 
        Processing Initiated: Thu Jun 11 22:27:11 2009
        Date Range Processed: yesterday
                              ( 2009-Jun-10 )
                              Period is day.
      Detail Level of Output: 0
              Type of Output: unformatted
           Logfiles for Host: ______.dev.___.com   <-- < Gwens note: I removed the host name here from log. I'm starting to wonder if this is a log for a large physical entity rather than just my virtual server IP (?). > 
  ################################################################## 
 
 --------------------- httpd Begin ------------------------ 

 Connection attempts using mod_proxy:
    67.228.177.87 -> auctions.godaddy.com:443: 1 Time(s)
    74.86.171.82 -> auctions.godaddy.com:443: 2 Time(s)
 
 A total of 10 sites probed the server 
    "POST
    174.120.24.226
    174.120.24.242
    209.62.110.178
    61.139.105.162
    61.139.105.166
    67.159.44.143
    67.159.45.10
    74.52.177.210
    74.55.95.250
 
 A total of 53102 possible successful probes were detected (the following URLs
 contain strings that match one or more of a listing of strings that
 indicate a possible exploit):
 
    http://ad.adserverplus.com/iframe3?A...t%2findex.html HTTP Response 302 
    http://ad.yieldmanager.com/iframe3?e...com/index.html HTTP Response 200 
    http://adserving.cpxinteractive.com/...2findex%2Ehtml HTTP Response 200 
   .
   .
   .
    http://ad.media-servers.net/rw?title...2findex%2Ehtml HTTP Response 200 
    http://ad.media-servers.net/rw?title...2findex%2Ehtml HTTP Response 200 
 .
 .
 .
  --------------------- pam_unix Begin ------------------------ 

 sshd:
    Authentication Failures:
       unknown (huizenzoeken.com): 29771 Time(s)
       news (huizenzoeken.com): 654 Time(s)
       apache (huizenzoeken.com): 621 Time(s)
     .
     .
     .
       squid (huizenzoeken.com): 1 Time(s)
       uucp (huizenzoeken.com): 1 Time(s)
    Invalid Users:
       Unknown Account: 30191 Time(s)
 
 
 ---------------------- pam_unix End ------------------------- 

 --------------------- SSHD Begin ------------------------ 

 
 Failed logins from:
    85.17.65.36 (huizenzoeken.com): 4611 times
    89.200.170.179: 207 times
    218.12.227.18: 18 times
 
 Illegal users from:
    85.17.65.36 (huizenzoeken.com): 29770 times
    89.200.170.179: 420 times
 
 
 Received disconnect:
    11: Bye Bye : 35019 Time(s)
 
 **Unmatched Entries**
 pam_succeed_if(sshd:auth): error retrieving information about user outroots : 1 time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user blondies : 1 time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user suriname : 1 time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user pam : 34 time(s)

 .
 .

 pam_succeed_if(sshd:auth): error retrieving information about user lucas : 36 time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user return : 1 time(s)
 
 ---------------------- SSHD End -------------------------

Thanks again for any thoughts.
Meanwhile I'll check the appropriate sub-forum to check on how to remove my MTA (mail server) software.

anomie · 06-12-2009, 12:28 PM

Ok, so... people are doing nasty things to your server. Recommendations/questions (for now), in no particular order.

Read about how to lock down sshd (http://daemonforums.org/showthread.php?t=74), and then do so.
Since no one should be accessing your web server right now, you will want to control access at the IP level. iptables(8) can of course handle the filtering, or you can do this using Apache directives. Post your <Directory> configurations from httpd.conf.
Do you really require mod_proxy? If not, keep it turned off.
It would be best to not shut off your MTA. (You need it to get system email.) Just check that it is listening only on localhost with: $ netstat -ltn | grep ':25\>'
What other services are you intending to provide on the server? It may be that your current iptables ruleset is still too generous.

GwenE · 06-12-2009, 06:14 PM

Thank you anomie. The link you provide is great... I have my work cut out for me. I'm learning slowly but surely.

billymayday · 06-12-2009, 06:59 PM

To be safe, I'd add a rule to drop outgoing port 25 requests. If you have your MTA listening on localhost, you may still be open to local vulnerabilities.