Someone was sending email from my server using the 'nobody' account. I stopped the user 'nobody' from being able to send emails in CPanel WHM. He is still sending the emails but now they are being bounced back to me. I want to stop this guy. The suspected website is a game website and the users can email one another. So I think (maybe) that is the problem but don’t know how to stop him.Thanks.


Red Hat Enterprise Linux Server release 5.2 (Tikanga)

CPanel

Here are some results

[root@server public_html]# awk '{ if ($0 ~ "cwd" && $0 ~ "home") {print $4} }' /var/log/exim_mainlog | sort | uniq -c | sort -nk 1
139 3
153 2

410 cwd=/home/suspected website/public_html – this is where it is coming from (I think)



The funny thing is.. all the spam is being sent to another one of my email addresses. Here’s a header from one of the bounced emails.

Return-path: <nobody@server.myserver.com>
Received: from nobody by server.myserver.com with local (Exim 4.69)
(envelope-from <nobody@server.myserver.com>)
id 1Rst3O-0008Js-I1
for sm@mail.com; Thu, 02 Feb 2012 03:34:06 -0600
To: sm@mail.com
Subject: Discount Giuseppe Zanotti Pumps
Content-Type: text/plain
From: <Discount Giuseppe Zanotti Pumps>
Reply-To: Discount Giuseppe Zanotti Pumps
Message-Id: <E1Rst3O-0008Js-I1@server.myserver.com>
Sender: Nobody <nobody@server.myserver.com>
Date: Thu, 02 Feb 2012 03:34:06 -0600

Welcome to LQ Security,

This type of activity can result from a couple of different intrusion vectors; typically, the result of a file being uploaded to a location such as /tmp or an overwritten web page file (e.g.) PHP, or even a PHP file that does not properly sanitize user input. It is also important to realize that one does not need a lot of privilege to engage in this type of activity, as you are seeing with the user nobody appearing as a culprit.

A good way to tackle this problem is to stop the email and web services and carefully examine your web files for signs of modification. You should also look for any hidden files or strange scripts in locations such as /tmp which have relatively loose permissions by design. You should also look very closely at your log files for signs of intrusion. The tool Logwatch can help in this regard. Once you have obtained logwatch, run the following command: Ideally, you could copy the logs to a safe machine and run logwatch from there as it is best to not tamper with a possibly infected machine any more than you have to. You should also have a close look with your own eyes at your Apache and Exim logs.

Also, what applications are you running in your webstack, such as Apache, PHP, MySQL, any content management programs (e.g. Drupal), etc. and what version? How up to date is your system and have you been applying patches regularly?

Let me re-iterate that it is important for you to stop these services, or take the machine off line while you perform your investigation. Once you have found the source of the problem, you can make a determination as to the level of compromise and hence proceed to harden the machine as appropriate.

Thanks for the reply, noway2. I'm going to try your suggestions.

Nobody is the apache user, so:
In WHM, go to tweak settings, then enable:
Track email origin via X-Source email headers

Then if he spams again, it will give in the email headers the path (/home/whatever/www/something.php) the spam was being sent from.

Also you can click on Mail Queue Manager in WHM, some of the messages he sent are likely in there, look at the headers and it will show the path in there.

abefroman, thanks a lot for the reply. I've done as you suggested but the spamming has stopped it seems. I got one bounced email yesterday (before I enabled "Track email origin via X-Source email headers"). So I'm waiting for the next batch of spam.

After enabling "Track email origin via X-Source email headers" my server has received 2 bounced messages. Headers for one of these messages is below. Anyone have any idea what might be happening? Thanks.


Delivered-To: me#1@gmail.com - "Note: this email was delivered to me from my server"
Received: by 10.112.75.231 with SMTP id f7cs21483lbw;
Sat, 4 Feb 2012 23:48:18 -0800 (PST)
Received: by 10.101.2.32 with SMTP id e32mr5681702ani.13.1328428097089;
Sat, 04 Feb 2012 23:48:17 -0800 (PST)
Return-Path: <>
Received: from server.myserver.com ([xx.xx.xx.xx])
by mx.google.com with ESMTPS id d9si12050327yhn.109.2012.02.04.23.48.16
(version=TLSv1/SSLv3 cipher=OTHER);
Sat, 04 Feb 2012 23:48:16 -0800 (PST)
Received-SPF: neutral (google.com: xx.xx.xx.xx is neither permitted nor denied by best guess record for domain of server.myserver.com)

client-ip=xx.xx.xx.xx;
Authentication-Results: mx.google.com; spf=neutral (google.com: xx.xx.xx.xx is neither permitted nor denied by best guess record for domain

of server.myserver.com) smtp.mail=
Received: from mailnull by server.myserver.com with local (Exim 4.69)
id 1RtwpR-0005ic-L6
for nobody@server.myserver.com; Sun, 05 Feb 2012 01:48:05 -0600
X-Failed-Recipients: me#2@yahoo.com - "Note: email address spam is being sent to - one of my email addresses"
Auto-Submitted: auto-replied
From: Mail Delivery System <Mailer-Daemon@server.myserver.com>
To: nobody@server.myserver.com
Subject: Mail delivery failed: returning message to sender
Message-Id: <E1RtwpR-0005ic-L6@server.myserver.com>
Date: Sun, 05 Feb 2012 01:48:05 -0600
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - server.myserver.com
X-AntiAbuse: Original Domain - server.myserver.com
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Source:
X-Source-Args: /usr/local/apache/bin/httpd -k start -DSSL
X-Source-Dir: my-domain-on-my-server.com:/public_html - "Note: I've known directory from day 1 but don't know which file"

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its recipients.
This is a permanent error. The following address(es) failed:


me#2@yahoo.com - "Note: email address spam is being sent to - one of my email addresses"
Mail sent by user nobody being discarded due to sender restrictions in WHM->Tweak Settings

------ This is a copy of the message, including all the headers. ------

Return-path: <nobody@server.myserver.com>
Received: from nobody by server.myserver.com with local (Exim 4.69)
(envelope-from <nobody@server.myserver.com>)
id 1RtwpR-0005iX-Je
for me#2@yahoo.com; Sun, 05 Feb 2012 01:48:05 -0600
To: me#2@yahoo.com
Subject: DOUqlPNSvKbX
Content-Type: text/plain
From: jvdhqolofp <fvtznp@hriczt.com>
Reply-To: fvtznp@hriczt.com
Message-Id: <E1RtwpR-0005iX-Je@server.myserver.com>
Sender: Nobody <nobody@server.myserver.com>
Date: Sun, 05 Feb 2012 01:48:05 -0600


wWFooc <a href="http://ewldkxtawrtp.com/">ewldkxtawrtp</a>,
dsxkhijxzmpv,
[link=http://plrnqcmsqhha.com/]plrnqcmsqhha
[/link], http://rzwkriqhjpka.com/

If you know the directory, that narrows it down a lot. Any 3rd party scripts on that domain? Look at the php files with in it, to see which send mail. Are there any that have obfucasted code, or look like a php shell script left by the hacker?

I finally found out what is happening. He broke into one of my sites. Not my server, just my game website. He broke into the admin part of control panel of the game script. I've changed all the passwords. If it starts happening again I will call in the professionals. But I don't know what all he has done and how he got the password. It definitely wasn't an easy password. This is the first time this has happened to me in nearly 15 years having websites (that I know of). And he didn't seem to be malicious (knock on wood). I guess I will need to wait until I see if it happens again.

Oh, one other question that I am concerned about. I changed the password to the control panel of the game website (and all others). But what if he leaves his browser open and he's logged into the control panel? And restarted the server. Will he still be able to, maybe, go in and change the password or something of that nature? Thanks.

I am really stupid. It took me awhile to catch on. This is a game website. I have it just as a complimentary service for my customers on another website. It's a game script I purchased and really haven't paid that much attention to what it does. I have a "Contact" link on the website. This person is sending the spam from that. It sends an email to my admin email address and "nobody" on my server sends the email. I think that is the problem. 3 or 4 days wasted looking into this. Oh well, I learned a few things. Thanks to everyone.

My gut instinct would be the game is vulnerable, and he found a way to bypass the login to the admin section, especially if you had a strong password.

You should check the site you to the software from to see if they have an update/know about the issue.

If the admin section is a directory, you can also protect that with an ACL in a .htaccess file. But not that won't help if public parts if the game are vulnerable.

There are a couple of lessons here:
1 - Any administrative interface needs to be blocked from general public access. Client certificates are a good option here. Other ways to accomplish this are to put the admin host on the localhost (only) address and use SSH to establish a tunnel to the server, then access the webpage. Since SSH can be locked down rather well, this has the side effect of enhanced security. As abefroman mentioned, this can be done in .htaccess or .htaccess in combination with changes to the host configuration file.

2 - Contact Us or other types of links can be vulnerable, especially if they operate via a mail server. All input from the public must be sanitized by reducing it to standard characters, length limiting to prevent buffer overflow, and then comparing the results against known good values before acceptance.

3 - Another possibility, besides a vulnerability in your program, is a brute force breaking of your password. Consider using a tool like fail2ban that will block such attempts after a few tries. This will raise the time to entry bar sufficiently to make attempting access to your site less attractive than others out there.

4 - Consider using modsecurity, which will analyze the packets and react to malicious attempts. Note that it takes a lot of effort to configure it before it will work properly for you and it defaults to being very strict.

5 - If all else fails, you can apply some rate limiting to your server which will prevent someone from trying to establish an excessive number of connections simultaneously. It has been my experience that game servers often times have a problem with this. Before you go this route it is vitally important to analyze the traffic as there are other approaches that may work better depending on what is happening. I mention it here to bring awareness only, not as a recommendation for you for now.