First I would suggest to walk through the system maintenance step by step to ensure that the system is updated. There are some quirks regarding cPanel that this should be checked several times a year. Think of it as being about the same maintenance schedule as you would change the oil on your car.

WHM 11 cPanel - HowTo - General Maintenance
https://sites.google.com/site/zenars...al-maintenance

I am not so certain that enabling SELinux would be the answer as it is still too experimental and there have even been recent exploits that target SELinux.

A vulnerability in SELinux
http://marc.info/?l=selinux&m=105490085132101&w=2

As the spam is originating from the root user, they may have exploited some of the system notification scripts.

You would want to search against the Exim identifier to get more details on what all happened. Of course you may have to modify the command a little if the log files have already rotated.

Just don't...
 

// Apologies to the OP for this OT post but this needs to be corrected.

Sure SELinux was a b*tch to handle in the old days but it has improved over the past ten years. Hearsay and personal experience older than four years are no valid basis for making claims on. Your claim is unfounded, fuels misinformation and creates FUD and the case you refer to is stale (2003). Here's some Real Life SELinux mitigation cases:
2006 CVS-2006-3626 local privilege escalation stopped by targeted policy
2007 CVE-2007-2446, CVE-2007-2447 Samba heap overflow and arbitrary code execution (RHSA-2007-0354)
2007 CVE-2007-5208 hplip arbitrary command execution (CVE-2007-5208)
2007 CVE-2007-3304 Apache httpd 1.3.37, 2.0.59, and 2.2.4 Prefork local denial of service (RHSA-2007:0556-2)
2008 APR Flash bytecode exploit (matasano@wayback, PDF)
2008 CVE-2008-0003 OpenPegasus CIM arbitrary code execution (RHSA-2008:0002-7)
So please either do your research or keep yourself from posting things like that.

:):) super, I really had not will to provide more information and oponent information about SELinux back from 2003 :)

Thx

sorry for off-topic
:)

hello guys, i would like to close this thread if possible, as i've solved the problem.

there was data leak, it's all i can say about it, most important is, it's stopped now.


thank you all for help and sensible replies.

catworld's 10-16-10 03:35 PM reply was very close to the truth.

cheers!

Glad you got it sorted out. You might want to mark the thread as solved so others know.

Some folks look at long standing unsolved threads to see what's up. Others might hit the thread searching for a similar problem.

I'm currently having the same issue. Everything is patched, cpanel updated, exim updated. Same exact problem, same exact spam. What did you end up doing to fix this?

Have you looked into the 'social engineering' aspect? When the software is or at least appears to be running normally the first thing I look into is who's had their hands on the hardware. While I'm considering that I'll check for any newly reported exploits and also plan a check for the possibility of a rootkit. (requires downtime, not always the most welcome request)

Exploiting these kinds of things isn't the hardest task, but it's by no means something a script-kiddie bot is going to accomplish, so long as you are up to date and firewalls are well established. It's not a bad starting point to consider either someone has targeted you directly from the outside (trade secrets?) or from the inside. (espionage for same, or disgruntled (ex)employee?) If you see no untoward external traffic making it's way in via logs, it's a pretty safe bet someone has messed it up from the inside, whether intentionally or not..

Only I have access to this server, it is a colocated server so support staff of the datacenter does not have access (I have IP kvm). It's an openvz node with about 40 vz's. Several of the VZ's are penetrated each night around 4-5 am. So far the VZs have little in common, some have exim, some just have sendmail, some have cPanel and some have directadmin. The node has all the latest updates and runs solusvm + openvz. The VZ's have all the latest updates.

I initially posted this in the following thread last night, but it was split off by a mod: http://www.webhostingtalk.com/showthread.php?t=991510

My problem is identical to his. Same spam, same everything. I straced the process and it doesn't seem to do anything except spam through sendmail. I've checked for installed SSH keys, I've checked for changes to pam, checked /etc/passwd & /etc/shaddow. I also checked spool for any crontabs that have malicious data, nothing found :(. I have ossec running and monitoring those files for changes on the node and a couple of the VZs.

Hi,
Just wondering on this one, since you said it was a "data breach"

Can you be a bit more specific about what the file name was, and or contents of the script you found, etc?

I think it would be very helpful to others if you can provide some file specifics, as well as how you ended up solving the issue (that is formatting server, setting up a blocking command of some sort, etc.).

I read in another post on another forum that this was caused by an auto generating encrypted perl script found in / or /tmp, so curious what you learned.

Best Wishes,
Jim

What version of Exim? There was a serious root vulnerability discovered not long ago:

http://www.exim.org/lurker/message/2...32d4f2.en.html

Exim was patched on my servers. There are vps servers without exim that were also affected. Still no exact explanation for what happened, but os reload seems to have solved the problem. Perhaps it started with exim, but not sure how non-exim systems could have been affected. Exim was patched as I found out about the exploit some time before it was even public. What also concerns me is the fact that I couldn't ever find the method they were regaining access with (ie. the rootkit.)

Since the OP and you have consistently refused to share any useful information, since you have denied yourself any chance of investigation by "fixing" things and since I'm not into guessing any (further) reply that doesn't hold leads, useful information or clues is of no value at all.

Hi Dman1,
Were you able to learn anything more in your review?

I've experienced the same "Wanna Date?" hack and have to say it's quite maddening.

Even had all of the top cPanel support people review and they could not figure out how it was being generated from root (after over 4 hours of review on their part and another 20 hours on ours).

1. The spammer in this situation is somehow injecting the body of the email and list of addresses into the server root and sending them via perl as root.

2. Suffice it to say running every malware scanning script and file checking script known to man has turned up nothing out of the ordinary.

3. Even wrapping sendmail and logging to the nth degree did not help in identifying the source.

4. The hack itself injects up to 70k messages within the span of a few minutes, virtually the same time every day during a 2 hour window of time.

5. Other than the perl process appearing momentarily, watching the process list the microsecond messages are created and injected into the queue turns up no details.

6. Since we know when the spam is being injected we just freeze the queue for a couple hours, then the minute after the email is injected we just delete them all, so spammer is wasting their time and ours-- so this has turned in an academic issue at the moment.


- What we do know is that there is a script somewhere in root facilitating the sending of email through perl and sendmail.
- We know that some script on a site on server is being used to pass the information to the root level script.
- We know the spammer has a test script which emails an email alert to spammer approximately every 4 hours to check is server remains compromised (not a cron):
Subject: Notification # 6274
Body: All OK! ______ //h6vK3pJywC
- We know the script in root has a means of deleting itself, since we found a copy of the script on one occasion, and in later episodes look in the same location during the 5 minute spam injection and no scripts were found in that location or any other afterward.


So, if anyone has further evidence to share that would be helpful.


Best Wishes,
Jim

Hey,

I'm basically stuck at the same point you are at. The perl script definitely runs as root, I can see it sending mail if I attach to it with strace. Standard penetration testing gets me nowhere fast. My problems persisted and I've taken a similar approach to yours. On affected systems I have a script running that kills their process before it can send anything out. Pretty crappy solution, but working for the time being. I was initially concerned that this was a result of exim exploits, but now know that's not true since qmail and sendmail servers were affected as well. I would guess this is a remote root exploit on a common linux service. I have not experienced the secondary notification script you have seen. By default I provision servers and vps that run exim to have a lot of logging enabled to deal with spam complaints and I don't see any mail like that being sent.

I really don't know what else to add.

Just curious as to what ports are open on the effected server?

Also wondering if anyone who faces having to rebuild a server ground-up has considered using tripwire?

http://sourceforge.net/projects/tripwire/