LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 10-16-2010, 08:08 PM   #16
joec@home
Member
 
Registered: Sep 2009
Location: Galveston Tx
Posts: 291

Rep: Reputation: 70

Quote:
Originally Posted by pr1soner View Post
all packages on this server are up to date, including kernel.
First I would suggest to walk through the system maintenance step by step to ensure that the system is updated. There are some quirks regarding cPanel that this should be checked several times a year. Think of it as being about the same maintenance schedule as you would change the oil on your car.

WHM 11 cPanel - HowTo - General Maintenance
https://sites.google.com/site/zenars...al-maintenance

I am not so certain that enabling SELinux would be the answer as it is still too experimental and there have even been recent exploits that target SELinux.

A vulnerability in SELinux
http://marc.info/?l=selinux&m=105490085132101&w=2

As the spam is originating from the root user, they may have exploited some of the system notification scripts.

Quote:
Originally Posted by pr1soner View Post
two example exim's log lines:
2010-10-14 16:29:51 [24688] 1P6Pkd-0006QC-8o <= root@server.XXX.XXX U=root P=local S=635 T="Hi Joe Smith. It's Glenda. Wanna date?" from <root@server.XXX.XXX> for at18sharks@aim.com
2010-10-14 16:29:51 [24691] 1P6Pkd-0006QF-9B <= root@server.XXX.XXX U=root P=local S=659 T="Hello Pothead666. It's Charlene. Wanna date?" from <root@server.XXX.XXX> for bielfeedback666@yahoo.com
You would want to search against the Exim identifier to get more details on what all happened. Of course you may have to modify the command a little if the log files have already rotated.

Code:
egrep 1P6Pkd-0006QC /var/log/exim_mainlog
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 10-17-2010, 05:04 AM   #17
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,409
Blog Entries: 55

Rep: Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582
Just don't...

// Apologies to the OP for this OT post but this needs to be corrected.

Quote:
Originally Posted by joec@home View Post
I am not so certain that enabling SELinux would be the answer as it is still too experimental and there have even been recent exploits that target SELinux.
Sure SELinux was a b*tch to handle in the old days but it has improved over the past ten years. Hearsay and personal experience older than four years are no valid basis for making claims on. Your claim is unfounded, fuels misinformation and creates FUD and the case you refer to is stale (2003). Here's some Real Life SELinux mitigation cases:
2006 CVS-2006-3626 local privilege escalation stopped by targeted policy
2007 CVE-2007-2446, CVE-2007-2447 Samba heap overflow and arbitrary code execution (RHSA-2007-0354)
2007 CVE-2007-5208 hplip arbitrary command execution (CVE-2007-5208)
2007 CVE-2007-3304 Apache httpd 1.3.37, 2.0.59, and 2.2.4 Prefork local denial of service (RHSA-2007:0556-2)
2008 APR Flash bytecode exploit (matasano@wayback, PDF)
2008 CVE-2008-0003 OpenPegasus CIM arbitrary code execution (RHSA-2008:0002-7)
So please either do your research or keep yourself from posting things like that.
 
1 members found this post helpful.
Old 10-17-2010, 08:16 AM   #18
sarajevo
Member
 
Registered: Apr 2005
Distribution: Debian, OpenBSD,Fedora,RedHat
Posts: 228
Blog Entries: 1

Rep: Reputation: 31
Quote:
Originally Posted by unSpawn View Post
// Apologies to the OP for this OT post but this needs to be corrected.


Sure SELinux was a b*tch to handle in the old days but it has improved over the past ten years. Hearsay and personal experience older than four years are no valid basis for making claims on. Your claim is unfounded, fuels misinformation and creates FUD and the case you refer to is stale (2003). Here's some Real Life SELinux mitigation cases:
2006 CVS-2006-3626 local privilege escalation stopped by targeted policy
2007 CVE-2007-2446, CVE-2007-2447 Samba heap overflow and arbitrary code execution (RHSA-2007-0354)
2007 CVE-2007-5208 hplip arbitrary command execution (CVE-2007-5208)
2007 CVE-2007-3304 Apache httpd 1.3.37, 2.0.59, and 2.2.4 Prefork local denial of service (RHSA-2007:0556-2)
2008 APR Flash bytecode exploit (matasano@wayback, PDF)
2008 CVE-2008-0003 OpenPegasus CIM arbitrary code execution (RHSA-2008:0002-7)
So please either do your research or keep yourself from posting things like that.
super, I really had not will to provide more information and oponent information about SELinux back from 2003

Thx

sorry for off-topic
 
Old 10-20-2010, 01:30 PM   #19
pr1soner
LQ Newbie
 
Registered: May 2010
Location: earth
Distribution: slackware
Posts: 10

Original Poster
Rep: Reputation: 1
hello guys, i would like to close this thread if possible, as i've solved the problem.

there was data leak, it's all i can say about it, most important is, it's stopped now.


thank you all for help and sensible replies.

catworld's 10-16-10 03:35 PM reply was very close to the truth.

cheers!
 
0 members found this post helpful.
Old 10-20-2010, 01:55 PM   #20
catworld
Member
 
Registered: Nov 2004
Location: Horseheads, New York
Distribution: Mandriva 2010.1 / KDE 4.5.2, Slax, Knoppix, Backtrack & etc...
Posts: 198

Rep: Reputation: 36
Glad you got it sorted out. You might want to mark the thread as solved so others know.

Some folks look at long standing unsolved threads to see what's up. Others might hit the thread searching for a similar problem.
 
Old 12-19-2010, 04:58 AM   #21
dman1
LQ Newbie
 
Registered: Dec 2010
Posts: 16

Rep: Reputation: 0
I'm currently having the same issue. Everything is patched, cpanel updated, exim updated. Same exact problem, same exact spam. What did you end up doing to fix this?

Last edited by unSpawn; 01-04-2011 at 11:02 AM. Reason: //Merge back as this seems persistent.
 
Old 12-19-2010, 09:06 AM   #22
catworld
Member
 
Registered: Nov 2004
Location: Horseheads, New York
Distribution: Mandriva 2010.1 / KDE 4.5.2, Slax, Knoppix, Backtrack & etc...
Posts: 198

Rep: Reputation: 36
Have you looked into the 'social engineering' aspect? When the software is or at least appears to be running normally the first thing I look into is who's had their hands on the hardware. While I'm considering that I'll check for any newly reported exploits and also plan a check for the possibility of a rootkit. (requires downtime, not always the most welcome request)

Exploiting these kinds of things isn't the hardest task, but it's by no means something a script-kiddie bot is going to accomplish, so long as you are up to date and firewalls are well established. It's not a bad starting point to consider either someone has targeted you directly from the outside (trade secrets?) or from the inside. (espionage for same, or disgruntled (ex)employee?) If you see no untoward external traffic making it's way in via logs, it's a pretty safe bet someone has messed it up from the inside, whether intentionally or not..
 
Old 12-19-2010, 01:41 PM   #23
dman1
LQ Newbie
 
Registered: Dec 2010
Posts: 16

Rep: Reputation: 0
Only I have access to this server, it is a colocated server so support staff of the datacenter does not have access (I have IP kvm). It's an openvz node with about 40 vz's. Several of the VZ's are penetrated each night around 4-5 am. So far the VZs have little in common, some have exim, some just have sendmail, some have cPanel and some have directadmin. The node has all the latest updates and runs solusvm + openvz. The VZ's have all the latest updates.

I initially posted this in the following thread last night, but it was split off by a mod: http://www.webhostingtalk.com/showthread.php?t=991510

My problem is identical to his. Same spam, same everything. I straced the process and it doesn't seem to do anything except spam through sendmail. I've checked for installed SSH keys, I've checked for changes to pam, checked /etc/passwd & /etc/shaddow. I also checked spool for any crontabs that have malicious data, nothing found . I have ossec running and monitoring those files for changes on the node and a couple of the VZs.

Last edited by dman1; 12-19-2010 at 01:44 PM.
 
Old 01-04-2011, 10:14 AM   #24
tvcnet
LQ Newbie
 
Registered: Jan 2011
Posts: 7

Rep: Reputation: 0
Hi,
Just wondering on this one, since you said it was a "data breach"

Can you be a bit more specific about what the file name was, and or contents of the script you found, etc?

I think it would be very helpful to others if you can provide some file specifics, as well as how you ended up solving the issue (that is formatting server, setting up a blocking command of some sort, etc.).

I read in another post on another forum that this was caused by an auto generating encrypted perl script found in / or /tmp, so curious what you learned.

Best Wishes,
Jim
 
Old 01-04-2011, 01:22 PM   #25
120
Member
 
Registered: Oct 2010
Posts: 46

Rep: Reputation: 9
What version of Exim? There was a serious root vulnerability discovered not long ago:

http://www.exim.org/lurker/message/2...32d4f2.en.html
 
Old 01-11-2011, 10:42 PM   #26
dman1
LQ Newbie
 
Registered: Dec 2010
Posts: 16

Rep: Reputation: 0
Exim was patched on my servers. There are vps servers without exim that were also affected. Still no exact explanation for what happened, but os reload seems to have solved the problem. Perhaps it started with exim, but not sure how non-exim systems could have been affected. Exim was patched as I found out about the exploit some time before it was even public. What also concerns me is the fact that I couldn't ever find the method they were regaining access with (ie. the rootkit.)
 
Old 01-12-2011, 01:25 PM   #27
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,409
Blog Entries: 55

Rep: Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582Reputation: 3582
Since the OP and you have consistently refused to share any useful information, since you have denied yourself any chance of investigation by "fixing" things and since I'm not into guessing any (further) reply that doesn't hold leads, useful information or clues is of no value at all.

Last edited by unSpawn; 01-12-2011 at 01:26 PM.
 
Old 01-19-2011, 11:53 AM   #28
tvcnet
LQ Newbie
 
Registered: Jan 2011
Posts: 7

Rep: Reputation: 0
Hi Dman1,
Were you able to learn anything more in your review?

I've experienced the same "Wanna Date?" hack and have to say it's quite maddening.

Even had all of the top cPanel support people review and they could not figure out how it was being generated from root (after over 4 hours of review on their part and another 20 hours on ours).

1. The spammer in this situation is somehow injecting the body of the email and list of addresses into the server root and sending them via perl as root.

2. Suffice it to say running every malware scanning script and file checking script known to man has turned up nothing out of the ordinary.

3. Even wrapping sendmail and logging to the nth degree did not help in identifying the source.

4. The hack itself injects up to 70k messages within the span of a few minutes, virtually the same time every day during a 2 hour window of time.

5. Other than the perl process appearing momentarily, watching the process list the microsecond messages are created and injected into the queue turns up no details.

6. Since we know when the spam is being injected we just freeze the queue for a couple hours, then the minute after the email is injected we just delete them all, so spammer is wasting their time and ours-- so this has turned in an academic issue at the moment.


- What we do know is that there is a script somewhere in root facilitating the sending of email through perl and sendmail.
- We know that some script on a site on server is being used to pass the information to the root level script.
- We know the spammer has a test script which emails an email alert to spammer approximately every 4 hours to check is server remains compromised (not a cron):
Subject: Notification # 6274
Body: All OK! ______ //h6vK3pJywC
- We know the script in root has a means of deleting itself, since we found a copy of the script on one occasion, and in later episodes look in the same location during the 5 minute spam injection and no scripts were found in that location or any other afterward.


So, if anyone has further evidence to share that would be helpful.


Best Wishes,
Jim
 
Old 01-20-2011, 02:39 PM   #29
dman1
LQ Newbie
 
Registered: Dec 2010
Posts: 16

Rep: Reputation: 0
Hey,

I'm basically stuck at the same point you are at. The perl script definitely runs as root, I can see it sending mail if I attach to it with strace. Standard penetration testing gets me nowhere fast. My problems persisted and I've taken a similar approach to yours. On affected systems I have a script running that kills their process before it can send anything out. Pretty crappy solution, but working for the time being. I was initially concerned that this was a result of exim exploits, but now know that's not true since qmail and sendmail servers were affected as well. I would guess this is a remote root exploit on a common linux service. I have not experienced the secondary notification script you have seen. By default I provision servers and vps that run exim to have a lot of logging enabled to deal with spam complaints and I don't see any mail like that being sent.

I really don't know what else to add.
 
Old 01-21-2011, 08:57 AM   #30
catworld
Member
 
Registered: Nov 2004
Location: Horseheads, New York
Distribution: Mandriva 2010.1 / KDE 4.5.2, Slax, Knoppix, Backtrack & etc...
Posts: 198

Rep: Reputation: 36
Just curious as to what ports are open on the effected server?

Also wondering if anyone who faces having to rebuild a server ground-up has considered using tripwire?

http://sourceforge.net/projects/tripwire/


Quote:
Originally Posted by dman1 View Post
Hey,

I'm basically stuck at the same point you are at. The perl script definitely runs as root, I can see it sending mail if I attach to it with strace. Standard penetration testing gets me nowhere fast. My problems persisted and I've taken a similar approach to yours. On affected systems I have a script running that kills their process before it can send anything out. Pretty crappy solution, but working for the time being. I was initially concerned that this was a result of exim exploits, but now know that's not true since qmail and sendmail servers were affected as well. I would guess this is a remote root exploit on a common linux service. I have not experienced the secondary notification script you have seen. By default I provision servers and vps that run exim to have a lot of logging enabled to deal with spam complaints and I don't see any mail like that being sent.

I really don't know what else to add.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
server compromised? eco Linux - Security 3 09-03-2010 11:58 AM
my server has been compromised, what next? Kropotkin Linux - Security 15 08-27-2009 06:15 AM
Server Compromised? lss1 Linux - Security 7 12-16-2005 12:49 AM
Server Compromised? stlyz3 Linux - Security 6 09-07-2005 04:28 PM
Server was compromised, need help Asiana Linux - Security 3 06-02-2004 12:39 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 01:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration