Server compromised (exim headers) ?
Welcome everybody (this is my first post) !
Today i noticed that there was spam sent from my server (centos5 + cpanel) from headers it looks like the spam was sent as root: X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [47 12] all packages on this server are up to date, including kernel. Is it possible to send e-mail message so these headers will show that the Originator's UID/GID is 0/0 ? but as _regular_ user ? if it's possible then there is still hope that the server is not hacked with root prvileges, but if it's not possible, that would indicate hacked server with root privs... i have tried to send e-mail from regular user's account but when doing so, in headers i see: X-AntiAbuse: Originator/Caller UID/GID - [558 555] / [47 12] |
Quote:
Quote:
You could: - generate detailed listings of processes, open files, users and network connections (host, pastebin or attach as plain text?), - stop your web server and MTA while investigating, - check user login records (just in case), - run all system and daemon logs through Logwatch without service exclusions and using "--Archives" (generate leads), - look for any network-using processes that run as UID root or - have odd CWDs like /var/www or - have odd process names (like your httpd is /usr/sbin/httpd but you see "/usr/local/bin/apache -DSSL"), - search your docroot(s) for any mailforms, - check if any web log, forum, web mail, statistics or other applications you run in your web stack have known LFI/RFI/other net-attackable vulnerabilities. |
Thank you for your reply.
Quote:
i'm going to use your hints, hope to find some more info, thanks for it. |
Possible 0day attack ?
Greetings,
this is my fourth server affected in past 15 days (Centos5 + cpanel), as the result, spam is being send by the root user: headers: X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.XXX.XXX X-AntiAbuse: Original Domain - tmail.com X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [47 12] X-AntiAbuse: Sender Address Domain - server.XXX.XXX two example exim's log lines: 2010-10-14 16:29:51 [24688] 1P6Pkd-0006QC-8o <= root@server.XXX.XXX U=root P=local S=635 T="Hi Joe Smith. It's Glenda. Wanna date?" from <root@server.XXX.XXX> for at18sharks@aim.com 2010-10-14 16:29:51 [24691] 1P6Pkd-0006QF-9B <= root@server.XXX.XXX U=root P=local S=659 T="Hello Pothead666. It's Charlene. Wanna date?" from <root@server.XXX.XXX> for bielfeedback666@yahoo.com as far as i know, this header can't be faked by regular user: X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [47 12] so it must be really root that sends this spam. all Centos packages are up to date and ksplice (rebotlees kernel updates) are in effect (if that have any matter). for me it's odd that someone (or some scanner) exploit a server with root privileges just to send spam. have anyone exprienced this problem ? or have any ideas or anything ? thanks upfront. |
Quote:
It's not clear from the information you posted; are these mails being sent via your mail server, or are these system emails from the root user destined for the terminal? If the machines are rooted (a root kit installed) or even wormed the first thing I would suspect, unfortunately, is ksplice. When they announced it's release I looked into it and had doubts as to it's security. Before anyone here can suggest possible mitigation we'd need to know the answers to the above. It's unusual for a root system account to be compromised. Usually it's the server app, rarely is anything in the underlying system hit. It can be done, but normally through the server(s) being exposed. I would be quite disturbed if the server was healthy but the root account has been cracked. What other servers and/or services are open on these machines? Please tell me not telnet... |
I'm going to merge this thread with your previous one to keep discussions in one place.
|
Quote:
Quote:
Quote:
"I don't believe we're aware of any known exploits going around." is their reply. Obviously i can't verify that info ;-) Quote:
Quote:
Quote:
no telnet :) |
Quote:
Is the server hosting a SQL database that's accessible from the outside? If it's only a backend to something apache is doing you should close off the port. Have you checked logs to see if any untoward burst of activity occurred seeking to connect to any of those services? You might want to run snort on the thing for a while and see if whoever comes back, once you shut off the mass mailings. I'd run wireshark on the internet facing adapter to see which, if any, port or ports are the vector of attack. If the body of the mails that are going out are not files residing on the hard drive then the content of each has to be coming in through an exploited channel along with the mail command that's sending them. That ought to be very easy to see with wireshark. Another tool I can't live without is iptstate. I believe there's a .deb for that. It's a console tool that shows all active connections in the heart of the IP stack, the addresses connected and on which port. My first go-to tool when I log in to a remote server for a hands-on session I run iptstate. I can tell right off if someone unauthorized is present, that snort or the mail subsys might not be working etc. Lastly, ftp. Is this for admins? Or is there a public use? I always shudder when there is a requirement for ftp to be facing the public. It's hard as heck to secure, harder to move off standard ports and there's no getting around passwords... in-the-clear usually, the script kiddies have a field day with dictionary attacks on ftp. If it's strictly for admins to update content, I'd use rsync over ssh, or even consider sshfs. (I have set up quite a few systems to utilize sshfs, never seen anyone poking at it, and I almost always move ssh off port 22 to thwart the automated mayhem) The only time I ever expose cups to the internet is on a honeypot. BTW do you run X on this machine? That is a major vector for compromise. I have never run X (let alone a desktop environment) on a public-facing server. Someone may have been able to access one of your services via 'normal,' apparently legitimate means then compromised the system via X. |
catworld, i am sure you want to deliver best answer, and your hints and points are valuable, but i know this all, have dealt with such cases so many times before that i can't even count it all. i have investigated all these services that are open to the public and if even one of them would be affected, i would figure that out, keep on mind i posted here after trying just about _everything_ that was possible to do in the production environment.
the original question was if anyone from the huge LQ community is aware of such problem - the spammer who gets root on latest centos5 machines, just to send spam. everything that is needed for someone to recognize such attack is already posted before, this must be 0day and i even caught the attacker's IP, perl process responsible for spam, debugging the process and attaching gdb to the pid gave me nothing but ptrace: Operation not permitted. Well, even if i have the source code of program that is sending spam i will find, hm, how he is sending it? that's useless, i'm trying to search for way he exploited it at first place, this is most important, and i believe this will hit others soon, not only me. do you recognize this IP ? : 65.110.42.90 the 6th machine that got owned, recorded this IP and that's all we have for now, except the AntiAbuse headers, btw the perl process was running as root, so it's confirmed now to be root exploit. sure there is a chance for password leak from third party or even sabotage possibility, but at this point i simply don't believe on that. i believe it's 0day, and someone must know something. whatever it is, thank you, catworld, for trying to help. appreciate. awaiting some news, hopefully. |
so not knowing who/where you are, the real answer must be you have responsibility for protecting something someone who can't be denied wants, or someone who is paying you for the same pissed off the wrong people, whomever/wherever that may be.
There is no "Oday" in public, so it must be gift wrapped specially for you. You'd have to provide a few tons more information for any useful help, but Uruguay is always a good answer. Best answer? |
Quote:
Quote:
As for your original question, can email header UIDs be spoofed, yes they can. Spammers can edit anything in the header. |
Quote:
Quote:
Quote:
if i own some machine with root access i would send spam as regular user (nobody/mailnull, etc) to keep the root access with me, in contradiction: if i have just regular user access i would do everything i can so the attack would look like coming from root, so the one who's investigating it focus on something else than he should. yes i asked 'whois' for that ip and sent email to the abuse with no reply so far, still waiting but not expecting any answer as none returned to me yet. Quote:
Uruguay? are you ok there ? |
Quote:
At this point you have a choice and you can turn this whole thing around. But that means no more interpretations, no more descriptions of what you think you are seeing and no more wild goose chases but providing detailed (anonymized) information like log excerpts, tool output and reporting from running checks from the CERT Intruder Detection Checklist that may help us help you. Please keep that in mind. |
Sorry, I'm too cryptic sometimes. What I was trying to put across is you should consider the "social engineering" aspect.
For instance, if you are running this for a widget manufacturer, you may suspect other widget manufacturers to be more interested in you than any random attacker. Add to that a possible exacerbating situation; did your company just fire somebody that has enough knowledge of your systems to compromise them, or at least enlist someone else to do so? Conversely, did your company just head-hunt someone from the competition, that would scare them enough to take illicit measures to try to figure out what's going on? (look at the recent Oracle/HP executive debacle) The mailings may not be the actual goal of the attacks, in fact it may well be an inverse honey pot. You'd find this unwarranted email traffic, eliminate it, watch for a while and see that it's indeed gone, and think you've dealt with the compromise, when actually a rootkit or back door remains. One thing I said is better stated thusly: did the entity you work for, or who's servers these are, or anyone associated with the reason for these systems existing say or do anything that angered someone else? Or perhaps 'intrigued' someone else to where they'd target you? EG maybe the CEO said something cryptic about a release in the next quarter and that shareholder are going to be very happy about it... such that someone in the industry (whatever it is) would take any measure to find out what's going on. I suggest you look into potential reasons for all this, because it does seem you are being directly, specifically targeted, rather than suffering from your typical skript-kiddie garbage. You didn't specifically say whether you ran wireshark and iptstate on the outside interface. You'd have the info the abuse handler at the remote ISP would need to sniff out a more specific source... which I will tell you he/she will hope is outside of their domain, eg the IP is either spoofed or is merely a relay. As for Uruguay, that was a highly angular non-sequitur. My wife understands me but that's about as far as that goes. But based on what you have provided by way of information, I still say "Uruguay" is as valid an answer as any other. :D |
Was SELinux in enforcing mode on affected machines ? After reading all comments of original poster I would say either there is social engineering/someone fired recently, or unskilled server maintenence
Cheers |
All times are GMT -5. The time now is 04:39 PM. |