LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices

Reply
 
Search this Thread
Old 01-08-2009, 05:32 AM   #1
lexthoonen
LQ Newbie
 
Registered: Jan 2007
Posts: 19

Rep: Reputation: 0
Server hacked, how to find out how they did it


Hi, I've seen that someone installed 'rapidleech' on my server, and some vnware. The (web) server hosts a few domains, but their stuff is in two folders on the same domain. I've checked log files, and it seems they didn't really log on to the system. So I checked the upload folders of the software on the domain, as it runs tikiwiki, gallery2 and on other domains joomla and wordpress. I didn't find anything suspicious. Safe mod is off on the server, but before turning it on I'd like to find out how they got on to it. Sofar, (s)he is only using it to download files to the server and then downloading it from the server. If I delete the folders, (s)he knows I've seen it and might do something worse.

I guess my priority is finding out how (s)he got on the server.

My question: what would you people do if you were me in this case?

Thanks!

p.s. the server runs ubuntu, apache, mysql.

Last edited by lexthoonen; 01-08-2009 at 05:33 AM.
 
Old 01-08-2009, 07:32 AM   #2
Hangdog42
LQ Veteran
 
Registered: Feb 2003
Location: Maryland
Distribution: Slackware
Posts: 7,785
Blog Entries: 1

Rep: Reputation: 414Reputation: 414Reputation: 414Reputation: 414Reputation: 414
Usually one of the best places to start is the CERT checklist

From following similar threads here, I've seen that if you want to find out how you were compromised you need to preserve the situation (i.e. don't reboot, but pull the network plug and if you need to shut down, pull the power cord) and then find a way to either put this machine in a place where you can investigate or create a copy of the machine you can investigate. There are threads with advice from much more knowledgeable people than me, so have a search in this forum.

Last edited by Hangdog42; 01-08-2009 at 07:35 AM.
 
Old 01-08-2009, 07:34 AM   #3
kentyler
Member
 
Registered: Dec 2008
Location: Cleveland Ohio
Distribution: CentOS
Posts: 235

Rep: Reputation: 36
Are you sure that the software you found is installed and they are doing something with it? It's possible that the software was downloaded and not used.
 
Old 01-08-2009, 07:57 AM   #4
Worksman
Member
 
Registered: Sep 2004
Location: Romania
Distribution: Ubuntu, Debian, Arch Linux, Gentoo, Slackware
Posts: 171
Blog Entries: 1

Rep: Reputation: 31
Well, it hardly makes sense. If you don't see anyone had logged on to the system, no-one could have been logged on. Check the current user accounts. See if there is any unknown user installed. If not, they probably have your user credentials (your key or your password) so I suggest you change those. Or maybe someone else's user credentials from your server. Be sure no-one else has sudo access but you.

Also don't forget to get the latest security updates for your server.
Turn off the sshd service (or any remote access/log on system), see if that someone is still doing stuff on your server.
 
Old 01-08-2009, 08:41 AM   #5
sparc86
Member
 
Registered: Jul 2006
Location: Joinville, Southern Brazil
Distribution: Debian, CentOS
Posts: 296

Rep: Reputation: 31
You should do all the stuff these guys told above. I would also suggest you to check the firewall logs (if you have a working firewall, of course).
 
Old 01-08-2009, 09:12 AM   #6
svalovic
Member
 
Registered: Mar 2004
Location: Slovakia
Distribution: debian lenny, squeeze, Ubuntu Netbook Remix, Puppy
Posts: 32

Rep: Reputation: 16
My tip is sudo. I think that ubuntu uses sudo for all users by default and if you haven't changed that, all your local users have admin access.
 
Old 01-08-2009, 09:31 AM   #7
farslayer
Guru
 
Registered: Oct 2005
Location: Willoughby, Ohio
Distribution: linuxdebian
Posts: 7,228
Blog Entries: 5

Rep: Reputation: 189Reputation: 189
I would look at the versions of the installed apps on the server. My Joomla site was compromised recently and files were uploaded to the server and script was added to the sites front page, because the version of Joomla on the site was out of date.

(Version 1.51 vs 1.58) Per my host Joomla prior to 1.56 is unacceptable.

The other apps on your server are just as suspect if they are out of date.
 
Old 01-08-2009, 10:58 AM   #8
lexthoonen
LQ Newbie
 
Registered: Jan 2007
Posts: 19

Original Poster
Rep: Reputation: 0
Thanks for the replies!

With this:

find /var/www/ -name "*".php -type f -print0 | xargs -0 grep r57 | uniq -c | sort -u | cut -d":" -f1

I found r57 on my server, on more places. With r57, they've been able to install RapidLeech which seems all they are using (and the VMWare to cover their tracks). By the way, it's been since april that r57 has been installed...

Now I've got to find out how they were able to install r57...
 
Old 01-08-2009, 11:01 AM   #9
lexthoonen
LQ Newbie
 
Registered: Jan 2007
Posts: 19

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by svalovic View Post
My tip is sudo. I think that ubuntu uses sudo for all users by default and if you haven't changed that, all your local users have admin access.

There are only a few users on the server, and they own a domain so could easily install this on their own domain...

(if I understand what you're saying that is)
 
Old 01-08-2009, 11:12 AM   #10
lexthoonen
LQ Newbie
 
Registered: Jan 2007
Posts: 19

Original Poster
Rep: Reputation: 0
I think i found it

i found lots of files in an 'image upload' section of tikiwiki, that shouldn't be there.

so, next plan of action is probably disabling file uploads.

getting rid of all r57 instances and then seeing if the server can work with php safe mode on.
 
Old 01-08-2009, 12:13 PM   #11
unixfool
Member
 
Registered: May 2005
Location: Northern VA
Distribution: Slackware, Ubuntu, FreeBSD, OpenBSD, OS X
Posts: 781
Blog Entries: 8

Rep: Reputation: 157Reputation: 157
Application vectors (ie, gaining access via a php-enabled website or via SQL) suck, but that's the risk you take in using such software. It is also the reason you didn't see anyone logging into your system...they didn't have to...they got in through an unsecure app. This could've happened because of buggy software, or the way you implemented the software.
 
Old 01-08-2009, 04:08 PM   #12
Hangdog42
LQ Veteran
 
Registered: Feb 2003
Location: Maryland
Distribution: Slackware
Posts: 7,785
Blog Entries: 1

Rep: Reputation: 414Reputation: 414Reputation: 414Reputation: 414Reputation: 414
Quote:
I think i found it
Evidence? This is one situation where you shouldn't "think" you know something. If you're not dealing in facts, you stand a very, very good chance of going astray.
Quote:
i found lots of files in an 'image upload' section of tikiwiki, that shouldn't be there.
With all due respect, that is likely just a symptom. Like I said before, you need facts and analysis, not guesswork.

Quote:
so, next plan of action is probably disabling file uploads.
Uh, no. At this point, YOU HAVE ABSOLUTELY NO IDEA WHAT HAS HAPPENED. There could be multiple entry points that you don't know about and can't see. You simply cannot trust what your machine is telling you.

I would strongly suggest taking this machine offline, pulling the power plug and then figuring out the proper way to diagnose what has happened. Everything else is just wasting time.

Last edited by Hangdog42; 01-08-2009 at 04:09 PM.
 
Old 01-08-2009, 05:14 PM   #13
Worksman
Member
 
Registered: Sep 2004
Location: Romania
Distribution: Ubuntu, Debian, Arch Linux, Gentoo, Slackware
Posts: 171
Blog Entries: 1

Rep: Reputation: 31
Wink

Quote:
Originally Posted by svalovic View Post
My tip is sudo. I think that ubuntu uses sudo for all users by default and if you haven't changed that, all your local users have admin access.
Negative. Only the user you create at instalation has sudo priviledges.
Use the visudo/vimsudo command to edit the sudoers list.
In general, and on Ubuntu, only users part of a certain group have sudo rights.
This group is probably 'adm' or 'admin'. vimsudo should tell you.

To exit without saving once you run vimsudo, for vim non-aware people, press ESC once, type q! and press enter. TIP: only root can modify the sudoers list or someone with root priviledges.

HTH

Last edited by Worksman; 01-08-2009 at 05:15 PM. Reason: another typo :)
 
Old 01-08-2009, 05:56 PM   #14
dguitar
Member
 
Registered: Jun 2005
Location: Portland, ME
Distribution: Slackware 13, CentOS 5.3, FBSD 7.2, OBSD 4.6, Fedora 11
Posts: 122

Rep: Reputation: 17
One thing that isn't always made clear in these types of threads is you need to do one major thing:

You will need to reinstall the OS.

But, you first need to figure out how they got in, otherwise you'll have solved nothing...
 
Old 01-08-2009, 06:01 PM   #15
lexthoonen
LQ Newbie
 
Registered: Jan 2007
Posts: 19

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Hangdog42 View Post
With all due respect, that is likely just a symptom. Like I said before, you need facts and analysis, not guesswork.
Thanks for all your advice people. I have learnt quite a lot today, quite a bit thanks to you all.

I have to admit, I haven't been very clear. I guess (I still do as you can see ) I'm in the middle somewhere between where you think I am and where you think I should be.

In tikiwiki, in an upload part, I found r57, a pretty good program to do whatever you want on a server once you have access to it.
On the server, in php.ini "allow_url_fopen" was made available, not a very wise thing to do.

I found out, reading the logs, that 'RapidLeech' was a programme that has really been used a lot lately. No excessive amount of mails have been sent nor seem websites altered in any way compared to backups except for here and there a few extra "r57" or "c99" (similar program) folders. (By the way, these weren't called like that but with the find command they were easily spotted. As well it was a combination of checking owners/group, size and looking for standard files that I spotted.) I found them on 2 domains on the site.

So, in a few minutes, I'll switch off the webserver. Can't really pull the plug as the machine is more or less 4500 km away. I'll get rid of all instances I've found, I've already been updating the software on it to the latest versions, have deleted a lot of unused folders of old versions of the software. Once that's done, it'll have to go back online again. In the worst case, I got backups, but I can't let the sites be down for a long time you see. Then, I'll have to monitor them (and the logs) pretty good to see if anything suspicious is going on. I'll check if I can find a good virusscanner for the server and what more I can do, but for now, this to me seems the most logical step.

Ok, here I go...
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Server has been hacked, help please Seventh Linux - Security 11 09-26-2006 11:57 AM
Hacked! How to find how he got in? newlinuxnewbie Linux - Security 17 10-08-2005 02:42 PM
Is my server hacked? kazjol Linux - Security 3 10-10-2004 12:09 PM
Server hacked php4u Linux - Security 1 07-05-2004 11:34 AM
server hacked!?!?! vittibaby Linux - Security 1 03-27-2004 12:31 PM


All times are GMT -5. The time now is 09:23 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration