LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 08-12-2010, 04:48 AM   #1
sanjsantoki
LQ Newbie
 
Registered: Aug 2010
Posts: 1

Rep: Reputation: 0
Question How to fix compromise server


Guys,

I am newbie in Linux. I have a virtual machine and as per the datacenter update it is compromised. From the server SSH attack is being done.

I have checked last commands by running histroy command but there isn't any maliciouse command performed.

Is there anyway to troubleshoot and fix this issue?

Regards,
Sanjay Santoki
 
Old 08-12-2010, 05:46 AM   #2
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985
the only way you'll ever have full faith in the system is to rebuild / restore from backup. You will never know for sure if something hasn't been modified and tracks covered. Modfiying your own bash history is massively trivial.
 
Old 08-12-2010, 03:28 PM   #3
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Please don't give that advice

Quote:
Originally Posted by acid_kewpie View Post
the only way you'll ever have full faith in the system is to rebuild / restore from backup. You will never know for sure if something hasn't been modified and tracks covered.
Like I requested before without investigating the (perceived) breach re-installation could expose the same infection vector (if any). So please don't give that advice but point them to the CERT link, isolate the machine if necessary and ask them to wait for more knowledgeable people or our LQ Security incident handlers (Unixfool, Hangdog42, win32sux, me) to take over, TIA.
 
Old 08-13-2010, 04:26 AM   #4
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
In the interests of taking this a bit further for the OP:

There seems to be an implication that the mechanism of this attack is already known to utilise ssh. It would be helpful to be clear why this interpretation is being taken.

Depending on what is on the backup, you might be better to think of rebuilding the machine from the ground up, but we are getting ahead of ourselves...

@sanjsantoki
Quote:
From the server SSH attack is being done.
It is not as clear as it might be whether you are asserting that:
  1. there is evidence that some person or persons unknown are trying to log in via ssh, but you have evidence that they haven't succeeded
  2. ... some person or persons unknown are trying to log in via ssh, but you have no evidence that they have succeeded
  3. ... some person or persons unknown are trying to log in via ssh, but you have evidence that they have succeeded

(and you should say what evidence you have to support whichever assertion you are making)

It makes a big difference which of the above applies, so please be absolutely clear which of those that you are asserting applies. (Note that everybody, more-or-less, gets some bad guys trying to get in via ssh, so if you only have evidence that ssh attacks are being tried, but you are sure that these attacks have not yet succeeded, it may only be an issue of ensuring that your ssh security is as good as it should be. If you have evidence that the attack has succeeded, you can really trust nothing.)

A note on language: I am sometimes accused of being excessively fussy with the use of language, but note that here there is a massive difference between:
  • "there is no evidence that this has happened"
and
  • "there is evidence that this has not happened"
so you will need to be careful with exactly what you write.

as far as I can tell from what has been written:
the OP sees evidence of an SSH attack
the OP uses the word 'compromise': I have a suspicion that the OP may be using 'compromised' as a synonym for the SSH attack: the word 'compromise' ought really only to be used if the ssh attack has been succesful, allowed someone in and allowed them to do something (although, that last stage ought to be trivial for an attacker who can perform the preceeding stages) and it is unclear to me whether that is true. Just filling up logs with evidence of unsuccessful ssh attacks isn't compromise, per se.

it may be that
Quote:
as per the datacenter update it is compromised
contains the actual evidence; what is the datacentre update, what information does that contain?

If the OP wishes to list information that is available, it may be wise to censor any mentions of the IP address of the datacentre machine and replace it with 'my_ip_address' or something.

Last edited by unSpawn; 08-14-2010 at 09:28 AM. Reason: //Thread copy OT parts
 
Old 08-13-2010, 11:01 PM   #5
joec@home
Member
 
Registered: Sep 2009
Location: Galveston Tx
Posts: 291

Rep: Reputation: 70
The questions become:

- What level of security has been breached in the server compromise?

Is it a simple Apache script that got uploaded through a Front Page exploit and is running in /tmp? Or is there a root level compromise that the server can never be trusted and would require an operating system reload as part of standard repair procedures?

- What are the Abuse policies in the Data Center you are operating in?

If the serve you are leasing is on outbound attack, many Data Center will require action on your part to at least stop the outbound attack within a same time frame before they disconnect the server for violation of terms of service. Some Data Centers might also require an mandatory operating system reload if the compromise has been determined to be a root level compromise. Some Data Centers my even automatically terminate service if X number of root level compromises occur in X number of weeks. It would be a good idea to get in touch with the Data Center with a friendly call to see what they require of you on your part.

So in this post it has not been established if this is a minor compromise or a root level compromise. If it is a root level compromise then the correct textbook answer is backup the data and start over. Either way you will want to look through the log files to determine what was the initial entry of the exploit to begin with.

Linux - HowTo - Investigate A Linux Compromise
https://sites.google.com/site/zenars...nux-compromise
 
Old 08-14-2010, 09:28 AM   #6
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Off topic parts of this thread have been placed in http://www.linuxquestions.org/questi...ver-ot-826201/.
 
Old 08-14-2010, 04:08 PM   #7
abefroman
Senior Member
 
Registered: Feb 2004
Location: lost+found
Distribution: CentOS
Posts: 1,430

Rep: Reputation: 55
I think he is saying, his server is trying to brute force SSH logins on other servers. Or at least that's what his host is trying to tell him.

This does not mean they got root.

If there is any sensitive data on there you will want to have the datacenter make an current image of your VPS to preserve data.

If you host a number of sites on this VPS you will want to contact the site owners to let them know a breach has a occurred.

As part of your initial analysis you will want to run rootkit hunter, get a list of current running processes, open ports, as well as a list of open files. Check what versions you are running, IE. the kernel, apache, php, etc. Look through your log files, messages, secure, domlogs, if you run mod_security look at that. Look at the current apache processes running, if you use cpanel on your VPS you can do:
Code:
wget 127.0.0.1/whm-server-status
You can check the root history, but that is not the only thing to go by as the hacker can stop history logging easily.

Look through your aide reports, snort logs, mod_sec logs, logwatch, etc. If you don't have those installed/reporting, now would be a good time to start.

To stop the particular current attack, run SSH on something other than port 22, and block egress port 22 on your firewall, although this doesn't mean they wont use your server for another attack.

If the hacker did get root, after your investigation, and after you are confident the hacker cannot get back in, you would want your host to install a clean VPS for you.
 
Old 08-16-2010, 10:20 AM   #8
unixfool
Member
 
Registered: May 2005
Location: Northern VA
Distribution: Slackware, Ubuntu, FreeBSD, OpenBSD, OS X
Posts: 782
Blog Entries: 8

Rep: Reputation: 158Reputation: 158
It's a bit surprising (well, maybe it isn't) that the OP hasn't clarified his initial post or responded to anyone here.

OP, can you supply evidence that you got indicating you've a compromised system?

Can you apply the steps listed from the Intruder Detection Checklist and post the results here? I hope you've not restarted the system, as that will sometimes obscure/wipe the audit trail or anything queued within memory.

Also, what flavor of Linux are you using and what purpose is your machine serving?
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Multiple reboots by CRON. Possible server compromise? janhouse00 Linux - Security 1 05-21-2010 07:41 AM
NTP compromise? Quakeboy02 Linux - Security 2 11-24-2008 12:28 AM
LXer: Debian Server restored after Compromise LXer Syndicated Linux News 0 07-14-2006 02:54 AM
access_log compromise ?? lawadm1 Linux - Security 8 12-17-2005 11:40 PM
Security Compromise apache Linux - Security 16 08-07-2004 10:29 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 06:42 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration