LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices

Reply
 
Search this Thread
Old 09-02-2010, 02:55 AM   #1
eco
Member
 
Registered: May 2006
Location: BE
Distribution: Debian/Gentoo
Posts: 412

Rep: Reputation: 48
Exclamation server compromised?


Hi all,

I found out that there was http traffic leaving my server to a domain I knew nothing about, so I ran

Code:
tcpdump -i eth0 -xXNvvv port 80 -w dump.hack
... for a couple of minutes before adding a iptables rule to block the IP.

Is there a way for me to see what was going through this by analysing the content of the dump.

I had a look through it but can't seem to find much info in there.

Any hack known that sends data through port 80 that I should be looking for on my box?

Any thoughts/advice are welcome!
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 09-02-2010, 06:55 AM   #2
Hangdog42
LQ Veteran
 
Registered: Feb 2003
Location: Maryland
Distribution: Slackware
Posts: 7,780
Blog Entries: 1

Rep: Reputation: 412Reputation: 412Reputation: 412Reputation: 412Reputation: 412
I think we're going to need some more details about what is going on. By the way, in the case of suspected compromises, this forum tends to work rather differently than others you may have seen. Here we put a high emphasis on gathering facts about the machine in question rather than speculating about what might be going on. So, with that in mind, here are some things to think about.....

- What is this machine used for and why do you think this traffic was suspect?
- What distro is running on it, and how well patched/maintained is it?
- Are there any existing intrusion detection systems in place?
- Have you examined your log files for anything out of the ordinary?

Some potentially helpful information can be obtained from:

lsof -Pwn
ps -axfwwwe
netstat -pane

With these, you are looking for anythign suspicious, and feel free to post outputs.

In addition, if you feel the machine has been compromised, I'd strongly suggest pulling the network cable, but do NOT reboot/turn off the machine. If you don't have physical access, you might use iptables to cut off all access except SSH from a trusted IP

Also, have a look at the CERT Checklist for a good process to develop more facts about the machine.
 
2 members found this post helpful.
Old 09-03-2010, 08:35 AM   #3
eco
Member
 
Registered: May 2006
Location: BE
Distribution: Debian/Gentoo
Posts: 412

Original Poster
Rep: Reputation: 48
Hi Hangdog42,

Thanks for your reply and sorry for my late reply.

Quote:
- What is this machine used for and why do you think this traffic was suspect?
The box is a kvm server hosting a couple of running VMs. The server is also the firewall.

I suspected an intrusion when my very limited upload bandwidth started being saturated by an http(80) upload to a domain name I did not recognise.

Quote:
- What distro is running on it, and how well patched/maintained is it?
I am running an up to date debian 5.x with the latest patches (up to a week)

Quote:
- Are there any existing intrusion detection systems in place?
None. I wanted to give snort a try but the learning curve seemed a tad high for the limited time I had. I guess I should regret not taking the time now.

Quote:
- Have you examined your log files for anything out of the ordinary?
My log files don't seem to show much but I was in a bit of a panic so I might go back to them now and have a closer look but nothing jumped out.

I suppose the three commands you mention are only any good during the attack. If it's a bot, I could reopen the port, run the commands, get the data and close it again. Although, the less gets out, the happier I am obviously.

Thank you for your link, I'll have a look at it over the weekend and run some tests. I took down all non critical VMs.

Is there a short learning curve intrusion detection software or am I going to have to accept snort is the one and get on with it?

Thanks for your help.
 
Old 09-03-2010, 11:58 AM   #4
Hangdog42
LQ Veteran
 
Registered: Feb 2003
Location: Maryland
Distribution: Slackware
Posts: 7,780
Blog Entries: 1

Rep: Reputation: 412Reputation: 412Reputation: 412Reputation: 412Reputation: 412
Quote:
The box is a kvm server hosting a couple of running VMs.
Can I ask what the purpose of the suspect VM is? A web server? Email server? General purpose? The reason I'm asking is that will be helpful to know the kinds of services that were exposed to the outside world.

Quote:
I suppose the three commands you mention are only any good during the attack
I guess it depends on how you define "during the attack". Basically those three are for looking at what is currently running on the system in fair depth. Since you've seen unusual traffic on port 80, it is pretty safe to assume that if the machine was cracked, they left software running and hopefully these commands will show something. Of course if they really know what they are doing, they may have replaced some common commands with cracked ones designed to hide their activities.

Quote:
If it's a bot, I could reopen the port, run the commands, get the data and close it again. Although, the less gets out, the happier I am obviously.
I don't think you need to re-open port 80. Unless the rogue program is checking for connectivity to some external location and shutting down if it doesn't have any, it is likely blissfully unaware that it is sending information into the bit bucket rather than its intended destination.

Quote:
Is there a short learning curve intrusion detection software or am I going to have to accept snort is the one and get on with it?
It is going to depend on the software you choose, but and IDS like snort definitely is going to require some effort. A simpler approach is to use a HIDS like Aide or Samhain, but those by themselves are not likely to be sufficient since they detect cracks after the event. You probably should be thinking about an overall security plan that includes hardening, detection and recovery. There isn't a single magic bullet here.


Quote:
I am running an up to date debian 5.x with the latest patches (up to a week)
Excellent! Far too many people don't bother with patches. It's nice to see someone getting that aspect right.

By the way, am I safe in assuming that the suspect machine is a VM? Also, if some of the logs/outputs/evidence is too large to post, let us know and we'll get a place where it can be stored and shared. There are several people in this forum who like doing this sort of investigation, so you will get help if you develop evidence.
 
2 members found this post helpful.
  


Reply

Tags
hack, tcpdump


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
my server has been compromised, what next? Kropotkin Linux - Security 15 08-27-2009 06:15 AM
Server Compromised? lss1 Linux - Security 7 12-16-2005 12:49 AM
Server Compromised? stlyz3 Linux - Security 6 09-07-2005 04:28 PM
Gentoo's server compromised? Couldn't be, right? jon_k Linux - Security 1 06-12-2005 06:46 PM
Server was compromised, need help Asiana Linux - Security 3 06-02-2004 12:39 PM


All times are GMT -5. The time now is 04:14 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration