LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices

Reply
 
Search this Thread
Old 07-25-2006, 11:45 AM   #1
dsschanze
Member
 
Registered: Aug 2004
Location: Gainesville, FL
Distribution: Linux Mint 12, Win7, iOS
Posts: 206

Rep: Reputation: 32
Why is my server getting hacked so much?


Hi,
I just finished installing my server again after it had been compromised for the 2nd time in 2 years. I went to login to root this morning and found that it had been compromised again earlier this morning, passwords changed and all. I ran a port scan late yesterday afternoon and found only http 80, https 443, ssh 22, ftp 21 and vnc ports to be open, all the rest being "filtered". Can anyone tell me what the heck is going on. By the way, my os is redhat 9.

-Derek
 
Old 07-25-2006, 12:03 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,300
Blog Entries: 54

Rep: Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855
I just finished installing my server again after it had been compromised for the 2nd time in 2 years.
Sad to hear that. Do you know the point of breach?


I went to login to root this morning and found that it had been compromised again earlier this morning, passwords changed and all. I ran a port scan late yesterday afternoon and found only http 80, https 443, ssh 22, ftp 21 and vnc ports to be open, all the rest being "filtered". Can anyone tell me what the heck is going on.
Unfortunately a portscan usually won't tell much. If the box is local then you can login and check (save) process, network and user activity listings and then run "rpm -Va --noscripts" and Chkrootkit or Rootkit Hunter (if installed) or boot a Live CD (like KNOPPIX, Helix) that includes those tools.
Before you start check out
Intruder Detection Checklist (CERT): http://www.cert.org/tech_tips/intrud...checklist.html
Steps for Recovering from a UNIX or NT System Compromise: http://www.cert.org/tech_tips/root_compromise.html
and the LQ FAQ: Security references for more info.


By the way, my os is redhat 9.
RHL9 is an EOL'ed, deprecated release. If you need it due to legacy applications, be sure to 0) update tru Fedora Legacy while you still can, 1) max out on host and network hardening and 2) prepare migration RSN to CentOS or Fedora Core or Red Hat if you want require RH or alike.
 
Old 07-25-2006, 12:16 PM   #3
dsschanze
Member
 
Registered: Aug 2004
Location: Gainesville, FL
Distribution: Linux Mint 12, Win7, iOS
Posts: 206

Original Poster
Rep: Reputation: 32
The point of breach was a root attempt on sshd. I was surprised as I had made new passwords since the last compromise last week and the passwords are not simple (either initials, birthdates, and other info that would not be available in a dictionary).

Also I don't see how it could be a root kit as I did a full system install yesterday (disk format and all).

Last edited by dsschanze; 07-25-2006 at 12:18 PM.
 
Old 07-25-2006, 12:29 PM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,300
Blog Entries: 54

Rep: Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855
The point of breach was a root attempt on sshd.
How do you know? Logs? Shell history? Did you allow root account access through ssh? What else could it possibly be? What does your httpd provide? Any PHP-based apps? Updated? Do you run a vulnerable version of RealVNC?


I was surprised as I had made new passwords since the last compromise last week and the passwords are not simple (either initials, birthdates, and other info that would not be available in a dictionary).
If you get in with root privileges all of that doesn't matter. Besides all the docs say not to use birthdates and stuff like that.


Also I don't see how it could be a root kit as I did a full system install yesterday (disk format and all).
If you run publicly accessable services w/o proper precautions then a fresh install ain't worth that much.

I suggest you read the docs and perform those steps anyway.
 
Old 07-25-2006, 01:04 PM   #5
slackhack
Senior Member
 
Registered: Jun 2004
Distribution: Arch, Debian, Slack
Posts: 1,016

Rep: Reputation: 46
sounds like some hackers have "found" you, and found you to be an easy target for some reason. my guess is that you'll continue to be compromised until you really harden your system more/differently than you have been doing.

changing your passwords to *very* difficult would be a good start (no initials, names, pet names, birthdays, etc.; make it more than 8 characters, include caps, lower case, numbers, and symbols). in my limited experience i've also found that getting daily reports from things like snort, tiger, logwatch (really like that one), etc., are helpful in identifying what's going on on the server, not to mention running chkrootkit and rkhunter daily, as well as checking the /var/logs.
 
Old 07-25-2006, 03:21 PM   #6
msound
Member
 
Registered: Jun 2003
Location: SoCal
Distribution: CentOS
Posts: 465

Rep: Reputation: 30
I just have to ask why root was permitted SSH access? If you absolutely must allow root to have remote ssh login access, you'd be better off using public/private key authentication to prevent hackers from taking stabs at your password.

If root doesn't need ssh access, open your sshd_config file and set the following two parameters:

PermitRootLogin no
AllowsUsers anyone, but, root

If you set the AllowUsers parameter, your system will automatically reject ssh access to all users that are NOT in the list.
 
Old 07-25-2006, 03:28 PM   #7
dsschanze
Member
 
Registered: Aug 2004
Location: Gainesville, FL
Distribution: Linux Mint 12, Win7, iOS
Posts: 206

Original Poster
Rep: Reputation: 32
Quote:
Originally Posted by unSpawn
The point of breach was a root attempt on sshd.
How do you know? Logs? Shell history? Did you allow root account access through ssh? What else could it possibly be? What does your httpd provide? Any PHP-based apps? Updated? Do you run a vulnerable version of RealVNC?
Well, I know from when I logged in as root, it told me the last ip address that had logged in and it wasn't one I was familiar with. Like I said, the password had been changed. I have not gotten httpd (Apache 2.0.40) up and running (providing anything) yet, so I don't think anything would have gotten in through that. Again, no PHP-based apps up as I am not offering website service yet. I am running the default version of VNC that comes with Redhat 9, but I always start the service when I need it, and stop the service when I am done using it, so it is never available when I am not in contact with the machine.
 
Old 07-25-2006, 03:31 PM   #8
dsschanze
Member
 
Registered: Aug 2004
Location: Gainesville, FL
Distribution: Linux Mint 12, Win7, iOS
Posts: 206

Original Poster
Rep: Reputation: 32
Quote:
Originally Posted by msound
I just have to ask why root was permitted SSH access? If you absolutely must allow root to have remote ssh login access, you'd be better off using public/private key authentication to prevent hackers from taking stabs at your password.

If root doesn't need ssh access, open your sshd_config file and set the following two parameters:

PermitRootLogin no
AllowsUsers anyone, but, root

If you set the AllowUsers parameter, your system will automatically reject ssh access to all users that are NOT in the list.
I will also try this one, as I can always use su as a normal user and get root access to do things, and if I absolutely must have root access, then I can change the settings for the time being.
 
Old 07-25-2006, 03:46 PM   #9
msound
Member
 
Registered: Jun 2003
Location: SoCal
Distribution: CentOS
Posts: 465

Rep: Reputation: 30
Quote:
as I can always use su as a normal user and get root access to do things
That's absolutely right! You could even take it a step further and setup sudo.
 
Old 07-25-2006, 04:06 PM   #10
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,300
Blog Entries: 54

Rep: Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855
OK. Unless there's more, let's wrap this up asking the final question: do you have a clear view of the steps you have to take to set up and harden your box?
 
Old 07-25-2006, 05:36 PM   #11
dsschanze
Member
 
Registered: Aug 2004
Location: Gainesville, FL
Distribution: Linux Mint 12, Win7, iOS
Posts: 206

Original Poster
Rep: Reputation: 32
Well, I have a little more to ask.

I ran nmap from another computer on the network on my server to come up with my port list. I have ftp, ssh, domain, http, https, cadlock (webmin) turned on for a reason, but don't know what rpcbind, kdm, and X11 are doing open. Can you tell me what those three are and if any of the services open are easy doors/windows for hackers to get into.

(The 1651 ports scanned but not shown below are in state: closed)
PORT STATE SERVICE
21/tcp open ftp
22/tcp open ssh
53/tcp open domain
80/tcp open http
111/tcp open rpcbind
443/tcp open https
1000/tcp open cadlock
1024/tcp open kdm
6000/tcp open X11
MAC Address: 00:50:04:A5:5E8 (3com)

Also, I ran chkrootkit this afternoon and found nothing to be a problem. I will continue to run this periodically and check logs.

Thanks for the help,
Derek
 
Old 07-25-2006, 06:45 PM   #12
Vgui
Member
 
Registered: Apr 2005
Location: Canada
Distribution: Slackware
Posts: 496

Rep: Reputation: 31
For the X11 port open, edit the 'startx' script and replace serverargs="" with the following:
Code:
serverargs="-nolisten tcp"
That will stop it from listening on port 6000 (and may fix the kdm port as well, which is the GUI login manager).

As the above posters said, root over ssh is definately a no-no. I'd definately look into a hardening guide for ssh (there are tons around). Basically though disallow root login, setup public / private keys, turn down the number of failed logins allowed, possibly change the port number, etc.

Lots of other things to watch out for related to security, like incorrect permissions on important files / folders, excess user accounts that are allowed login, etc.

Definately try to read up as much as you can, since it is such a broad topic it's rather hard to list out all the "steps" to better security.

btw sad to hear about your continued break ins.
 
Old 07-25-2006, 07:01 PM   #13
Linux_Regular
LQ Newbie
 
Registered: Jul 2006
Distribution: Ubuntu Edgy Testing
Posts: 5

Rep: Reputation: 0
Just another small thing,
If you run a rootkit hunter, remember to run a fresh copy from a floppy disk, if you have been compromised, you can't rely on the installed version being "un-compromised".
 
Old 07-25-2006, 07:47 PM   #14
Matir
Moderator
 
Registered: Nov 2004
Location: San Jose, CA
Distribution: Ubuntu
Posts: 8,507

Rep: Reputation: 118Reputation: 118
I would strongly advocate running iptables in a default-deny mode to protect services on your machine. A good setup, IMHO, is as follows:
Code:
# Set up tables
iptables -P FORWARD DROP
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT

# Accept from localhost
iptables -A INPUT -i lo -j ACCEPT

# Accept packets that are part of existing connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Accept incoming SSH connections
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
More services can be added with lines similar to the last one in the above script.
 
Old 07-26-2006, 05:53 AM   #15
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,300
Blog Entries: 54

Rep: Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855Reputation: 2855
I ran nmap from another computer on the network on my server to come up with my port list.
Be aware that if you are behind a router there will be a difference in results if it doesn't forward ports. Also be aware Nmap uses a static port-to-name mapping (just like the system uses /etc/services). If you want perfect results use the version/bannerscan option or do a local "netstat -nap".


don't know what rpcbind, kdm, and X11 are doing open. Can you tell me what those three are and if any of the services open are easy doors/windows for hackers to get into.
Rpcbind aka portmap. Stop and remove if you don't provide NFS, NIS or other RPC services. If you remove, make sure you remove services like NFS as well. If unsure stop anyway. If you really need to provide NFS the check here(RHL), here(distro-agnostic) and here(Deb) for details on how to restrict access. Kdm/Gdm/Xdm are Display Managers (DM). Check their configs (/etc/X11/?dm/) for allowing network access which they shouldn't. X11/Xorg has been covered by others.


Also, I ran chkrootkit this afternoon and found nothing to be a problem. I will continue to run this periodically and check logs.
You box was compromised. You can not keep running this setup. Check out Steps for Recovering from a UNIX or NT System Compromise: http://www.cert.org/tech_tips/root_compromise.html and Formatting and Reinstalling after a Security Incident (SF): http://www.securityfocus.com/infocus/1692. Then check out the LQ FAQ: Security references wrt more host and network hardening, choose a maintained O.S. release and reformat, repartition and reinstall from scratch. Do not install network services and applications you don't need in the configuration phase (like FTP, HTTP and MySQL, PHP etc, etc). Basically the only thing you need is SSH. If you need graphical access use TightVNC and tunnel over SSH. Make sure you use different and strong passwords as mentioned before, stop (and remove) unnecessary services, block inbound services in the firewall and disable or remove router port forwarding before you connect the box to the 'net for updates.

Now, with all the config stuff and docs mentioned, do you have a clear view of the steps you have to take to reinitialise, configure and harden your box now?
 
  


Reply

Tags
repair, security


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Is my server hacked? kazjol Linux - Security 3 10-10-2004 12:09 PM
Server hacked cpanelskindepot Linux - Security 46 07-05-2004 06:19 PM
Server hacked php4u Linux - Security 1 07-05-2004 11:34 AM
server hacked!?!?! vittibaby Linux - Security 1 03-27-2004 12:31 PM
web server hacked. sarin Linux - Security 12 10-05-2002 03:51 PM


All times are GMT -5. The time now is 11:42 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration