Well it happened! I opened up SSH on my box about 2 weeks ago so I could do some maintenance on the run. Well this morning I go to log in as root, and BAM!! My root password was changed! I quickly check to see if my regular account works, and that was fine, so far as I can tell. I checked my logwatch e-mails, which I haven't checked in about a week, and about 5 days ago someone attempted to login as 'root' some 2000 times and failed. Further down in the report the 'ssh' section had all the failed passwords, looks like a dictionary attack!! Which due to my stupidity would have no problem finding root's password in time. Sure enough the next days logwatch had the same attack, all from the same IP, then the next day, no attack. So I think I have a pretty good idea when this happened. I was able to view the password file to look for additional users, I only have a few, and none were added. So maybe he just comprised 'root', which is all he really needs anyway! I closed off SSH to the world, so only I can login at the terminal, but the other services are running, until I get home to shut things down. I have another box I can get up and running to take over while I do an autopsy on this one, which brings we to my questions:

System is Fedora Core 1

1. Where do I start? I want to be able to see what changes he made if possible. I did see 2 strange files 'shadow-' and I think 'groups-' or 'sgroup-' something to that effect. I know there should be a 'shadow' file for passwords, and that was there, but what about the '-' on the end of the other file? I am not sure they should be there.

2. Do I need to wipe this system, or can I just change the root PW and make sure it is strong this time!

3. What can I do to prevent this in the future, I really need ssh up and running!


Thanks!!
Rusty

Do your logs ever show a successful root login? Tons of failed attempts with a sudden stop doesn't absolutely mean you were cracked.


If you have been compromised, this isn't going to keep them out. They may have installed other methods to access the box. Unplug this box from the network.

In the Security References sticky at the top of this forum. There is a section devoted to forensics. Rkhunter and chkrootkit are probably good places to start.

You need to wipe the system. There is no other way to make sure that you got everything they may have installed. Changing the root password is nowhere near enough.

Well for starters, not allowing root access through ssh would be a fine idea. Strong passwords is another. You probably also want to investigate using key based authentication.

And when you get this box restored, I would suggest running a file integrity checking system like Tripwire, Aide or Samhain. If those are installed before you connect to the net they can tell you what files have been changed if another breach occurrs.

Any of the programs on the server could have been rewritten to allow access in the future, so to be safe it is best to reinstall.

In the future, A) don't allow root to login to ssh. Log in using your normal user account and use sudo instead. B) Choose a better password for root and your regular user password. C) Change your username. The hacker knows it now. D) Enter your user name in AllowUsers. ( man 5 sshd_config ) This will deny all other connection attempts. This is why I suggested changing your regular user name. One more thing the cracker has to guess.

I personally would reinstall this box from scratch due to my paranoia...

A few tips for the future.

1. Don't allow direct root login via ssh. You can disable this by setting PermitRootLogin no in /etc/ssh/sshd_config. Also make sure you have PermitEmptyPasswords no in there somewhere.

2. Set good passwords. I auto-scramble my root password to something totally random and long. I then stash this password away (envelope in a lockbox) and use sudo for day-to-day root access. If I have to open the envelope for any reason the password gets changed and a new envelope gets created.

3. Use the firewall. Setup iptables to drop everything and then explicitly go back and enable what you need to use.

4. Move SSH to non-standard port. I used to get thousands of login attempts a day via ssh.. I moved it to a different port and now I don't get all the script kiddies on the block attempting to login...

hi there,
generating some public/private keys can make life easier as well.
this has also helped a lot in my case. this won't protect you from port scanning but will reduce quite a bit of the standard attacks.
regards,
slackie1000

WOW you guys are fast!!!
OK , some more background. The hacked box is behind a Smoothwall firewall, and all access other the web, and e-mail have been turned off at the firewall. I kinda thought I was going to have to wipe the system, but ..... well anyway. SSH was on the standard port, but now I agree, I should move it. Will any port above 1024 work, or should I keep it below 1024. I have always had difficulties setting up tripwire, is there a good "how to" somewhere? As far as forensics, where should I look, I am curious to see what they changed, and how. If they didn't clear the history file, I should be able to see what commands were used, but is there anything else??

Thanks
Again!!!
Rusty

Here is a link to some good RH 9 tutorials. They might help with the tripwire setup.
http://www.redhat.com/docs/manuals/l...ide/index.html

Any port will work for SSH providing it isn't in use for something else. I typically put it on a port under 1024, but that is only because I'm firewalled at various places I connect from and not allowed to use most ports.

In an attempt to make this as painless as possible, can I reload the OS onto 1 hard drive. Use this drive for the OS, install the 'hacked' drive, but only for the data that is on there. Transfer all the data to the 'new' drive, leaving all the configs, and OS programs alone, then remove the 'hacked' drive. Does this pose any security issues??

Rusty

Here's what I suggest: (1) get rid of password authentication altogether, switch to keypair only (2) only allow your normal user to log in, like jschiwal said, and do admin work through sudo. As for switching ports, I don't think it really matters. A portscan will still reveal that your SSH server port is open, and connecting to it will reveal the fact that it's SSH. In other words, it can (and will) still be attacked, which is why I say switch to keypair auth only. The length of time it would take to successfully brute-force a keypair system is several magnitudes longer than that required to brute-force a password system. The same argument goes for switching your username, IMO. Anytime you merely hide information away (i.e. switching ports or user names) you're only buying yourself a small amount of time up front, without actually boosting security. It's like comparing steganography to cryptography; obviously, crypto is better. Always operate with the least privilege and most rigorous authentication mechanism possible.

One last note: if you always access this server from the same IP address or block, you can add a rule to your firewall to only accept connections on SSH from that address/block. If you have a very clever firewall, you should also be able to limit the number of connections per time period from a particlar address or block, limiting the effectiveness of dictionary attacks. OpenSSH has this capability, too, IIRC; but since it's not at the firewall level it's not as flexible.

i definitely think you should take the box offline and reinstall if you have any concerns at all. but i'm just wondering where the evidence of being hacked is? someone tried to compromise you with a dictionary attack, but there is no evidence they succeeded. passwd- group- shadow- files are common, if not necessary, on all systems. i even have a gshadow- file.

definitely do a forensics check using the resources and help available here, but imho a complete reinstall just because you *think* you were hacked might not be necessary. if after going through everything there's any question at all that you were hacked, then by all means reinstall (and also implement the other security measures: better passwords, alternative ssh port, IDS, etc.) but if you can't find anything, you might just consider yourself lucky this time -- while having learned some good lessons on security -- and save yourself a lot of work.

99.99% sure I was hacked, A dictionary attack was used (logwatch report showed that), and my root PW was changed Right now it is damage control and recovery time, hoping to be back up and running 100% by tomorrow!

For my own info, how do I find out how mant days since a PW was changed? I know there is a way to have the OS force a user to change their PW after a preset time, but how does it know how much time has passed? I would like to know exactally when the root PW was changed. Hopefully I can associate that time to an IP address to report!

Rusty

That's a nice hacker, he changed your password because it was too simple.
In case you get hacked again and he would loose his zombie.

Normally, "ls -l /etc/shadow". This won't tell you WHICH password was changed however. And if you were hacked it's a simple and standard thing to to do cover your (the cracker's) tracks. This may included changing timestamps on /etc/shadow, deleting or altering logfiles, replacing the ls command with a trojan version, etc. Basically, anything you think you can trust ... you can't.
If you were hacked, how would you know this? If you're using for example, "cat" to look at your password file for changes, who says the cracker didn't just install some custom version of cat that filters out the new user from /etc/passwd? You don't. That's the point. You might be able to expose this trick by booting your system from a Knoppix CD and using the known-good "cat", "ls", "ps", etc. from there.

I believe the time information is kept in /etc/pwd.db and /etc/spwd.db which are database files containing login information. See pwd_mkdb(8) for more info. (At least, on BSD this is the way it's done. On Linux... maybe, I don't know.)

No matter what OS you're on, that information has to be stored in some password file somewhere; so if you're lacking for clues just start going through and lookin' at them. Another indication may be the mtime for the password files, but a clever cracker would touch() those back to the original dates to avoid this.

However, if the password does not have an expiry time, the time when it was last changed may not be noted, as it is not necessary to compute any times based on that password.