-   Linux - Security (
-   -   System compromised (

BruceCadieux 09-29-2003 10:40 AM

System compromised
I am not sure how it is happening but someone is gaining access to my desktop.

I do not have vnc, or rfb running, my router is blocking everything but my http and ftp servers.

I don't have ssh, telnet or anything in my start up services.

Which files can check to see what is happening.

I have run chkrootkit and it finds nothing.

I need help please.

unSpawn 09-29-2003 11:56 AM

I am not sure how it is happening but someone is gaining access to my desktop.
What are the symptoms? Please explain as detailed as you can. If you need to support what you're saying with logs and config files, please do so.
- check the output (/tmp/ps.log) and post the results from running "/bin/ps axfw 2>&1|tee /tmp/ps.log".
- check the output (/tmp/ns.log) and post the results from running "/bin/netstat -pane -A inet 2>&1|tee /tmp/ns.log".
- check your firewall default policies, log and report any unusual traffic or entries you can't explain.
- If you ran an filesystem integrity checker like Aide, Samhain or tripwire, please post the report. The same goes for running "rpm -Va 2>&1|tee /tmp/rpmverify.log", but please look at the logfile first.
For all logs report anything unusual or entries you can't explain.

I don't have ssh, telnet or anything in my start up services.
Well then, what *do* you run? Run "chkconfig --list 2>&1|tee /tmp/chkconfig.log". For all daemons listed in /etc/rc.d/init.d check their config files to see if they log to /var/log/messages or have their own log. Then check those logs. Don't forget Xinetd if you run that.

Which files can check to see what is happening.
In essence, the logfiles. They're in /var/log. Check all for today and go back at least a week. If you run into a file called "wtmp" run "last 2>&1|tee /tmp/last.log" or "btmp" then run "lastb 2>&1|tee /tmp/lastb.log". For all logs report anything unusual or entries you can't explain.

I have run chkrootkit and it finds nothing.

BruceCadieux 09-29-2003 12:42 PM

I have looked at the log files and saw nothing odd, I am going to run the commands you posted and will post the results.

Thank You very much for your help.

Please don't abandon me yet. LOL

I'll be back

BruceCadieux 09-29-2003 01:35 PM

I checked all the log files and ran the commands you specified. I saw absolutelty nothing that looked starnge, out of place or unusual.

I would post the results but it would take several pages to list them all.

To answer your question about what the symptoms were, well lets just say fighting over control of the mouse and keyboard with the remote user was not a whole lot of fun. Three times this happend this morning.

The only things I had running were a small web page, and an ftp server that requires a log in, it is not a public server.

I did shut them down, and blocked port 21 and 80 on my firewall/router and that ended the problem. Unless he/she went away for a while.

I just started them back up to see if or when the individual will come back.

I do not have ssh running as a service, however there is always a /tmp/ssh-XXX60oz/agent showing up in netstat , even if I shut it down, the tmp remains, and if I delete it it just returns.

I am confused to say the least. I have been running linux for a couple years, and never really had any security issues. This is the first, and I just don't know what to do to remedy it, other then shut down the web and ftp servers.

clacour 09-29-2003 02:10 PM

First, a warning -- I'm not a security guru. If one shows up, pay more attention to them.

Next, please give more detail. "I've been hacked" is too vague to be of much use. What, specifically, tells you you've been hacked?

I've had many instances when I thought my (home) system had been compromised (usually because of unusual/unexpected network activity), only to find out something else was going on.

So if there are weird things going on, you may or may not have been hacked. Obviously, if the mouse is going various places on the screen, clicking on icons, typing text, etc, you're right, you've been hacked. I/we need details.

Just to get the unpleasantness out of the way, the only right way to handle a box that has been hacked is to rebuild it from ground zero. Be thinking about what needs to be backed up (no executables!!) in case that proves necessary.

Rule 1 when you suspect a box has been hacked: Don't trust anything any more than you absolutely have to. In particular, don't trust anything currently running on the system, or any place it can write to.

The hard drive is guaranteed to be a suspect. If you have a network at home, any machine it can get into should also be suspect.

I've seen the name chkrootkit, and know in general what it does, but I don't know the details. If it's a script running standard tools like ls, md5sum, etc, it can't be trusted either. For that matter, even if it's a binary that does an internal checksum, if it was on the hard drive, it can't be trusted.

What to do first depends on your goal. If you want to try and catch the hacker, to spy on him and see what he's doing, you want to leave the box up (to begin with, at least) and use tools like netstat and lsof to look at network activity. Be aware that hackers (especially the ones good enough to write rootkits, as opposed to script-kiddies) know that you're going to want to do this, so replacing network tools (like netstat and lsof), file utilities (like ls, mv, cp, find), and process tools (like ps) are standard procedure. I'll go into more detail on how to do this in a later post, if you're interested.

If you just want your system fixed, reboot from a rescue CD (one that runs the OS from the CD).

NOTE: If you have any information on that box that would be embarassing or financially damaging to you (or anyone else), TURN IT OFF, NOW!!

Trying to beat a hacker after they've already gotten control of your box is like trying to beat someone in a 100-yard dash, with them starting at the 90-yard line. Theoretically possible, yes. Even remotely likely? No.

You need to boot the system off a known-good CD so that your basic tools for checking system integrity are good ones. If you have been hacked, you have to assume that every program you might use to check on things has been corrupted, as have things like the RPM database.

I recommend Knoppix, or one of the rescue CDs like LNX-BBC , but your Red Hat CD will probably be adequate. In the case of the Red Hat CD, don't tell it to do anything like mounting a filesystem -- you want to be as sure as possible you're running from CD.

If the stuff on the hard disk IS good, the odds are very high that you have NOT been hacked.

Here's how to use your Red Hat CD (or any other which is run-from-cd) to check the system.

Boot off the CD (which gives you a safe kernel and good binaries), and run rpm -Vp <package name> against the package on the CD. For example, "rpm -Vp /cdrom/RedHat/RPMS/binutils-". (That's from RH 9, so the version number will be different on your CD.)

An intelligent rootkit will update the Red Hat RPM database, so just doing an ordinary "rpm -V binutils" isn't reliable, and even "rpm -Vp /cdrom/RedHat/RPMS/binutils-" can't be trusted if you're running the command "rpm" from the hard drive.

Once you've booted from the CD, here's a list of the most likely candidates for changes on CD #1:
kernel-**** (whatever yours uses. Ignore this if you've recompiled your own kernel)
lsof (you might not have this installed)

Others worth checking:

If all of those are ok, you can go back to running off the hard drive, and the odds are high that you have NOT been hacked. Even if you have been, with your basic utilities, and especially RPM, confirmed as good, you can look for other things with some assurance you're not being lied to.

If rpm HAS been confirmed as untouched, do the following:

Mount, in turn, each one of the RH 7.3 CDs.
On each one, cd into /cdrom/RedHat/RPMS (substitute your mount point for "/cdrom") and run the following script:

for i in $(rpm -qa)
rpm -Vp $i 2>/dev/null
done | tee /tmp/rpmverify
You're primarily concerned about executables that have changed in size and/or md5sum.

If you have run up2date, your current version will not match the CD on some filesets. If you saved the downloaded rpms, you can verify against those, same as you did with the ones on CD.

Finally, like I said at the beginning: Give more details. If you have something weird going on, but haven't been hacked, we may be able to help, but we'll need a lot more info.

BruceCadieux 09-29-2003 02:49 PM

I can't give any better details other then the fact that someone else had control of my desktop, they were opening applications, trying to logout/restart, opening terminals, they even opened xmms and started playing music.

I was simply fighting for mouse control when they were attemting to log me out.

About all I could do was yank the ethernet cable to stop them.

I agree I really dont trust the system now, and will probably format and reinstall, just to be sure, because I am just not confident that I can clean it up properly.

However I am still going to need to find out how or what they are doing to get in before I restup the machine. Otherwise I fear they will do the same thing to get in with the new set up.

clacour 09-29-2003 03:38 PM

Well, if the only two services you had running were http and ftp, than it's pretty well got to be one of those two.

You said earlier that you didn't have telnet, sshd or anything like that running.

I'm guessing, but I suspect you think that means they can't do things like vnc, telnet, etc.

A port is just a way to help sort IP traffic. Normally, port 80 stuff goes to http, port 20 and 21 goes to an ftp server, etc.

It doesn't HAVE to work that way, though. If the hacker substitutes his own ftp program (my guess is that's the one that got hacked), he can send telnet/ssh/rsh traffic across the ftp port(s) and since his version of ftp is what's running now, it will handle them just fine.

If he got fancy, he could even continue to handle ftp requests.

Fighting with you for the mouse, clicking icons, and typing text is, I have to agree, pretty solid evidence that you've been hacked.

It might actually be possible to fix your system, although I'd only recommend it as a learning experience. Your average "real" hacker is going to evaporate instantly if there's any sign they've been detected -- they want to leave you guessing. This sounds like someone who was playing with you, and who therefore may not have done as much to compromise your system.

As I mentioned, my suspicion is that your FTP service got hacked, especially if you are running wu-ftpd (which was the standard in RH 7.3, as I recall).

In RH 9, Red Hat switched to vsftpd. I'd recommend you do, too.

Whether you install 7.3 again or a more recent version, make sure the first thing you do is run up2date against it. (From behind a firewall that doesn't allow anything except SSL (port 443) traffic through.) Red Hat allows you one free system, and if you have more than one, you can either download to one and update the others from it, or go ahead and shell out the money for a regular subscription. (At $5/month, I consider the subscription well worth the money.)

Far and away the most common reason people get hacked (well, except maybe for opening strange email attachments, which ought to be considered "aiding and abetting" the hacker) is software that talks to the network that is out of date.

Either don't run services (of any flavor), or keep your services up to date.

Hope this helps,


J.W. 09-29-2003 03:42 PM

Dude - I don't know enough to suggest any ways to block security intrusions, but I would echo clacour's comments about the primary importance of securing your personal data before doing anything else. Clearly, if your mouse, etc was being controlled by someone other than yourself, it certainly seems reasonable to believe your box was hacked, and that it would be dangerous to leave it open/exposed. Furthermore, from the (small) amount I know, the only reliable way to recover from that situation would be to flush their entire system and start from scratch, using "known to be valid" source CD's, as has been outlined in greater detail by others.

The key point though is that the end result of the cleanup operation, if you believe the intruder may have obtained login ID's, credit card numbers, etc, then simply restoring your box to a pristine state will not protect you.

I would advise that if you immediately change your password to any financial sites you may be using, your bank, and your Email account(s). I would also contact my credit card companies and report my card as being stolen/lost and get a replacement. If you are concerned that your checking account info may have been acquired, etc, then cancel that account immediately and open a new one. Obviously the extent of these sorts of actions would depend on how much personal data you were keeping on that box.

Doing all this stuff is a hassle and will create some short term pain for you, but compared to the pain and potentially high costs to you of not taking action, it probably would be well worth it. Good luck getting all this sorted out. -- J.W.

BruceCadieux 09-29-2003 04:18 PM

My systems are always up to date with the latest patches from up2date, it is RedHat9 and is running vsftp.

I was assumming it was through the ftp ports, only because when I shut them down on my router the problem went away. Of course the individual may have just stopped for the time being.

There is nothing of great importance on the machine that I am concerned about, I have backups of all my data on other machines. It is simply used as a multimedia machine and serves up a very simple web page, she is the only one who has ftp priveledges. Well at least I thought so.

I am only guessing that somehow they are using ssh, rssh, or some other means to access my Xserver. I am not sure how.

My next question would be, if I reformat and set it back up what can I do to prevent this again.

I would be setting up the ftp and web server again, these were the only two ports accessable to the outside world.

I have all the latest updates and security patches, the latest kernel from redhat........

I am trying to just figure out how they did it. My log files reveal nothing usefull.

hanzerik 09-29-2003 04:38 PM

You sure you don't have any un-needed services running? If there is a rootkit installed already you wont be able to tell, because they can set it up to ignore those services and files associated with them when you do a ls or ps. you could try running nmap against your machine an seeing what ports are open. Even netstat can be compromised to not show info about certain ports. I would agree with the others; lock your router down, and reinstall. If you have another computer i would use it to run nmap against your compromised machine, or some other port scanner. You can get KNOPPIX-STD, it has some nice tools for network security related tasks.

unSpawn 09-29-2003 04:50 PM

I was assumming it was through the ftp ports / I am only guessing that somehow they are using
You're not helping us help you, so this will get you nowhere. Read the advice given, act on it and supply us with factual info instead of talking *about* it.
Clacour is right about checking the rpm database from the rescue cdr, but this does not necessarily means that when it turns out OK you're in the green. Without proper response though exploring any other angles is going to be a waste of time.

My next question would be, if I reformat and set it back up what can I do to prevent this again.
I could sum up a lot, but in short: read the docs on hardening your box and act on it.

BruceCadieux 09-29-2003 04:50 PM

Of all my services the only ones allowed outside the router are http and ftp.

BruceCadieux 09-29-2003 07:55 PM

Don't I feel like the biggest damn fool there ever was!!!!!!

I was sitting here tonight again and my mouse started moving again, so I open netstat -a to see whats going on and I see nothing as usual. All of a sudden the command prompt in my console starts typing, so I think I will see what he/she is going to try.

The typing was I just got a creative nomad jukebox I am thinking well thats nice but what the hell are you doing on my PC. I get back a reply how are you doing that??

I yank my ethernet cable again, but the keyboard and mouse are still moving and typing!! Thats when I look down under my monitor and see the lights flashing on the reciever (cordless mouse/keyboard)

I go to the neighbor and ask if they happened to buy a Kensington cordless mouse/keyboard and they say yes but it acts all crazy and does stuff all by its self.

I started laughing histerically, and tell them that I have two of them, and if they are not set on different frequencies then they can't be used near each other, well I never would have imagined that 80 feet away and through walls these things would transmit their signal!!!!

So my problem is solved, I feel like a jerk, but releived at the same time.

I want to thank everyone who helped. I didn't reformat, and reinstall, I just plugged in an old wired mouse/keyboard for now. The neighbor is being great about it and said they will return theirs for a different model.

J.W. 09-29-2003 08:13 PM

Sweet - what a relief that must be. If there was such a thing, this would definitely get the post of the day award. Who would have thunk it would have been because of this? Wow. In any case, I'm stoked that it turned out to be because of such an unlikely (but now obvious) cause.

I'm curious though - just how many watts do those cordless mice use? Holy cow - that's some range. -- J.W.

BruceCadieux 09-29-2003 08:26 PM

I have used Logitechs for years without ever experiencing anything like this. These are my first Kensingtons, I love them they work great, form good distances too, but I never would have imagined they would work from that far away!!

Maybe Kensington should look into making wifi adapters LOL!! :D

All times are GMT -5. The time now is 01:52 PM.