LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 01-03-2005, 02:56 PM   #1
robintw
Member
 
Registered: Jul 2004
Posts: 48

Rep: Reputation: 15
Might have been hacked


Hi

I think that my server might have been hacked. I left my pc on while I was away (so I could access it via SSH). When I came back it was off - yet we had no power cut. When I turned it on I found the command shutdown (with some arguements afterwards) in the root command history. I then looked in the log files (/var/log/message) - and found some lines about failed logins via SSH.

I tried to use the locate command and it gives no response at all. I have tried running updatedb and it still doesn't work. I am worrying that a hacker might have managed to disable the locate command so I couldn't find some horrible spyware/trojan he'd installed. The find command, by the way, still works.

My passwords were adaquete but not especially secure. I have changed them all obviously. I do run a webserver (apache) and a ftp server (vsftpd).

Is there any way I can definitly check if I have been hacked - and if so remove any horrible things he may have done?

Robin

P.S. I have tried to track the IP address that the failed logins came from but it couldn't be traced. I have stopped runnin the ssh server now
 
Old 01-03-2005, 03:43 PM   #2
Linux.tar.gz
Senior Member
 
Registered: Dec 2003
Location: Paris
Distribution: Slackware forever.
Posts: 2,534

Rep: Reputation: 100Reputation: 100
Running a ssh server with weak passwords is a BIG mistake.
 
Old 01-03-2005, 03:50 PM   #3
robintw
Member
 
Registered: Jul 2004
Posts: 48

Original Poster
Rep: Reputation: 15
Yes I've realised that.

Is there anything I can do - apart from stopping the SSH server and changing my passwords?

Robin
 
Old 01-03-2005, 03:55 PM   #4
kermit00
LQ Newbie
 
Registered: Jan 2005
Posts: 3

Rep: Reputation: 0
Quote:
hacker, n:

[originally, someone who makes furniture with an axe]

1. A person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary. RFC1392, the Internet Users' Glossary, usefully amplifies this as: A person who delights in having an intimate understanding of the internal workings of a system, computers and computer networks in particular.

2. One who programs enthusiastically (even obsessively) or who enjoys programming rather than just theorizing about programming.

3. A person capable of appreciating hack value.

4. A person who is good at programming quickly.

5. An expert at a particular program, or one who frequently does work using it or on it; as in ‘a Unix hacker’. (Definitions 1 through 5 are correlated, and people who fit them congregate.)

6. An expert or enthusiast of any kind. One might be an astronomy hacker, for example.

7. One who enjoys the intellectual challenge of creatively overcoming or circumventing limitations.

8. [deprecated] A malicious meddler who tries to discover sensitive information by poking around. Hence password hacker, network hacker. The correct term for this sense is cracker.
And you should format and reinstall, (making backups of course, but only of your personal stuff, dont go backing up /bin or anything =]). That would be the best way. Otherwise you might never find the script kiddie overused rootkit that this "uberleet" password guessing "cracker" placed there.
 
Old 01-03-2005, 03:58 PM   #5
robintw
Member
 
Registered: Jul 2004
Posts: 48

Original Poster
Rep: Reputation: 15
Does the fact that locate isn't working almost certainly mean that this hacker (or cracker) has installed some horrible thing?

I'd prefer not to have to wipe if at all possible - i can't remember how to set everything up again!

Robin
 
Old 01-03-2005, 04:14 PM   #6
SlackerLX
Senior Member
 
Registered: Dec 2004
Location: Herzliyya, Israel
Distribution: SuSE 10.1; Testing Distros
Posts: 1,832

Rep: Reputation: 47
Good hacker knows how to use the broom, if you get my meaning.
Format recommended
 
Old 01-03-2005, 04:21 PM   #7
SlackerLX
Senior Member
 
Registered: Dec 2004
Location: Herzliyya, Israel
Distribution: SuSE 10.1; Testing Distros
Posts: 1,832

Rep: Reputation: 47
I doubt, however that it was qualified hacker. Rather amateur studying nmap or smth...
Good hacker changes the log too and does not shutdown the box. Final command without ability to chance it, for hacker is inacceptable and sometimes unforgivable mistake

You could have power shortcut...

Last edited by SlackerLX; 01-03-2005 at 04:22 PM.
 
Old 01-03-2005, 04:30 PM   #8
Capt_Caveman
Senior Member
 
Registered: Mar 2003
Distribution: Fedora
Posts: 3,658

Rep: Reputation: 69
Re: Might have been hacked

When I turned it on I found the command shutdown (with some arguements afterwards) in the root command history.
Any other commands in root's bash_history? Take a look at the system logs to see if you can find the date-time when the shutdown took place.

I then looked in the log files (/var/log/message) - and found some lines about failed logins via SSH.
Do they correspond at all to the shutdown time from the system logs? There is fairly wide-spread scanning for sshd servers with weak passwords identified through a dictionary attack. Not that uncomon, so this may be coincidence.

I tried to use the locate command and it gives no response at all.
Does the actual command exist or does it just hang?

My passwords were adaquete but not especially secure.
Please elaborate. What does adequete mean...are you passwords simple dictionary words, numbers + letters, etc. The ssh "bruteforce" attack only scans for a limited set of paswords, so it may have failed if you hav more than rudimentary passwords. That's actually why I don't like calling it a bruteforce attack, when it's more accurately a dictionary attack with a fairly simple and crappy dictionary. Check the output of last -i to see if any logins correspond to the time period that the ssh attack occurred.

I have changed them all obviously. I do run a webserver (apache) and a ftp server (vsftpd).
Have you kept them updated and patched or were you using vulnerable versions?

Is there any way I can definitly check if I have been hacked
You can definitely show that you have been hacked (find files, rootkits, etc), however it's extremely difficult to prove that you haven't been hacked. A skilled attacker will likely be able to hide their tracks from standard detection techniques, though in this case rebooting the server and leaving info in a bash_history isn't exactly subtle.

I'd definitely recommend downloading and running chkrootkit and rootkit hunter as well as kern_check, available here (the locate error sounds like a borked system.map). I'd also take a look at /etc/passwd for any new users or users other than root with a UID of 0. And run a check for SUID/SGID root files with:
find / -user root -perm -4000 -print
find / -user root -perm -2000 -print

Also look at the system logs for any abnormal log messages, including things like application/kernel errors or panics. The system instability (locate command not working) hints at a rootkit and might require you to reboot the system using a cd-rom based distro and then mounting the hd read-only in order to do any kind of analysis. For now though, I'd at least take the system offline until you can be reasonably sure that it's clean, though I'd hold off on a format at least to you can do some forensic analysis. Lastly, does anyone else have physical access to the machine?
 
Old 01-03-2005, 04:32 PM   #9
kermit00
LQ Newbie
 
Registered: Jan 2005
Posts: 3

Rep: Reputation: 0
Well, seeing as he said there were a good few failed login attempts, it was most likely some idiot guessing ssh passes, or some idiot's scanner/bot guessing ssh passes. And like all other idiots of the type, they like to install rootkits, especially ones that they didnt write (=P). So you could grab checkrootkit, rkhunter, or another similar program, but the best idea would be to format. You dont, and you might get screwed over later.

edit: hah, never mind. just listen to caveman =]

Last edited by kermit00; 01-03-2005 at 04:42 PM.
 
Old 01-03-2005, 04:45 PM   #10
Capt_Caveman
Senior Member
 
Registered: Mar 2003
Distribution: Fedora
Posts: 3,658

Rep: Reputation: 69
I get failed ssh logins on a daily basis, so it don't make much sense to format everytime a failed attack shows up in the logs. Better to do some analysis first and see if that was the actual means by which the system was compromised (or even if it was compromised at all). If it wasn't the means of entry, then you are likely to repeat the same mistake when you re-install only to get screwed over again, later.
 
Old 01-03-2005, 05:25 PM   #11
kermit00
LQ Newbie
 
Registered: Jan 2005
Posts: 3

Rep: Reputation: 0
oh of course, not everytime you get a failed attempt. But you do want to every time you get a couple failed attempts, then a successfull attempt, and nobody else is supposed to have root =P (that said, why are you letting root log on from anywhere but the first terminal?)
 
Old 01-03-2005, 07:36 PM   #12
jlangelier
Member
 
Registered: Jun 2003
Location: Denver, CO
Distribution: Debian
Posts: 95

Rep: Reputation: 15
If you can, it's a good idea to limit ssh access to certain IP addresses. For example, the only SSH traffic should be from my work site, so I limit it in my hosts.allow like so:

Code:
sshd:123.45.678.
This allows only IP's beginning with 123.45.678. to access ssh.

Quote:
I'd prefer not to have to wipe if at all possible - i can't remember how to set everything up again!
Another good idea is to keep a hardcopy notebook for every machine you admin. Write down EVERY administrative task you do to it. Write down any shell commands you do as root which affect your setup. Write down software installs in your /usr, write down changes you make to /etc files and other system stuff.

Last edited by jlangelier; 01-03-2005 at 07:39 PM.
 
Old 01-04-2005, 05:40 AM   #13
robintw
Member
 
Registered: Jul 2004
Posts: 48

Original Poster
Rep: Reputation: 15
Hi

Thanks for all the replys.

I have worked out that the unsuccessful logins via SSH come from a bot that is touring the web See here

Which log file lists the shutdowns? I can't seem to find a shutdown bit in /var/log/message. My log stops on Dec 31 - just stops - the command before was something I executed via SSH. Then it goes to a Jan 2 entry with the beginning of the boot up.

I just run locate like this:

Prompt>locate internet
Prompt>

Just gives no response like that.

My root password was my middle name. Not secure I know - I just can't remember secure ones and didn't think I'd get hacked! I was wrong - obviously!

Ummm I have used the Yast Online Update program - and I assume that updates apache/vsftpd/php etc.

I will look in my etc/passwd file when I get home - I am at college at the mo.

What is the program mingetty? I found 3 copies of it running - and then it gave a couple of screenloads of errors when I tried to close it.

When I get back home I will try chkrootkit. It might be better to wipe and reinstall - it's prob about time to do that - and its the only way to be absolutely sure. The only problem is I have to wait till my new hdd arrives - I have 100Gb of data I can't afford to lose.

Thanks for all your help

Robin

P.S. The notebook idea is good - I would have printed out some of my log files but I can't get printing via samba working (another topic for another post sometime!). I will use that way of SSH if I use SSH at all on my reinstalled/fixed system.
 
Old 01-04-2005, 01:29 PM   #14
robintw
Member
 
Registered: Jul 2004
Posts: 48

Original Poster
Rep: Reputation: 15
Hi again,

I'm at home now and have run chkrootkit. It seems to have found a packet sniffer for (strangely) the dhcp service. One wonders why they would want to sniff that service - but never mind. All my ttys on Ctrl-Alt-F1 -- Ctrl-Alt-F6 seem to have disappeared. Very Strange.

I think I will wipe and start again. I will install on my brand new 160gb hdd and then mount the old one and copy across the important data. I will probably use RedHat, and Bastille Linux and make my passwords secure. I will also NOT use SSH - as it is not essential for me.

Thanks for all the help you have given to me - I think i've learn't my lesson.

Robin
 
Old 01-04-2005, 11:22 PM   #15
Capt_Caveman
Senior Member
 
Registered: Mar 2003
Distribution: Fedora
Posts: 3,658

Rep: Reputation: 69
The dhclient flagged by chkrootkit is likely a false positive. The linux dhcp client listens on a PF_PACKET socket which chkrootkit detects as a sniffer (incorrectly). Not sure about the missing ttys, but mingetty controls the virtual terminals (ttys 1-6) so if mingetty is borked for some reason, that would likely explain the failure of CTL-ALT-F1-6. Out of curiosity what does rpm -Va output?

You still haven't said if any of the logins were successfull though. The last -i command will show you any around that time period if the logs haven't been scrubbed (chkrootkit does check for log and wtmp/utmp deletions).

In general, doing a format and full reinstall is always going to be the safest option (once a compromise has been detected it's the only choice). But if you still aren't sure of a likely means of entry, then you need to spend added time hardening the box when you re-install, even if using a different distro. Make sure to shutoff un-needed apps, install a good firewall, do some penetration testing, obviously use better passwords, etc). Probably a good idea to do some reading the unSpawn's security references thread (near the top of the forum) and checkout some of the general hardening guides. Bastille is definitely a really good start.

Be carefull when transferring files from the old system. Probably not a good idea to directly mount the old drive on the new system. Best way is to transfer files that you've visually inspected are clean to some type of removable media first and then transfer to the new system. Make sure that you don't keep any binaries at all.

Last edited by Capt_Caveman; 01-04-2005 at 11:23 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Have I been hacked? Please help linuxboy69 Linux - Security 11 09-07-2005 07:20 AM
Hacked? mikeshn Linux - Security 2 03-12-2004 01:57 PM
Help! Have I been hacked? Tenover Linux - Security 1 11-19-2003 03:24 PM
Did we just get hacked? vous Linux - Security 4 11-17-2003 08:11 AM
am i being hacked? tearinox Linux - Security 5 11-13-2003 06:00 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 10:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration