I have Red Hat 7.0 running as a mail server (CommuniGate Pro). I have shut down NFS, Telnet, rlogin and rsh. I ran chkconfig and verified that they were shut down. I also ran (ps) with the -aux parameters to check running processes. All looked very good.

Today I look at the console and there is a message saying that promiscuous mode has been enabled on eth0 again. I thought that strange, so I used ifconfig to turn off promiscuous mode and then attempted to run (ps) to see the running processes. I received the following error:

" Segmentation Fault (core dumped) "

On shutdown the system had trouble shutting down the /usr partition. I rebooted the server and logged in. I ran (ps) again but get the same error as above. Has anyone seen this type of error before and if so where might the problem be?

Thanks

Oh hell.

Doesn't sound too good amigo, you may have been rooted.


Here's a link to some very good advice on what to do next:

http://cert.uni-stuttgart.de/archive.../msg00089.html


John

Thanks for the info Rootboy. I am looking into the chkrootkit software. I have another question for you. The next day and since then ps command does not give me any errors. Neither does the top command. Is this a sign that the cracker has setup everything the way he wants, or did he give up and leave?

Thanks again Rootboy

Well, if he did "leave" then he must have taken his programs with him.

And I doubt that.

Even if he did "leave", he has left himself a way to get back in any time he wants.

Hopefully I'm wrong about this (but you hit the criteria for being rooted smack dab on the head), but you need to do the chkrootkit thing and also check your "ps" and "top" MD5 sums against known good ones. The size of the files and the creation dates can be faked, MD5 sums are a bit trickier

This is what your "ps" command should look like:

MD5: 2b7cb086d724ea5511d1b4e33195cc5c
SHA-1: 25d5d9911cf8fc7115668e550409592760cc755f
Size: 65148 (bytes)

(Courtesy of http://www.knowngoods.org/)

And this is where I'd ask some questions in the security lists and get a second opinion.

However, keep me posted on what you find out, I've never been on this side of an exploi... Uhhh, nevermind.

And remember, tripwire is your friend.


Anytime, let me know what you find out. And good luck.


John

John:
Yes, the mail server has been rooted. du, find, ifconfig, killall, ls, netstat, ps, and top are infected. Possibly 3 different rootkits. t0rn v8, RH-Sharpe's, and Showtee. Maybe even Romanian rootkit. Bindshell is also infected and Slapper worm may be installed. Man have I been nailed. I can stop people from getting into Windows NT and 2000 as I've been around them longer. I like Linux because of it's text side. Reminds me of my days in DOS, Kinda like a DOS on Speed, but I have a lot to learn. I am assuming from all this my best bet is to format the drives and start from ground zero. What do you think of a product called Bastille? Will it improve the security? When I rebuild the mail server I will install Mandrake 8.1 instead of Redhat 7.0. Thanks for all the help. I will let you know the results. Thanks again.

Frank

Now that's interesting.

It would appear that your IP address got passed around amoungst some kiddies. Otherwise, why would one guy hit you with multiple rootkits?

Do a google search on your IP address and see if it shows up in any forums (don't post your IP address here).

I'll do some research on the various kits and try to find out what the vuln was. Or did you figure that out already?


It's a lot of fun, but just like any other OS, you've got to stay on top of the exploits. Or so I've been told


Yeah, probably so. I certainly wouldn't trust it with just a fresh install. You might ask around, security is not my strong suit.

Here is a good place to start:
http://www.linuxsecurity.com/


And be forewarned that the linux kernel doesn't really care where a file is to run it. It can be on your FAT32 partition and still be run by linux. If I wanted to be cute, I would put a copy of the utilities that I was interested in over on that side hoping that you wouldn't look there.

Although, I can't think of a reason why that would necessarily be a problem. You would have to gain root access to get to these files, and once you did, you would presumably have everything that you needed without squirreling anything away.


I wouldn't necessarily switch distros, not without doing some research first. What happened to you was that you fell victim to a package that had a vulnerability. Find that package and fix that.

Switching to another distro will just expose you to their vulnerabilities. There are some distros that concentrate primarily on security, Debian and Slackware come to mind.

These distros achieve a higher level of security by using older versions of a program that has been tested more throughly (or so you would be led to believe). That and any non-essential programs are typically left out of the distro. And these are usually a PITA to set up.

Some of the BSD's are supposed to be pretty tight, but I suspect that a lot of their strengths come from being even more obscure than linux is

Just don't tell Theo that I said that


John

John:

I did some checking on the Slapper worm that was found by chkrootkit
and it seems that the worm requires Apache and SSL to be installed.
To my knowledge, those aren't installed. I will be doing some port scans to see what may be active that the rooted commands aren't telling me.

Thanks for the information about remote storage of the files.
I don't think they could have pushed these onto the other servers because we have the mail server in a DMZ, but I will check the other
boxes.

I haven't been able to find any information on what software packages were the leak for the rootkits, but I am working on that.

I assume that when you say Google Search you are meaning Google's Search engine for the web. I did find the IP in a list of compromised servers in an E-mail from a company that was tracing DDOS attacks. The E-mail was old though, Sep 2002. I found nothing else. If there is something special to a Google Search that I am not aware of could you please explain.

Thanks
Frank

Hmmm, I don't know. Maybe they got into it via a different exploit (sounds like they would have had to).


Good idea!


Keep me posted. I'm still curious as to why the multiple attacks.


Bingo! What do you wanna bet this was how they got your IP address?

Is this an, err, legitimate company? I would like to know their thinking as to why posting your IP address is necessary.

And way back in September? And it was rooted then? Ouch!

Remember that there are intrusion detection programs that you can use (tripwire comes to mind). And I would be shutting down unused ports and unnecessary processes like mad.


The most usful feature for me is the linux sub-search, try www.google.com/linux


Anytime


John

John:

I stumbled into a directory today that does not show up with the ls command. I was doing a search with find for chkrootkit, which I installed the other day but seems to be missing. In the process I saw a directory go by that I could not find by any other means. I can cd to it and find several sub-directories. I know blue means a directory, grey is a text file, what are green file names?

It appears that this might be the workings of the adore worm. I found a crontab_entry that of course has some hotmail address. It has directories like pids, local, get, strobe, adore, lsof etc. I am thinking of just deleting the whole sub-directory. I probably will once I through viewing the C programs that are here.

Thanks
Frank

Good place to recycle a LQ post here about what to do with a hacked box. It ain't a complete HOWTO, but you'll get the idea.

HTH somehow.

That would be executables.



You need to delete the whole enchilada, but I would try to gather as much info before wiping the thing out first. At the very least contact Microsoft about those hotmail accounts.


Check out unSpawn's post, he has some excellent advice.


John

John:
I've replaced the mail server and really went crazy on the passwords. It seems secure for now. I looked at the logs and found 34 attempts to log into the server within 1/2 hour. The log did not show an IP, but showed a Machine name. All attempts failed, but he doesn't seem to want to give up.

After re-installing Red Hat, and being able to read the files again I found a file that looks like some hybrid scanner output. It looks like the following:

unclebuck.com => mail.unclebuck.com[23]

----------------------FIN

mail.unclebuck.com => 0.0.0.0 [21]
user willy
Password nilly
Type I

Do you recognize the output? Do you know the scanner used?


I have a suspicion that he has also invaded the internal network. I don't know how he found it, but I am looking into that.

Thanks
Frank

What was his machine name?

I did a lookup of unclebuck on netcraft and this is what I found:

OS: Windows 2000
Server: Microsoft-IIS/5.0
Last changed: 2-Feb-2003
IP address: 216.21.236.31
Netblock Owner: Register.com, Inc

You can get more details on www.unclebuck.com by doing a whois on netsol's site (network solutions).

Just guessing here (I'm way out of my depth), but I would guess that Uncle Buck has been cracked as well.

I didn't find any hints of anything fishy going on there...

www.unclebuck.biz is a different story however

The "mail.unclebuck.com[23]" part looks like a telnet connection (port 23) , so would that be how he is doing "maintenance" on your box?

Or possibly a connection to the outside world in general...

http://www.linuxdocs.org/HOWTOs/DSL-HOWTO/secure.html


Looks like his username and password for an FTP connection.

http://www.redhat.com/docs/manuals/t...ftpserver.html

Possibly his link to download your stuff?

Agggghhh!


My pleasure, and it's been very educational (unfortunately at your expense).


John

P.S., Sooner or later you probably should inform UncleBuck that he has been cracked as well.

See if he will give you a copy of his logs to work with. With any luck, the cracker will lead us to his own site from there

John:
After checking the log files and a thorough search I find no evidence that he found the internal network. We rebuilt the internal machines anyway.

Most of the hits that I found trying to connect to the old mail server was from other countries, Romania, Belgium etc.


Thanks again
Frank

John:
PS -- I informed Unclebuck about being compromised a couple of weeks ago. I forgot to mention that to you. He said he would look into it.

Thanks
Frank