Hi Geeks,

[root@safari sysconfig]# utmpdump /var/log/wtmp | grep 128.99.1.64
Utmp dump of /var/log/wtmp
[7] [03146] [:0 ] [adme ] [:0 ] [ ] [128.99.1.64 ] [Tue Jun 17 13:51:45 2003 ]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Tue Jun 17 14:24:49 2003 ]
[7] [02647] [:0 ] [adme ] [:0 ] [ ] [128.99.1.64 ] [Tue Jun 17 14:29:30 2003 ]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Wed Jun 18 01:51:18 2003 ]
[7] [02678] [:0 ] [adme ] [:0 ] [ ] [128.99.1.64 ] [Wed Jun 18 18:00:39 2003 ]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 01:33:04 2003 ]
[7] [05771] [:0 ] [thc ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 01:33:34 2003 ]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 01:36:57 2003 ]
[7] [06013] [:0 ] [thc ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 01:37:12 2003 ]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 01:46:22 2003 ]
[7] [02724] [:0 ] [thc ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 01:52:15 2003 ]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 02:16:45 2003 ]
[7] [03297] [:0 ] [adme ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 02:16:58 2003 ]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 02:21:46 2003 ]
[7] [02739] [:0 ] [adme ] [:0 ] [ ] [128.99.1.64 ] [Thu Jun 19 18:04:35 2003 ]


that wasnt me :-(

here is the crackers IP:
[root@safari sysconfig]# jwhois 128.99.1.64
[Querying whois.arin.net]
[whois.arin.net]

OrgName: Northrop Grumman Corporation - Automation Sciences Laboratory
OrgID: NGCASL-1
Address: One Hornet Way
City: El Segundo
StateProv: CA
PostalCode: 90245-2804
Country: US

NetRange: 128.99.0.0 - 128.99.255.255
CIDR: 128.99.0.0/16
NetName: NORTHROP-NET
NetHandle: NET-128-99-0-0-1
Parent: NET-128-0-0-0-0
NetType: Direct Assignment
NameServer: NRTC.NORTHROP.COM
NameServer: C15.LN.NET
NameServer: NS.NRTC.NORTHROP.COM
Comment:
RegDate: 1986-01-09
Updated: 2000-11-08

TechHandle: DEC3-ARIN
TechName: Cass, Dwight
TechPhone: +1-310-332-8934
TechEmail: dec@nrtc.northrop.com

# ARIN WHOIS database, last updated 2003-06-18 21:05
# Enter ? for additional hints on searching ARIN's WHOIS database.
[root@safari sysconfig]#


i didnt find more signs that my machine is hacked... how can I find out what the cracker made with my system? please do not answer install your system from scratch.

thx for help

greetz
adme

Checking `lkm'... You have 7 process hidden for readdir command
Warning: Possible LKM Trojan installed


i have Redhat 9

-------------------------

Searching for suspicious files and dirs, it may take a while...
/usr/lib/perl5/5.8.0/i386-linux-thread-multi/.packlist /usr/lib/openoffice/share/gnome/net/.directory /usr/lib/openoffice/share/gnome/net/.order /usr/lib/openoffice/share/kde/net/applnk/OpenOffice.org/.directory /usr/lib/openoffice/share/kde/net/applnk/OpenOffice.org/.order /usr/lib/qt-3.1/etc/settings/.qtrc.lock

Unfortunately the LKM detection on RH8 and up isn't flawless.

Ok. First thing is you need to know anything of the advice below can be thwarted. The only real way to find out is to reboot your system with a bootdisk, have a filesystem integrity checker like Aide, Samhain or tripwire and run it, provided you also had the databases saved off the system onto readonly media. A weak second choice is booting a bootdisk and verify tru the rpm database. This is weak because the database can be tampered with and WILL NOT find files you didn't put on the system.

1. In any case save your complete logdir somewhere safe first, also /etc.
2. check the connections you run: \netstat -anp. This gives you the PIDs, process names and interface of visible listening apps. The preceding backslash is necessary if you need to override any alias called "netstat",
2. check your processes: \ps axfwwwwwwe, this gives you all visible processes in forest format, long commandline and part of their environment. *Saving process info would be a good idea, I run cryogenic (static) but you won't have the time to introduce it,
3. check your interface statistics: ip link show. This should show if any interface is in promiscuous mode, THE REAL WAY (current chkrootkit method is flawed and I've got word from the developer a fix would be out in June).,
4. If you didn't have an integrity checker, the minimal effort is running "find / -mtime -# -print" where # is the number of days +1 you suspect the system changed. This gives you the inodes (file changed).
5. Shut down any network interface and don't reconnect the box to the 'net again until proven safe.

Please examine and post the info, go over your logfiles, recheck your login records with last and lastb, examine /etc/password group and shadow files.

I dont have built an integrity check like tripwire

[root@safari gugus]# find / -mtime -2 -print | tee changes.txt
this is only a abstract of files I think shoudnt have changed:
/lib/modules/2.4.20-8/modules.dep
/lib/modules/2.4.20-8/modules.generic_string
/lib/modules/2.4.20-8/modules.pcimap
/lib/modules/2.4.20-8/modules.isapnpmap
/lib/modules/2.4.20-8/modules.usbmap
/lib/modules/2.4.20-8/modules.parportmap
/lib/modules/2.4.20-8/modules.ieee1394map
/lib/modules/2.4.20-8/modules.pnpbiosmap

/
/dev
/dev/pts
/dev/pts/0
/dev/pts/1
/dev/pts/2
/dev/log
/dev/console
/dev/dri
/dev/dri/card0
/dev/psaux
/dev/ptmx
/dev/shm
/dev/tty1
/dev/tty2
/dev/tty3
/dev/tty4
/dev/tty5
/dev/tty6
/dev/urandom
/dev/initctl
/dev/ptal-printd
/dev/gpmctl

am I right? /proc is virtually and changes often.

with the Redhat Rescue system
[root@safari gugus]# rpm --verify -a | tee rpm.txt
S.5....T c /etc/hotplug/usb.usermap
S.5....T c /etc/sysconfig/pcmcia
.......T c /etc/mail/sendmail.cf
S.5....T c /etc/mail/statistics
SM5....T c /etc/mail/submit.cf
missing /usr/lib/libpq.so.2.0
S.5....T c /etc/openldap/ldap.conf
S.5....T c /etc/krb.conf
S.5....T /usr/lib/openoffice/share/fonts/truetype/fonts.dir
.......T /usr/share/fonts/KOI8-R/100dpi/fonts.dir .......T /usr/share/fonts/KOI8-R/100dpi/fonts.dir
S.5....T c /etc/pam.d/system-auth
..5....T c /etc/inittab
S.5....T c /etc/ldap.conf
S.5....T c /etc/sysconfig/rhn/up2date-uuid
S.5....T c /etc/yp.conf
.......T c /usr/share/fonts/default/Type1/fonts.dir
S.5....T c /etc/cups/cupsd.conf
.......T c /etc/cups/printers.conf
....L... /usr/lib/libglide3.so.3
S.5....T c /etc/xml/catalog
S.5....T c /usr/share/sgml/docbook/xmlcatalog
..5....T /usr/share/fonts/KOI8-R/75dpi/fonts.dir
..5....T /var/lib/wnn/ja/dic/gerodic/g-jinmei.dic
..5....T /var/lib/wnn/ja/dic/pubdic/bio.dic
..5....T /var/lib/wnn/ja/dic/pubdic/chimei.dic
..5....T /var/lib/wnn/ja/dic/pubdic/computer.dic
..5....T /var/lib/wnn/ja/dic/pubdic/full.fzk
..5....T /var/lib/wnn/ja/dic/pubdic/jinmei.dic
..5....T /var/lib/wnn/ja/dic/pubdic/kihon.dic
..5....T /var/lib/wnn/ja/dic/pubdic/kougo.fzk
..5....T /var/lib/wnn/ja/dic/pubdic/koyuu.dic
..5....T /var/lib/wnn/ja/dic/pubdic/setsuji.dic
..5....T /var/lib/wnn/ja/dic/pubdic/special.dic
..5....T /var/lib/wnn/ja/dic/pubdic/std.fzk
..5....T /var/lib/wnn/ja/dic/pubdic/symbol.dic
..5....T /var/lib/wnn/ja/dic/pubdic/tankan.dic

no binarys

(most of these file i know i worked with it)


S file Size differs
M Mode differs (includes permissions and file type)
5 MD5 sum differs
D Device major/minor number mis-match
L readLink(2) path mis-match
U User ownership differs
G Group ownership differs
T mTime differs

as you can see is lastlog inactiv - sadly
$ lastb
lastb: /var/log/btmp: No such file or directory
Perhaps this file was removed by the operator to prevent logging lastb info.





ifconfig: received and trasmitted bytes are in my opinion normally.

in /var/log/ i found nothing suspected

I cant see any abuse of my system?

thank you very much for your help.

find / -mtime -2 -print | tee changes.txt
How come? I told you days+1. Hell, go wild! Make it 5 or more.

am I right? /proc is virtually and changes often.
Yes. It's a virtual FS.

rpm --verify -a | tee rpm.txt
Nothing to see here.

as you can see is lastlog inactiv - sadly
$ lastb
lastb: /var/log/btmp: No such file or directory
Perhaps this file was removed by the operator to prevent logging lastb info.
No, it's usually not just *made* on install.
How about info from "last"?

ifconfig: received and trasmitted bytes are in my opinion normally.
No, you're looking for the "promiscuous" flag. It would either show using "ip" from Kutzenov's ip2route package or plainly *not*. Libpcap apps have a different way of setting the promiscuous flag, and ifconfig doesn't pick that up.

in /var/log/ i found nothing suspected
Did you go tru *all* of the logfiles? The .gz ones too?
Checked all the logs for services you run/ran?

I cant see any abuse of my system?
No, and I can't either with the details you gave.
Doesn't mean there isn't or there is.

*You didn't post all the necessary info, and if I get it correctly and you already rebooted to load the rescue cd then that info (ps|netstat) is gone. Reread my post and try to complete the remaining tasks with the comments from this one.

I know unSpawn is the security expert here, so I'm only going to say three things; 1) Follow unSpawn's directions to the letter, 2) Just for kicks, look at the history file of the cracked accounts, stupid crackers forget to modify/erase the history file, and 3) if you can, disconnect the computer from the network while you are playing catch up with the cracker.

I was hacked exactly the same way twice already...

I performed a fresh installation of Red Hat 9 and noticed the problem (remote login) from the same IP address at Northrop... In my case the entry in wtmp appears as a root login, what made me feel even more unconfortable...

As I could not find the reason, I've decided to reinstall the system from scratch and... I was hacked again from the same IP address in less than one day!!!

How is it possible? I performed the installation from the ISOs I've got from Red Hat FTP and I haven't installed nothing in this new system (I used the default DESKTOP install). Also, iptables security was set to HIGH with the default exception leaving the DHCP open...

they do say that you should ALWAYS format, reload after a confirmed crack on a system, purly because you have no SURE way of knowing that your system has been totally cleaned of any backdoors etc. etc. :|, sorry, not that helpful I know, but its probably the best thing to do in a situation such as this

Thanks anyway, but the second install was really from the scratch, including formating the partitions...

Just a shot in the dark, but did you install the latest errata/patches from the Redhat website. There are some known exploits available for unpatched installs, that would make hacking in a second time relatively trivial. Also, you might want to explicitly DROP all packets from the Northrop/Grumman block of IP addresses via iptables, though it's almost a certainty that those systems are hacked as well and are just proxies for the attack. I'm also assuming that you did pick a new root password and don't have vulnerable services (telnet, portmapper, nfs) turned on.

Well, the patches / erratas download and installation were just the first thing I was doing with the system as soon as the installation finish... I leaved the system downloading and just before the patches were installed... there was our guy again! So I haven't a time frame to install the patches.

I did changed the root's password in the machine but I really didn't add the IP address in any iptables rule... Maybe because I trusted that was enough to leave it in HIGH level of security. Anyway, now there's already a second address in the wtmp and the system was shutdown again, ready to be reinstalled!

This time I'll try to previously download all patched packages / erratas and install them before connecting the machine. I also will block the 2 known address...

Anyway, I thougth the default HIGH level of security of IPTABLES should block all incoming traffic non explictly allowed...

The HIGH setting is alright, but is by no means an absolute brickwall against attacks, especially against someone as persistant as your "friend". Downloading the latest errata and burning it to a cd on a seperate system sounds like a good start, make sure to physically unplug your network connection untill you have it installed.

As for iptables, just add a rule to DROP anything from that IP block. As root:

iptables -I INPUT -p all -s 128.99.0.0/16 -j DROP

That along with lokkit set to HIGH should hopefully help. You might also want to send an email to Northrop's tech (name is at the bottom of the whois query). Given that it's a smaller IT department, you might actually get somebody to do something on their end. It's also probably a bad thing that a major US defense contractor saves their data on 0wn3d systems. But then again, if they clean that system, you'll probably start getting attacks from other "proxies". If you have another linux box on the network, you might want to fireup ethereal/tcpdump and see if you can sniff HIS traffic to get an idea of what he's trying to do.
HTH

i think this is a bug in wtmpx from Redhat.

kind regards

adme: Which bug? I don't see anything in bugzilla that would cause that, but I could be wrong.

fanton: Ignore the northrop part and substitute the offending ip address for 128.99. I didn't realize that you weren't the original poster.

Strange things may happen...

Continuing my adventure, today I've installed Red Hat 9 from the scratch for the 3rd time.

I took all cares before install the system:

- unplugged the netword cable before performing the installation
- choosed a new password to use with root
- choosed to format the partitions again...

Again I decided for the standard DESKTOP installation and iptables with the HIGH security level leaving the DHCP open (still without the cable...)

I performed the install and entered X to update the packages downloaded previously from RHN... But before I start, Ie decided to make a dump of wtmp and... SURPRISE!!! There was already a record of a logon from the 128.99.1.64!!!

Here's the dump:

[fanton@ganimedes-lx fanton]$ utmpdump /var/log/wtmp | grep 128
Utmp dump of /var/log/wtmp
[7] [03172] [:0 ] [root ] [:0 ] [ ] [128.99.1.64 ] [Sat Jul 05 10:32:00 2003 BRT]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Sat Jul 05 11:25:42 2003 BRT]
[7] [03077] [:0 ] [root ] [:0 ] [ ] [128.99.1.64 ] [Sat Jul 05 11:29:16 2003 BRT]
[8] [00000] [:0 ] [ ] [:0 ] [ ] [128.99.1.64 ] [Sat Jul 05 11:43:57 2003 BRT]

Let me remember you that I AM STILL WITH THE NETWORK CABLE DISCONNECTED!!!

Well, it seemed to me that something is wrong from the inside, that no one was really connecting to the system (no wonder there are no traces of invasion...), but something really strange is taking place...

So I've noticed that this problem just happens in the machine I've installed as a desktop, that is, the only machine that boot's in X Windows. So I've changed the default run level in inittab from 5 to 3 and guess... Yes, the logon from the "bad guy" stopped to appear, even after I enter X Windows!

(Before I did that there's a logon from 128.99.1.64 at every boot!
Most strange thing, the logon was being registered as taking place from terminal :0 and not a PTS, meaning that someone has to be sitting at my chair along with me!!!)

So, not entering runlevel 5, that is, not passing by xdm, avoids the problem. But what is it? What's causing it? There's a trojan inside the RH9 distribution? Is there a bug or problem in xdm in RH9?