I am running Linux Redhat Guiness 7.0 on a PIII 700 with an 810 chipset set in multicast broadcasting. Recently I had a hacker, that was immediately detected after he had created a superuser "b". After a complete check the superuser "b" was removed and the netowrkd card was taken out of the promiscuous mode. Nothing else seemed to have been changed or installed.

When the reboot command was executed the server hung and would not reboot. The server was manually switched off and then booted back up. During the reboot process the server got hung up after completing all the directory file checks and would not finish booting up. Ctrl+Alt+Delete was pressed to execute a reboot and an upgrade install was attempted. but could not be done since all filesystems had not been unmounted cleanly.

The server was rebooted but this time would not go past the filecheck for /root. Then the server was rebooted into the linux rescue mode where I attempted to follow the instructions at:
jezndi.org/linuxrec.html

On reboot the server repeated the "Partition check" and the "VFS" line adding "readonly" the second time as such:
VFS: Mountedroot (ext2 filesystem) readonly.
change_root: old root has d-count=1
trying to mount old root ... OKAY
Freeing unused kernel memory: 64k freed
Warning: unable to open an initial console.
Kernel panic: No init found. Try passing init= option to kernel.

The I rebooted the server using the rescue floppy and excuted:
linux init=/bin/bash

This now produced the following result:
Partition check:
hda: hda1 hda2 < hda5 hda6 hda7 hda8 hda9 >
autodetecting RAID arrays
autorun ...
... autorun DONE.
VFS: Mountedroot (ext2 filesystem) readonly.
Freeing unused kernel memory: 76k freed
Warning: unable to open an initial console.
Kernel panic: No init found. Try passing init= option to kernel.

It may be noteworthy that if you now try to do an upgrade install you receive the prompt that no partitions are found and that when you exit from linux rescue you also have this line:
/mnt/source unmount failed (16)

During the rescue mode I have verified that /dev/console does exist on the harddrive.

It would be appreciated if someone could let me know what the commands would be to get this server to boot up.

Thanks,
Ed

"It would be appreciated if someone could let me know what the commands would be to get this server to boot up."

The first thing to do is to clean up your file systems which are probably damaged. Boot your rescue CD and run fsck against every partition except swap. Be sure that you have the filesystem type correct or else fsck can do more harm than good if it does not auto recognize the file system correctly.

Do you have a backup which was taken before before the intrusion?

___________________________________
Be prepared. Create a LifeBoat CD.
http://users.rcn.com/srstites/LifeBo...home.page.html

Steve Stites

Hi Steve,

The problem is getting the box to properly start the kernel on the hard drive...

When you run rescue you first need to mount all the partitions... If you boot normally the partitions mount but the kernel does not seem to find where / is so it can not find /sbin/init to mount /dev/console. Since the hacker used the new SSH attack method which also compromises the kernels of Linux 9.*, this kernel path is most likely one of the specific problems.

We will be using rescue today to manually mount all the file systems, run the file check, then do an update install of the old kernel, and finally update and patch the kernel with the latest version of the kernel and SSH patch. In order to use our backup I believe that we would still have to go throught this rescue procedure.

If you have a better suggestion please le me know...

Thanks for your reply...

Ed

"When you run rescue you first need to mount all the partitions..."

No. You must run fsck against unmounted partitions. Do not use Knoppix for your rescue CD because it looks for partitions and tries to mount them. Use a rescue CD which does not try to mount partitions. Then run fsck -t (filesystemtype) /dev/hdxy against every partition but swap. Or if you use Knoppix then the first thing that you do in Knoppix is unmount all of your hard drive partitions.

"If you boot normally the partitions mount but the kernel does not seem to find where / is so it can not find /sbin/init to mount /dev/console. "

I think that one of your problems is that there are probably errors in / that need to be fixed with fsck before you can use / as a mounted file system.

"then do an update install of the old kernel, and finally update and patch the kernel with the latest version of the kernel and SSH patch."

You may or may not have to do these things after you run fsck.

"In order to use our backup I believe that we would still have to go throught this rescue procedure."

Yes. Probably. Actually if you are going to completely restore a partition then you should run mkfs and recreate the partition file system before you do the restore.

___________________________________
Be prepared. Create a LifeBoat CD.
http://users.rcn.com/srstites/LifeBo...home.page.html

Steve Stites

sorry this is not a reply ..I m just subscibing for the thread

Sorry for the inconvenience caused

Thanks Steve... we must have missed something because everytime we tried to reboot some data files got deleted. We are installing a new OS and will be applying some custom patches to the kernel and SSH. Unfortunately we need to run multicast broadcasting and we found that 7.0 is best for this.

Sure wish I could get a look at the new version of ptrace-kmod.c that this hacker used... I have the old version but based on what we recorded in the logs it has been improved.

Thanks again for your help.

Ed