Boot hangs after SELinux: Disabled at runtime

DaveJL · 03-08-2010, 05:36 PM

Hello,

My Redhat box hangs on boot up after "SELinux: Disbled at runtime" ...

Code:

raid1: raid set md0 active with 2 out of 2 mirrors
md: ... autorun DONE.
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
SELinux:  Disabled at runtime.
type=1404 audit(1268063367.600:2): selinux=0 auid=4294967295 ses=4294967295

This boot hanging problem started after I ran /etc/rc.d/rc.sysinit to reset the hostname, but the power went out before the script completed.

I'm suspecting that the hard kill messed up some boot files, and I'm hoping a kind & wise soul can help guide me on how to restore them properly.

Thanks very much.
Dave

John VV · 03-08-2010, 06:16 PM

off hand i would guess that the hang is the file system rewriting the se context to the WHOLE drive
this can take anywhere from 5 min to 2+hours depending on the size of the drive/s

to temporally turn off SE use the command " setenforce 0 "

DaveJL · 03-08-2010, 06:32 PM

Thanks for the ray of hope, John.

Yeah, just now (after 2 hours), the boot output pushed an inch forward to include "RAID1 conf printout":

Code:

SELinux:  Disabled at runtime.
type=1404 audit(1268063367.600:2): selinux=0 auid=4294967295 ses=4294967295
RAID1 conf printout:
 --- wd:2 rd:2
 disk 0, wo:0, o:1, dev:sda3
 disk 1, wo:0, o:1, dev:sdb3

But, like before, it's hanging here again.

So, does the boot process appear to be progressing properly, such that it's just a matter of time before the boot completes properly? (The HD size is 500 GB.)

Thanks again,
Dave

John VV · 03-08-2010, 06:48 PM

that is what happens every time you turn SE no and off
the driver need to be updated

that is why " setenforce 0" is preferd for temporally turning it no and off.

DaveJL · 03-08-2010, 07:21 PM

Ok, thanks.

Just for the record, I did not purposely change the SE setting.
After the hard-kill during /etc/rc.d/rc.sysinit , the boot-up started to hang like this. I didn't even know what SE was before this problem.

Should I just let it continue to hang and hope it eventually finishes whatever it is doing?

Thanks again,
Dave

DaveJL · 03-08-2010, 10:44 PM

I'm now in Recovery Mode.

My /etc/fstab :

Code:

/dev/md1                /                       ext3    defaults,usrquota        1 1
/dev/md0                /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
LABEL=SWAP-sdb2         swap                    swap    defaults        0 0
LABEL=SWAP-sda2         swap                    swap    defaults        0 0

fdisk -l yields:

Code:

# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0001c6cf

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   fd  Linux raid autodetect
/dev/sda2              14         655     5156865   82  Linux swap / Solaris
/dev/sda3             656       60801   483122745   fd  Linux raid autodetect

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0002ec98

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          13      104391   fd  Linux raid autodetect
/dev/sdb2              14         655     5156865   82  Linux swap / Solaris
/dev/sdb3             656       60801   483122745   fd  Linux raid autodetect

Disk /dev/md1: 494.7 GB, 494717566976 bytes
2 heads, 4 sectors/track, 120780656 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table

mdadm -E -s shows:

Code:

# mdadm -E -s
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=629bfb37:53a8883f:537f768e:e9290991
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=adc78fa5:fac6da39:5ea2599b:29ca6912

When I try to mount /dev/md0, I get the error:

Code:

# mount -t ext3 /dev/md0 /mnt
mount: wrong fs type, bad option, bad superblock on /dev/md0,
       missing codepage or helper program, or other error
       (could this be the IDE device where you in fact use
       ide-scsi so that sr0 or sda or so is needed?)
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

dmesg | tail shows:

Code:

# dmesg | tail
[ 1349.012161] unionfs: new lower inode mtime (bindex=0, name=mdadm)
[ 1356.803854] md: md1 still in use.
[ 1356.870311] md: bind<sdb3>
[ 1356.906846] raid1: raid set md1 active with 1 out of 2 mirrors
[ 1356.906867] md1: detected capacity change from 0 to 494717566976
[ 1356.906978]  md1: unknown partition table
[ 1389.504015] EXT4-fs (md0): unable to read superblock
[ 1398.655582] EXT3-fs: unable to read superblock
[ 1808.394513] EXT2-fs: unable to read superblock
[ 1836.473091] EXT3-fs: unable to read superblock

When I try to fsck /dev/md0, I get:

Code:

# fsck /dev/md0
fsck 1.40.8 (13-Mar-2008)
e2fsck 1.40.8 (13-Mar-2008)
fsck.ext2: Invalid argument while trying to open /dev/md0

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

I'm not sure how to proceed ... any guidance would be much appreciated.

Thanks,
Dave