Corrupted /etc/fstab

When I rebooted my comp a few minutes ago, it came to a halt saying "Kernel panic: init not found. Try passing init= option to the kernel".

I used the rescue CD to mount my root filesystem to see what has happened. I chrooted to /mnt/sysimage to check and see if my fstab was okay, since that was the last thing I changed ( I added a a new line for a a non-root primary partition to be mounted under /mnt/. I am absolutely positive that I did not do anything to the line that mounts the root partition).

Trying to access the fstab file ( which I believe was corrupted by some weird thing that happened before I rebooted ) I got an input/output error.

I tried
rm -f /etc/fstab
and
cat > /etc/fstab
even
ls -l /etc/fstab

all say "fstab : Input/output error"

fsck.ext3 found the root partition /dev/hda6 clean.

What I suspect is happening here is that when the kernel is trying to mount the root partition, it barfs because it cant read the fstab. And hence it cant load /sbin/init.

Once I could remove the fstab, I could write a new one alright... but how can I remove/unlink it?

Maby you should try to overwrite fstab by copying a file with the same name to the same directory. Hope it helps

Thanks for the suggestion , but nope. I had tried that already.
Cannot stat fstab. I/O error.

How did you fs check your partition. Did you give the -f option. If the partition is marked as "clean", fsck will not really check the file system.

Thanks bud, that was the tip I needed. I was wondering why fsck didnt take too long to finish. the -f option cleared the i/o error, removed the fstab inode.

Now I have a barebones fstab that reads..
/dev/hda6 / ext3 defaults 1 1
/dev/hda1 /boot ext3 defaults 0 0
/dev/hda5 swap swap defaults 0 0
none /dev/pts devpts gid=5, mode=620 0 0
none /proc proc defaults 0 0

/dev/hda6 is my root partition, and I can see /sbin/init when I mount it from the rescue disk.
but when I reboot, the kernel still cant find my init. Wonder why!!?

Have you tried to pass the init= option to lilo?
How was this fstab file made? Did you edit it, or was it generated somehow?
Here is my fstab:

# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/hda4 / ext2 errors=remount-ro 0 1
/dev/hda6 none swap sw 0 0
proc /proc proc defaults 0 0
/dev/fd0 /floppy auto user,noauto 0 0
/dev/cdrom /cdrom iso9660 ro,user,noauto 0 0

as you can see, my root has different dump and pass parameters. I don't know if that matters though.

About the difference between your fstab and mine, I think the parameters are distro-specific. I'd tried to pass init= option at the lilo prompt, but it never made any sense to me, because my init was at the default location. The kernel searches for /etc/init, /bin/init, /sbin/init, /bin/sh in that order and panics if none of these could be found. So I had a hunch that somehow my root-filesystem is not being mounted right. So I tried changin the ext3 to ext2 and rebooted.. just to see what happens.. and voila.. it booted right up, like nothing had happened. As you can see from the tune2fs dump of the superblock, the goddamn filesystem has a journal.. and it mounted just fine from the rescue CD prompt as explicit type ext3. But somehow the kernel can mount it ext2, but cannot mount it ext3. The ext3 support is complied in as a module, do you think that's probably why? Tonight I'm going to complie ext3 support "in" and try with that one. But it was the same kernel that used to boot it till day before y'day as ext3, so it is baffling. But I was using an initrd back then.


(01:04am)root@FLOATINGM:~# /sbin/tune2fs -l /dev/hda6
tune2fs 1.27 (8-Mar-2002)
Filesystem volume name: /
Last mounted on: <not available>
Filesystem UUID: b18594ba-8a76-11d6-814f-c51f36cf5305
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal filetype sparse_super
Filesystem state: not clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 9322496
Block count: 18642952
Reserved block count: 932147
Free blocks: 16456401
Free inodes: 9013426
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16384
Inode blocks per group: 512
Last mount time: Fri Nov 29 00:52:16 2002
Last write time: Fri Nov 29 01:02:42 2002
Mount count: 2
Maximum mount count: -1
Last checked: Thu Nov 28 23:09:17 2002
Check interval: 0 (<none>)
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal UUID: <none>
Journal inode: 8
Journal device: 0x0000
First orphan inode: 0
(01:04am)root@FLOATINGM:~#

Whow, I really don't understand what went wrong there. Is your patition being mounted now?

Yes, it is being mounted as ext2 ( that's what it says when it boots up. Is there a way to know for sure? ). It baffles me too; I'm feeling kind of uneasy to see that I dont have 'full' control over its behavor yet.

The kernel doesn't know how to read your ext3 filesystem (because the
support is built as a module, not built in), so it gets to the filesystem, and
barfs. Always build in support for the type of filesystem your root is,
since that's the first thing mounted, and if Linux doesn't know what kind of
filesystem it is, it can't mount it. . .

However, the kernel can mount an ext3 filesystem as ext2 (not vice versa),
when you tell it that it should mount an ext2 filesystem, it has no problems.

I'm tired, so some of this may be inaccurate. If so, I apologize, and
probably shouldn't have posted this until I'm more awake. . .

Looks like what type ( ext2/ext3) the root filesystem is listed as in /etc/fstab doesnt really matter. It is auto determined by the kernel, no matter if the necessary modules for mounting the determined type are present or not. All of my problems were caused by the initrd. The ramdisk virtual filesystem created from initrd.img ( The redhat kernel makefile uses /sbin/mkinird to create this for the kernel that's being compiled ) during the boot process didn't have the ext3 module, but that was mounted as the root filesystem anyway. Later on, when it was time to mount the real root filesystem and pivot_root to it, the kernel barfed because this particular initrd didn't have the ext3 module whereas it should have. Once I compled the kernel with ext3 support built in and passed a noinitrd option to the booting kernel, it booted just fine. This is my conclusion, but it is bounded by my limited knowledge about 'initrd's and how exactly they work.

Any further clarifications are welcome. Thanks for the support, guys.

I wish I had used this particular set of search words-- oh about 13 hours ago!!!!!! I hate initrd. But, I suppose if they want a kernel to install on systems with added IDE controllers and such, it must be. However, it would appear that there are a few bugs in the the scripts which can make things inconsistent.

I am suffering right now from one. Someone's faulty script wrote inode tables and a default cluster size of zero on my Dos/win98se partition. I have been trying to recover my stuff and am still looking for a template for fixing it in hex. RH9.0

So what does this mean then? It's all pretty vague... It says the file system was modified, but doesn't say how or what... (BTW I'm using Knoppix to check a gentoo partition).

root@ttyp0[knoppix]# fsck -f /dev/hdb3
fsck 1.34-WIP (21-May-2003)
e2fsck 1.34-WIP (21-May-2003)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 3A: Optimizing directories
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/hdb3: ***** FILE SYSTEM WAS MODIFIED *****
/dev/hdb3: 130017/3933120 files (1.2% non-contiguous), 521322/7865825 blocks
root@ttyp0[knoppix]#

Any ideas on how to get a more comprehensive output?

Q

Try the -V option for more verbosity. See man fsck.

The red hat initrd is an image in the boot directory. So when you make any changes at all you can interfere with the the process of unmounting the ramdisk and mounting the actual root "/" directory. Initrd is--as far as I know--ext2, and is an image of a root directory--but not necessarily an image of your root directory. If important files don't match exactly what is in the image, it cannot be unmounted--leading to kernel panic. In my limited experience, the error message was the same as yours. I ended up having to boot up on "rescue"--the boot disk didn't boot--to manually salvage all of the updates and data into an unused partition, and reload everything. (I figured the reload would take less time than anything else. I could be wrong.)
Another thing I found, is that the scripts for the up2date program left out the final "LILO" comand. ( This has since been fixed--I think. I run the command anyway before rebooting.)
So, when you do anything which changes fstab or mtab (or several other rather important files) you also have to "make initrd" ; follow the directions exactly in the man page. I understand that the scripts are very complicated, but I think a lot of this stuff was easier when it was done manually because I usually could retrace my steps and "back out" of one of my screw-ups.
Another thing which really got my gander up is the program which allows the linux filesystems to be read from windows. It does not mount/unmount cleanly--at least the version I had--and ended up corrupting both windows and linux. That is why I had to re-load everything. (Of course I didn't have current Images of my partitions--that's something I tell everybody else to do!)