Dear Ladies and Gentlemen,

Big Problem
I have a server with the last five years of my work running FC4 which I recently shutdown and rebooted as I wanted to relocate it as a result of an office move. The following error appeared on start up: "cannot find / in /etc/fstab or /etc/mtab - remounting root filesystem in read-write mode".
What I am a little ashamed to admit at this point is that I was 'fiddling' around with the fstab file shortly before the 'last' shutdown.

Amateur Self-Diagnosis
While I thought I was being careful, I can only think that somehow I managed to delete the lines in /etc/fstab referring to the '/' and '/boot' mount points. Very careless, I hear you say... and 'to boot', I do not appear to have any backup of /etc/fstab.
The only alternative that I think is feasible is that I interupted the mandatory fsck which follows a reboot after a server has been running for >100 days or so (the server inquestion had been up for 147 days ish), and that this has caused the filesystem to remount read-only.

My troubled server is called 'mother', and a df following an error-riddled restart now looks like this:

[root@mother ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/shm 1915288 104 1915184 1% /dev/shm
/dev/hde1 7937796 4968436 2566132 66% /home
/dev/mapper/vg1-lv1 7937796 4968436 2566132 66% /var/spool/mail
/dev/sdb2 7937796 4968436 2566132 66% /var/lib/pgsql/data
/dev/sda2 7937796 4968436 2566132 66% /var/lib/pgsql/log
df: `/home/public': No such file or directory

The /etc/fstab file looks like this:
# This file is edited by fstab-sync - see 'man fstab-sync' for details
/dev/devpts /dev/pts devpts gid=5,mode=620 0 0
/dev/shm /dev/shm tmpfs defaults 0 0
/dev/proc /proc proc defaults 0 0
/dev/sys /sys sysfs defaults 0 0
/dev/sda6 swap swap defaults 0 0
/dev/hde1 /home ext3 defaults 1 2
/dev/vg1/lv1 /var/spool/mail ext3 defaults 1 2
/dev/sdb2 /var/lib/pgsql/data ext3 defaults 1 2
/dev/sda2 /var/lib/pgsql/log ext3 defaults 1 2
#daughter:/home/public /home/public ext3 defaults 1 2
#daughter:/home/postgres /home/postgres ext3 defaults 1 2
#daughter:/home/public/archive/mother_home/opt/jakarta-tomcat-5.5.9/ /home/opt/jakarta-tomcat-5.5.9 ext3 defaults 1 2
/dev/fd0 /media/floppy auto pamconsole,exec,noauto,utf8,managed 0 0
/dev/hda /media/cdrom auto pamconsole,exec,noauto,managed 0 0
[root@mother ~]#

When I boot from a linux rescue disk and allow it to try to find existing fedora installations, it reports an error attempting to

mount the filesystem, and a df following arrival at the command prompt this looks like this:

/dev /dev
/tmp/loop0 /mnt/runtime
/dev/hde1 /mnt/sysimage/home
/dev/sdb2 /mnt/sysimage/var/lib/sql/data
/dev/sda2 /mnt/sysimage/var/lib/sql/data

I have two other servers: one called 'sirocco' also running FC4 (Linux sirocco.cassiancapital.com 2.6.17-1.2142_FC4 #1 Tue Jul 11

22:41:14 EDT 2006 i686 i686 i386 GNU/Linux) and a new machine called 'daughter' running Fedora 11.

Trying to solve the problem myself, I took a look at the /etc/fstab for sirocco and daughter below by way of analogy, to find that

both have references to "/" and "/boot" in the detail:

[root@sirocco ~]# more /etc/fstab
# This file is edited by fstab-sync - see 'man fstab-sync' for details
/dev/VG00/LV00 / ext3 defaults 1 1
LABEL=/boot /boot ext3 defaults 1 2
/dev/devpts /dev/pts devpts gid=5,mode=620 0 0
/dev/shm /dev/shm tmpfs defaults 0 0
/dev/proc /proc proc defaults 0 0
/dev/sys /sys sysfs defaults 0 0
/dev/VG00/LV01 swap swap defaults 0 0
/dev/hda3 /home ext3 defaults 0 2
[root@sirocco ~]#

[root@daughter by-path]# more /etc/fstab
#
# /etc/fstab
# Created by anaconda on Fri Sep 25 17:26:02 2009
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or vol_id(8) for more info
#
UUID=7fe47370-ddf9-4952-9bf4-be887d1b8d8c /boot ext3 defaults 1 2
/dev/mapper/vg0-lv0 / ext4 defaults 1 1
/dev/mapper/vg1-lv0 /home ext4 defaults 1 2
/dev/mapper/vg0-lv1 swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
#devpts options modified by setup update to fix #515521 ugly way
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0

Cut to the Chase
In order to get mother working properly again, my conclusion and best guess from the investigative work above is that I have to add

two lines back into /etc/fstab as follows:
/dev/vg0/lv0 / ext3 defaults 1 1
/dev/sda1 /boot ext3 defaults 1 2

But I'm s**t scared of doing this without some expert guidance for fear of losing everything for good and, besides, I can't seem to

edit /etc/fstab anyway which is now resolutely claiming to be permanently read-only.

Please help!!

Best,

Nick W (Knee-co)

Appendix:
[root@mother ~]# fdisk -l

Disk /dev/hde: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hde1 1 14593 117218241 83 Linux

Disk /dev/sda: 18.1 GB, 18198822912 bytes
255 heads, 63 sectors/track, 2212 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 12 96358+ 83 Linux
/dev/sda2 13 62 401625 83 Linux
/dev/sda3 63 572 4096575 83 Linux
/dev/sda4 573 2212 13173300 f W95 Ext'd (LBA)
/dev/sda5 573 1081 4088511 83 Linux
/dev/sda6 1082 1208 1020096 82 Linux swap / Solaris
/dev/sda7 1209 2212 8064598+ 83 Linux

Disk /dev/sdb: 36.3 GB, 36397645824 bytes
255 heads, 63 sectors/track, 4425 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 1020 8193118+ 83 Linux
/dev/sdb2 1021 4425 27350662+ 83 Linux

Disk /dev/dm-0: 8321 MB, 8321499136 bytes
255 heads, 63 sectors/track, 1011 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/dm-0 doesn't contain a valid partition table
[root@mother ~]#

Well you are right

it sucks to be you.

Not that I can be much help but have you made a backup copy of the drive? Something like

dd if=/dev/sda of=/dev/some/backup/drive.image

Then you can play till your hearts content and you still have your data. If you are not familar with dd you might look at

http://www.linuxquestions.org/questi...ommand-362506/

If you could determine just where the fstab file was located on the disk you may be able to recover it with dd.

Hope that helps.

thanks for your response... does it sound to you more likely that I mis-edited the fstab file, or that I have some sort of file corruption?

nick

It's likelly you just accidentally removed the "/" and "/boot" lines. BTW. The reason why your rescue disk failed to mount the partition is likelly because your boot and / partitions are in a LVM. Get your self a liveCD with LVM support and see if you can mount your partitions.

What I think I would do is to make a backup image first. Almost any live-linux-cd will allow you to do that.

Then from in the live-cd I would look at the contents of the different partitions to see which part of the file structure they contain and write it down.

You should then be able to build a fstab file that corresponds to what is on the disk. Copy it in place and try it. If it works great. If not back up and recheck everything and try again.

You can always put the original image back on the disk from the backup image you made in the first step.

Most live-cd's have gparted or similar on them that will allow you to look at the partitions on the drive (BUT DON'T CHANGE THEM) and that may help in getting fstab rebuilt. You may also take a look at the mtab you have. Not sure if the system will automagicly rebuild it for you or not. So you may have to rebuild it too.

Just be sure to make your backups first. Did I mention that you need to make backups first.

guys - thanks so much for the tips - I'm going to have a go with what you suggest once I'm back from a brief work trip abroad. They have given me the confidence to believe that not all is lost!
I'll report back once I've had a go with a live cd.

Nick

Gents,
Thanks for posting replies and for pointing me in the right direction.
I manged to boot from a live CD (knoppix) and, once I found which partition /etc/fstab was on, edited it accordingly.

It turns out that I had indeed carelessly deleted the following line the last time I edited the file:

/dev/sda7 / ext3 defaults 0 0

Anyway, 'situation normal' now and blood pressure stable.

Thanks again

Nick

Glad to hear it