Unclean shutdown from power failure... filesystem check errors

macisaac · 12-04-2003, 06:09 PM

I have a linux server running redhat 9.0 (kernel 2.4.20) and experienced a power failure yesterday. Consequently, the machine shutdown uncleanly and upon reboot prompted me with:

> Yours system appears to have shut down uncleanly
> Press Y within 5 seconds to force file system integrity check .....

I pressed Y and it performed the system check. Then I was prompted with:

> ***An error occurred during the file system check.
> ***Dropping you to a shell; the system will reboot
> ***when you leave the shell.
> Give root password for maintenance
> (or type Control-D to continue):

I entered the root password and was provided with:

bash: id: command not found
bash: id: command not found
bash: id: command not found
bash: [ : too many arguments
(Repair filesystem) 1 #

I then ran e2fsck on all of the partitions (with the -y flag) except root and swap. All partitions were reported as clean. I then rebooted the computer. The root and swap file systems were checked as being clean and mounted with an [OK] indication. The remaining filesystems were checked as clean, but [FAILED]. The system then dropped back to you:

> ***An error occurred during the file system check.
> ***Dropping you to a shell; the system will reboot
> ***when you leave the shell.
> Give root password for maintenance
> (or type Control-D to continue):

At this point I am unsure how to proceed.

Here are the filesystems I have in place:

/boot hda1
/swap hda2
/ hda3
/home hda5
/usr hda6
/usr/local hda7
/tmp hda8
/var hda9

I am still relatively new to linux, but have been working hard on the system, as it is providing domain control via samba, networking, and daily backups of data directories from other machines (win and mac) in our lab. As such, I need to get the system up and running again as quickly as possible.

Any and all help would be greatly appreciated.

Thanks in advance.

yapp · 12-05-2003, 04:06 AM

type this to check your root partition:

Code:

mount / -o remount,ro
e2fsck -f -v /dev/hda3

If you want to test for bad blocks/sectors:

Code:

e2fsck -f -c /dev/hda3

Then change the partition type to ext3 right away:

Code:

tune2fs -j /dev/hda3

Update /etc/fstab, set the file system type to ext3.
perhaps you need to reboot, but maybe this command works:

Code:

mount / -o remount,rw -t ext3

it remounts your drive in read-write mode again, and sets the file system type to ext3.

If that worked, tell init it should enter the runlevel for graphical login:

Code:

telinit 5

you can only perform these actions while you're in single-user mode. (which is the case right now) If you're not in single user mode, type "telinit 1" to enter this mode. It closes all programs and network connections.

macisaac · 12-05-2003, 02:13 PM

Thank you for the advice. The filesystems were already formatted as ext3, so I remounted root and ran e2fsck as you indicated. Still the same problems at reboot.

Then I changed to runlevel 5 (as indicated), it complained for a while, and then tryied to start x, but failed. It also tried to create a new x config file, but did not work. At present, it looks like I'm in runlevel 3. But I think my machine is possessed. My guess right now is that my filesystems are clean, but some of my files have be corrupted/deleted.

Just to Note:
Prior to your response I also started the machine in rescue mode. In rescue mode, all of my filesystems were mounted to a /mnt/sysimage directory, and all data was recoverable. However, in rescue mode I discovered I was missing a number of files, e.g. fstab. I created a new fstab file (which I had to do to run e2fsck properly in rescue mode). Now that I am back to a runlevel 3 mode , I inspected my /etc/fstab and it is the old one.

I'm confused, and I'm not too sure what my machine is doing.

Any ideas?

Cynthia Blue · 12-26-2003, 06:41 PM

I have just had a similar problem. I did have my system safeguarded to shut down automatically after being powered by the UPS for five minutes, but something went wrong and it didnt shut down cleanly.

However, I ran the "fsck -y / " and it seems to have come up clean. Is there anything which I need to do now? I'm running RH 8 and the system has been very stable for a year. Will the fixed it performed allow the machine to continue it's stability? Or is there anything I need to do (other than make sure my auto-shutdown works)?

I can reach my web and mail servers okay now. Our power was out for a brilliant 12 hrs.

Thanks.

-- Edit I take that back. My web server is working, but my mail server is not. Checking more for details....

Cynthia Blue · 12-26-2003, 07:50 PM

Okay strangely enough, my POP3 mail server is working, but my IMAP is not. Looking at services, I cannot start or stop IMAP, and it says in the comments "You must enable xinetd to use this service." Hrm, it was working fine before. I restart xinetd and it doesnt change this message. I'm using the default IMAP that comes with Red Hat 8.

I'd love some help.

I changed my UPSMON to shut down auto after one minute. Hope that helps too.