superblock error after resizing logical volume

austin_it · 10-19-2007, 12:07 PM

I'm running Red Hat Enterprise Linux on a Dell PowerEdge 1800 server, RAID 5.
I recently used the Logical Volume manager to decrease the size of one of the volumes (/dev/VolGroup00/home).
I unmounted it prior to resizing.

But now I get an error when Linux is loading:

The filesystem size (according to the superblock) is 3779584 blocks
The physical size of the device is 379904 blocks
Either the superblock or the partition table is likely to be corrupt!
UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY
(i.e. without -a or -p options)
[FAILED]

***an error occurred during the filesystem check
***dropping you to a shell. the system will reboot
***when you leave the shell

I've tried to restore the superblock's backups using the e2fsck -f -b <backup_block> <volume> command, but that didn't work.

Is there a way to change the superblock size manually?

I was able to edit /etc/fstab to prevent Linux from performing the filecheck on this volume, so I got Linux to load. Now I'd like to resolve the fsck error.

Would it make sense to delete this logical volume, create a new one w/ the same name & properties, then restore its data from a backup tape? I'm a little hesitant to delete the volume if restoring from a backup won't work.

Also, I'm not familiar w/ some of the options for the lvcreate command (such as the stripes options).

I'm pretty much a newbie w/ Linux, but am willing to listen and learn.

oskar · 10-20-2007, 06:16 AM

What didn't work with fsck? What's the error?

I think gpart should be able to reconstruct superblocks too, but I'm not entirely sure.

austin_it · 11-28-2007, 09:42 AM

I did finally get this problem resolved.
The root of the problem was that I used Webmin to reduce the logical volume size.
Webmin did not reduce the file system size prior to reducing the logical volume size. Just a warning for others out there.

Here's how I resolved:

1. Created new Logical Volume (e.g. /dev/VolGroup00/newLV). I used the Gnome Logical Volume Manager, but lvcreate would have worked as well.

2. Mounted new Logical Volume to a mount point

Code:

# mount /dev/VolGroup00/newLV /mnt

3. Check for open files on the old file system/logical volume and kill them

Code:

# lsof /old_fs_mountpoint
# kill <pids_from_previous_command>

4. Use cpio to perform block copy of entire file system from the old volume to the new volume. Using cpio will ensure that all files and directories, even ones that begin with (.) will be copied. I was fortunate that most of the data in my old volume was retained. I only had to restore one directory tree using a tape backup (see Step 7).

Code:

# cd /old_fs_mountpoint
# find -xdev | cpio -pvdam /mnt

5. Unmount volumes and remount new volume to old volume's mountpoint

Code:

# umount /old_fs_mountpoint
# umount /mnt
# mount /dev/VolGroup00/newLV /old_fs_mountpoint

6. Edit /etc/fstab to make the new volume mount to the correct mountpoint by default, and to keep the old volume from mounting at all. Helpful website for understanding /etc/fstab: http://www.tuxfiles.org/linuxhelp/fstab.html

7. Restore any directories/files from archive if necessary

Code:

# cpio -idvmu /dir_to_be_restored/* < /dev/st0

Repeat for each directory to restore.

8. Run fsck to check the volume, and you may even want to reboot.

Code:

# umount /dev/VolGroup00/newLV
# fsck -f /dev/VolGroup00/newLV

9. Once you are confident that the new volume is working properly, you can delete the old volume.

Hope this helps someone else at some point!

Just remember that when you want to reduce the size of a volume, reduce the filesystem size first.
And don't use Webmin to do it!

dmedhora · 11-29-2007, 03:56 AM

Hi, I'm having a similar problem here with my
filesystem. I'm not getting further help with
my problem but since its pretty much the same as
this one I thought I'd mention it -

Basically my problem is I have a partition/filesystem
that I cannot mount. I would like to know how to
find the beginning of the partition so that I can
recreate it starting from there and then hopefully
be able to fsck and mount the filesystem which is
still hopefully on it.
The culprit filesystem is inbetween 2 mountable
filesystems and the set up is something like this
below:-
/dev/sda11 10G This mounts ok
/dev/sda12 50G Doesn't mount, doesn't fsck
/dev/sda13 20G This mounts ok

I've tried *many* things in order to mount sda12
but nothing seems to work. A recent article I read
about how to find the beginning of a partition
talked about a reiserfs instead. So, what is the
signature on an ext3 filesystem that can be used
to search for its beginning, then its simply a
matter of seeking it, and copying 50G from there
to another partition and fscking that..I guess.
If I'm wrong please help me with the right method.

I've tried testdisk which couldn't read the files
on this partition. I haven't tried gpart yet.
It can't even find the superblock in the standard
locations like 8193 or 32768.

A recent fdisk listing shows:-

bash-3.00# fdisk -ls /dev/sda

Disk /dev/sda: 122.9 GB, 122942324736 bytes
16 heads, 63 sectors/track, 238216 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 20318 10240240+ 83 Linux
/dev/sda2 20319 30477 5120136 83 Linux
/dev/sda3 30478 40636 5120136 83 Linux
/dev/sda4 40637 238216 99580320 f W95 Ext'd (LBA)
/dev/sda5 40637 44700 2048224+ 83 Linux
/dev/sda6 44701 54388 4882720+ 83 Linux
/dev/sda7 54389 56326 976720+ 83 Linux
/dev/sda8 56327 57295 488344+ 83 Linux
/dev/sda9 57296 61170 1952968+ 82 Linux swap
/dev/sda10 61171 80546 9765472+ 83 Linux
/dev/sda11 80547 99922 9765472+ 83 Linux
/dev/sda12 99923 196803 48827992+ 83 Linux
/dev/sda13 196804 238216 20872120+ 83 Linux

and the same with fdisk -lu shows:
bash-3.00# fdisk -lu /dev/sda

Disk /dev/sda: 122.9 GB, 122942324736 bytes
16 heads, 63 sectors/track, 238216 cylinders, total 240121728 sectors
Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 63 20480543 10240240+ 83 Linux
/dev/sda2 20480544 30720815 5120136 83 Linux
/dev/sda3 30720816 40961087 5120136 83 Linux
/dev/sda4 40961088 240121727 99580320 f W95 Ext'd (LBA)
/dev/sda5 40961151 45057599 2048224+ 83 Linux
/dev/sda6 45057663 54823103 4882720+ 83 Linux
/dev/sda7 54823167 56776607 976720+ 83 Linux
/dev/sda8 56776671 57753359 488344+ 83 Linux
/dev/sda9 57753423 61659359 1952968+ 82 Linux swap
/dev/sda10 61659423 81190367 9765472+ 83 Linux
/dev/sda11 81190431 100721375 9765472+ 83 Linux
/dev/sda12 100721439 198377423 48827992+ 83 Linux
/dev/sda13 198377487 240121727 20872120+ 83 Linux

bash-3.00# e2fsck /dev/sda12
e2fsck 1.38 (30-Jun-2005)
Couldn't find ext2 superblock, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/sda12

The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device> ( But this gives the same result)

Then if you see the test disk log above, it says to try:
e2fsck -b 32768 -B 4096 /dev/sda12
But I get the same result.

But the main reason why all this is happening is
because i wiped out the first 512 bytes of the
damn disk without knowing the repurcursions.
That beckons another question. Where can I find
a backup copy of the partition table in Fedora core4?
Many thanks!