LinuxQuestions.org - Can data be recovered? md0 bad superblock?

- Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)

- - Can data be recovered? md0 bad superblock? (https://www.linuxquestions.org/questions/linux-software-2/can-data-be-recovered-md0-bad-superblock-562513/)

Can data be recovered? md0 bad superblock?

Greetings!

Sitting on my bench are two 80 gig drives from a Fedora Core 2.6.10-1.771_FC2 machine. When I try to boot from these drives I get:

Code:

JBD: IO error reading journal superblock

EXT3-fs: error mounting EXT3

Pivotroot: pivot-root(/sysroot,/sysroot/initrd) failed:2

umount /initrd/proc failed:2

kernel panic - not syncing: no init found.  try passing init=option to kernel

When I try to mount the disks on another machine running SME server (CentOS) I get this message:

Code:

[root@qpak /]# mount -t ext3 /dev/md0 /mnt/hda/

mount: wrong fs type, bad option, bad superblock on /dev/md0,

      or too many mounted file systems

The disks are partitioned like this:

Code:

Disk /dev/hda: 80.0 GB, 80026361856 bytes

255 heads, 63 sectors/track, 9729 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes



  Device Boot      Start        End      Blocks  Id  System

/dev/hda1  *          1          13      104391  83  Linux

/dev/hda2              14        9612    77103967+  fd  Linux raid autodetect

[root@qpak log]# fdisk -l /dev/hdb



Disk /dev/hdb: 80.0 GB, 80026361856 bytes

255 heads, 63 sectors/track, 9729 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes



  Device Boot      Start        End      Blocks  Id  System

/dev/hdb1  *          1        9599    77103936  fd  Linux raid autodetect

/dev/hdb2            9600        9729    1044225  82  Linux swap

Other bits that might be helpful:

Code:

[root@qpak /]# cat /proc/mdstat

Personalities : [raid0] [raid1]

md0 : active raid0 hdb1[1] hda2[0]

      154207744 blocks 256k chunks



md1 : active raid1 sda1[0] sdb1[1]

      104320 blocks [2/2] [UU]



md2 : active raid1 sda2[0] sdb2[1]

      156183808 blocks [2/2] [UU]



unused devices: <none>



[root@qpak /]# fsck.ext3 -n /dev/md0

e2fsck 1.35 (28-Feb-2004)

/dev/md0: Attempt to read block from filesystem resulted in short read while reading block 530



/dev/md0: Attempt to read block from filesystem resulted in short read reading journal superblock



fsck.ext3: Attempt to read block from filesystem resulted in short read while checking ext3 journal for /dev/md0



[root@qpak /]# mdadm --detail /dev/md0

/dev/md0:

        Version : 00.90.01

  Creation Time : Sat Dec 11 13:32:39 2004

    Raid Level : raid0

    Array Size : 154207744 (147.06 GiB 157.91 GB)

  Raid Devices : 2

  Total Devices : 2

Preferred Minor : 0

    Persistence : Superblock is persistent



    Update Time : Sat Dec 11 13:32:39 2004

          State : clean

 Active Devices : 2

Working Devices : 2

 Failed Devices : 0

  Spare Devices : 0



    Chunk Size : 256K



    Number  Major  Minor  RaidDevice State

      0      3        2        0      active sync  /dev/hda2

      1      3      65        1      active sync  /dev/hdb1

          UUID : e15094d1:7e51ff3b:49a9cd22:af8d88fe

        Events : 0.1

Since this pair of disks represents all the data for a business that didn't bother to back up their data prior to a power surge they are quite interested in getting it back. The person that set up a RAID 0 for important data and didn't set up a backup procedure won't answer the phone...

Any ideas?

Thanks!

Try using the -sb option for mount or the -b option for fsck and indicate an alternate superblock such as 8193.
You could also try the mount option "errors=remount-ro"

It seems that the md device is OK but the ext3 filesystem on it isn't.

Good Luck!

Thanks for the quick reply!

I'm not too clear on the -sb option for mount. This is what I get:

Code:

[root@qpak mnt]# mount -t ext3 -sb 294912 /dev/md0 /mnt/hda/

mount: invalid option -- b

Usage: mount -V                : print version

      mount -h                : print this help

      mount                    : list mounted filesystems

      mount -l                : idem, including volume labels

So far the informational part. Next the mounting.

The command is `mount [-t fstype] something somewhere'.

Details found in /etc/fstab may be omitted.

      mount -a [-t|-O] ...    : mount all stuff from /etc/fstab

      mount device            : mount device at the known place

      mount directory          : mount known device here

      mount -t type dev dir    : ordinary mount command

Note that one does not really mount a device, one mounts

a filesystem (of the given type) found on the device.

One can also mount an already visible directory tree elsewhere:

      mount --bind olddir newdir

or move a subtree:

      mount --move olddir newdir

A device can be given by name, say /dev/hda1 or /dev/cdrom,

or by label, using  -L label  or by uuid, using  -U uuid .

Other options: [-nfFrsvw] [-o options] [-p passwdfd].

For many more details, say  man 8 mount .

All of the alternate superblocks (so far) give the same result:

Code:

[root@qpak mnt]# e2fsck -b 294912 /dev/md0

e2fsck 1.35 (28-Feb-2004)

/dev/md0: Attempt to read block from filesystem resulted in short read while reading block 530



/dev/md0: Attempt to read block from filesystem resulted in short read reading journal superblock



e2fsck: Attempt to read block from filesystem resulted in short read while checking ext3 journal for /dev/md0

Is this the correct way for errors=remount-ro?

Code:

[root@qpak mnt]# mount -t ext3 -o errors=remount-ro /dev/md0 /mnt/hda/

mount: wrong fs type, bad option, bad superblock on /dev/md0,

      or too many mounted file systems

What does "sudo file -s /dev/md0" say.

Do you get an output from "sudo /sbin/tune2fs /dev/md0". Look at the "blocks per group". That should indicate what the spare superblocks are.

Your mount command was wrong.
mount -t ext3 /dev/md0 /mnt/hda/ -o sb=<sparesuperblock>

try:
mount -t ext3 /dev/md0 /mnt/hda/ -o sb=8193

I just did a bit of a test/demo where I created an image of an ext3 filesystem to play with.

Code:

tune2fs -l test.iso

tune2fs 1.39 (29-May-2006)

Filesystem volume name:  testdir

Last mounted on:          <not available>

Filesystem UUID:          1ed4a26f-86d1-4e6f-89aa-892aa12973fc

Filesystem magic number:  0xEF53

Filesystem revision #:    1 (dynamic)

...

Blocks per group:        8192

Fragments per group:      8192

...

I don't know if the filesystem is on an md device, is the first ext3 superblock on block0? My next spare superblock would be on block 8193, which is 1 + Blocks per group.

For the mount command I was able to mount it with the command:
mount -t ext3 test.iso testdirectory/ -o loop,sb=8193

( I like to try things out myself, if possible as I post a reply)

I seem to remember another command that examined the filesystem and listed the spare superblocks, but I don't remember what it was, or even if I was looking at an ext3 filesystem.

Here are some results:

Code:

[root@qpak mnt]# sudo file -s /dev/md0

/dev/md0: Linux rev 1.0 ext3 filesystem data (needs journal recovery) (errors)



[root@qpak mnt]# sudo /sbin/tune2fs -l /dev/md0

tune2fs 1.35 (28-Feb-2004)

Filesystem volume name:  <none>

Last mounted on:          <not available>

Filesystem UUID:          3113c1cb-49f2-4eba-b28f-78f6158887af

Filesystem magic number:  0xEF53

Filesystem revision #:    1 (dynamic)

Filesystem features:      has_journal ext_attr dir_index filetype needs_recovery sparse_super

Default mount options:    (none)

Filesystem state:        clean with errors

Errors behavior:          Continue

Filesystem OS type:      Linux

Inode count:              19283968

Block count:              38551936

Reserved block count:    1927596

Free blocks:              36340257

Free inodes:              19107843

First block:              0

Block size:              4096

Fragment size:            4096

Blocks per group:        32768

Fragments per group:      32768

Inodes per group:        16384

Inode blocks per group:  512

Filesystem created:      Sat Dec 11 13:32:39 2004

Last mount time:          Sat Jun  2 15:26:22 2007

Last write time:          Sun Jun  3 10:40:00 2007

Mount count:              68

Maximum mount count:      -1

Last checked:            Sat Dec 11 13:32:39 2004

Check interval:          0 (<none>)

Reserved blocks uid:      0 (user root)

Reserved blocks gid:      0 (group root)

First inode:              11

Inode size:              128

Journal inode:            8

Default directory hash:  tea

Directory Hash Seed:      5e19380f-956a-4e05-b0c6-1376734dc1fd

Journal backup:          inode blocks



[root@qpak mnt]# mke2fs -n /dev/md0

mke2fs 1.35 (28-Feb-2004)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

19283968 inodes, 38551936 blocks

1927596 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=41943040

1177 block groups

32768 blocks per group, 32768 fragments per group

16384 inodes per group

Superblock backups stored on blocks:

        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,

        4096000, 7962624, 11239424, 20480000, 23887872





[root@qpak mnt]# mount -t ext3 /dev/md0 /mnt/hda/ -o sb=23887872

mount: wrong fs type, bad option, bad superblock on /dev/md0,

      or too many mounted file systems

[root@qpak mnt]#

It appears that the RAID 0 is approx 157Gb of which 867Mb are in use. Too bad they didn't set up a RAID 1, they didn't need the space.

Thanks for your help!

Try "/sbin/dumpe2fs /dev/md0"

Look at the parts that look like this:

Code:

Group 1: (Blocks 8193-9999)

  Backup superblock at 8193, Group descriptors at 8194-8194

  Reserved GDT blocks at 8195-8233

  Block bitmap at 8234 (+41), Inode bitmap at 8235 (+42)

  Inode table at 8236-8392 (+43)

  1607 free blocks, 1256 free inodes, 0 directories

  Free blocks: 8393-9999

  Free inodes: 1257-2512

When I looked on a large ext3 filesystem on my ieee1394 external drive, I noticed that only block groups 1-9 had backup superblocks.

Code:

Group 9: (Blocks 294912-327679)

  Backup superblock at 294912, Group descriptors at 294913-294963

  Reserved GDT blocks at 294964-295936

  Block bitmap at 295937 (+1025), Inode bitmap at 295938 (+1026)

  Inode table at 295939-296450 (+1027)

  0 free blocks, 16384 free inodes, 0 directories

  Free blocks:

  Free inodes: 147457-163840

The filesystem size according to tune2fs is 147GB

Code:

echo  'scale=3; 38551936*4096/1024/1024/1024' | bc

147.063

OK,

The locations of backup superblocks are the same as the locations shown in "mke2fs -n /dev/md0".

The output of "/sbin/dumpe2fs /dev/md0" is too big to post here. I sent the output to a file and here are some parts.

It starts like this:

Code:

Filesystem volume name:  <none>

Last mounted on:          <not available>

Filesystem UUID:          3113c1cb-49f2-4eba-b28f-78f6158887af

Filesystem magic number:  0xEF53

Filesystem revision #:    1 (dynamic)

Filesystem features:      has_journal ext_attr dir_index filetype needs_recovery sparse_super

Default mount options:    (none)

Filesystem state:        clean with errors

Errors behavior:          Continue

Filesystem OS type:      Linux

Inode count:              19283968

Block count:              38551936

Reserved block count:    1927596

Free blocks:              36340257

Free inodes:              19107843

First block:              0

Block size:              4096

Fragment size:            4096

Blocks per group:        32768

Fragments per group:      32768

Inodes per group:        16384

Inode blocks per group:  512

Filesystem created:      Sat Dec 11 13:32:39 2004

Last mount time:          Sat Jun  2 15:26:22 2007

Last write time:          Sun Jun  3 10:40:00 2007

Mount count:              68

Maximum mount count:      -1

Last checked:            Sat Dec 11 13:32:39 2004

Check interval:          0 (<none>)

Reserved blocks uid:      0 (user root)

Reserved blocks gid:      0 (group root)

First inode:              11

Inode size:              128

Journal inode:            8

Default directory hash:  tea

Directory Hash Seed:      5e19380f-956a-4e05-b0c6-1376734dc1fd

Journal backup:          inode blocks





Group 0: (Blocks 0-32767)

  Primary superblock at 0, Group descriptors at 1-10

  Block bitmap at 11 (+11), Inode bitmap at 12 (+12)

  Inode table at 13-524 (+13)

  23058 free blocks, 16371 free inodes, 2 directories

Group 1: (Blocks 32768-65535)

  Backup superblock at 32768, Group descriptors at 32769-32778

  Block bitmap at 32779 (+11), Inode bitmap at 32780 (+12)

  Inode table at 32781-33292 (+13)

In the middle of the file there are Group listings that indicate the presence of directories.

Ends like this:

Code:

Group 1176: (Blocks 38535168-38551935)

  Block bitmap at 38535168 (+0), Inode bitmap at 38535169 (+1)

  Inode table at 38535170-38535681 (+2)

  15691 free blocks, 16307 free inodes, 0 directories



/sbin/dumpe2fs: /dev/md0: error reading bitmaps: Can't read an block bitmap

Thanks for your help!

I was afraid that the assumptions made by mk2fs -n would be wrong, and supplied the dumpe2fs info to provide the actual backup superblocks instead of assumed ones.

Try each of them in the options to mount or fsck.ext3.

This link may help you recover data if you can't get the filesystem repaired:
http://www.geocities.com/CapeCanaver.../7731/lde.html

Also check for Live distro's with tools to repair disks or recover files.

You also might consider making an image copy of the drive before trying to modify it.