LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-26-2015, 01:29 AM   #1
millenium
LQ Newbie
 
Registered: Jun 2015
Posts: 3

Rep: Reputation: 0
RAID 5 won't mount


I have a RAID5 array with 3 WD RED disks (sdb, sdc sdd), where one drive gave up. I replaced the drive and rebuild. Now the array won't mount.

My mdadm conf:
Code:
# mdadm.conf#
# Please refer to mdadm.conf(5) for information about this file.
#


# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE partitions containers


# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes


# automatically tag new arrays as belonging to the local system
HOMEHOST <system>


# instruct the monitoring daemon where to send mail alerts
MAILADDR root


# definitions of existing MD arrays
ARRAY /dev/md0 UUID=76d1edff:a1188431:0cf33d4d:3ef09af2


# This file was auto-generated on Fri, 24 Apr 2015 06:49:33 +0200
# by mkconf $Id$
'cat /proc/mdstat' gives:

Code:
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]md0 : active raid5 sdd[2] sdc[0] sdb[3]
      5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
'fsck /dev/md0' gives

Code:
fsck from util-linux 2.20.1e2fsck 1.42.9 (4-Feb-2014)
fsck.ext4: Filesystem revision too high while trying to open /dev/md0
The filesystem revision is apparently too high for this version of e2fsck.
(Or the filesystem superblock is corrupt)


The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>
'mdadm -E /dev/sd[bcd]' gives

Code:
/dev/sdb:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 76d1edff:a1188431:0cf33d4d:3ef09af2
           Name : movano:0
  Creation Time : Mon Apr  6 00:07:18 2015
     Raid Level : raid5
   Raid Devices : 3


 Avail Dev Size : 5860271024 (2794.40 GiB 3000.46 GB)
     Array Size : 5860270080 (5588.79 GiB 6000.92 GB)
  Used Dev Size : 5860270080 (2794.39 GiB 3000.46 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 85875e25:589122cb:689b974e:1cf4b988


    Update Time : Fri Apr 24 02:56:16 2015
       Checksum : d8b33362 - correct
         Events : 232


         Layout : left-symmetric
     Chunk Size : 512K


   Device Role : Active device 1
   Array State : AAA ('A' == active, '.' == missing)
millenium@ubuntu:~$ sudo mdadm -E /dev/sd[bcd]
/dev/sdb:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 76d1edff:a1188431:0cf33d4d:3ef09af2
           Name : movano:0
  Creation Time : Mon Apr  6 00:07:18 2015
     Raid Level : raid5
   Raid Devices : 3


 Avail Dev Size : 5860271024 (2794.40 GiB 3000.46 GB)
     Array Size : 5860270080 (5588.79 GiB 6000.92 GB)
  Used Dev Size : 5860270080 (2794.39 GiB 3000.46 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 85875e25:589122cb:689b974e:1cf4b988


    Update Time : Fri Apr 24 02:56:16 2015
       Checksum : d8b33362 - correct
         Events : 232


         Layout : left-symmetric
     Chunk Size : 512K


   Device Role : Active device 1
   Array State : AAA ('A' == active, '.' == missing)
/dev/sdc:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 76d1edff:a1188431:0cf33d4d:3ef09af2
           Name : movano:0
  Creation Time : Mon Apr  6 00:07:18 2015
     Raid Level : raid5
   Raid Devices : 3


 Avail Dev Size : 5860271024 (2794.40 GiB 3000.46 GB)
     Array Size : 5860270080 (5588.79 GiB 6000.92 GB)
  Used Dev Size : 5860270080 (2794.39 GiB 3000.46 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 9732beb4:34a4d203:a8c351d1:788be3c0


    Update Time : Fri Apr 24 02:56:16 2015
       Checksum : 5ba6b0ea - correct
         Events : 232


         Layout : left-symmetric
     Chunk Size : 512K


   Device Role : Active device 0
   Array State : AAA ('A' == active, '.' == missing)
/dev/sdd:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 76d1edff:a1188431:0cf33d4d:3ef09af2
           Name : movano:0
  Creation Time : Mon Apr  6 00:07:18 2015
     Raid Level : raid5
   Raid Devices : 3


 Avail Dev Size : 5860271024 (2794.40 GiB 3000.46 GB)
     Array Size : 5860270080 (5588.79 GiB 6000.92 GB)
  Used Dev Size : 5860270080 (2794.39 GiB 3000.46 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : b2db48cc:b885a2b6:d1085a95:ecbe42a4


    Update Time : Fri Apr 24 02:56:16 2015
       Checksum : cd68b428 - correct
         Events : 232


         Layout : left-symmetric
     Chunk Size : 512K


   Device Role : Active device 2
   Array State : AAA ('A' == active, '.' == missing)
When I try to mount the array as EXT4 (originally EXT4 before dead disk) I get this:

Code:
mount: wrong fs type, bad option, bad superblock on /dev/md0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
'dmesg | tail' gives

Code:
[   85.150872] EXT4-fs (md0): Couldn't mount because of unsupported optional features (1fd00001)
 
Old 06-26-2015, 11:35 PM   #2
S.Haran
LQ Newbie
 
Registered: Jun 2014
Location: Boston USA
Posts: 15

Rep: Reputation: Disabled
You might want to try Testdisk to see if it finds a filesystem. If not try Photorec which will scan for your data based on file extension.

Further action, if needed, will depend on the results you get.
 
Old 06-27-2015, 01:58 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 15,805

Rep: Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168
And did you try the fsck with the alternate superblock(s) as the message told you ?.
 
Old 06-27-2015, 07:02 AM   #4
wpeckham
Senior Member
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, Fedora, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, Vsido, tinycore, Q4OS
Posts: 2,294

Rep: Reputation: 929Reputation: 929Reputation: 929Reputation: 929Reputation: 929Reputation: 929Reputation: 929Reputation: 929
Personal take

I would work on that for about an hour, then just rebuild the array blank, and restore from backup.

You DO have a backup, yes?

Actually, being somewhat adventuresome, I might rebuild using btrfs with some of the raid features turned on.

Last edited by wpeckham; 06-27-2015 at 07:07 AM.
 
Old 06-28-2015, 02:39 AM   #5
millenium
LQ Newbie
 
Registered: Jun 2015
Posts: 3

Original Poster
Rep: Reputation: 0
I will try #2 and #3

Unfortunately my backup system didn't include the RAID array ;-( Lesson learned, the hard way.
 
Old 06-28-2015, 05:32 AM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 15,805

Rep: Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168
Nor should it - the array is at the (emulated) device level. You seem to have reconstructed that ok. For the superblock problem, if fsck won't fix it, do a mkfs and restore your data. Tools like photorec are for people that don't have backups - it scans the disk looking for tell-tale signs of files. Sometimes works, sometimes not. And what is the level of confidence ?.

As for @wpeckham comments on btrfs RAID, no better in some respects - I use it (RAID5) for my important data, but you still need backups. Differential snaps are a boon, but for RAID[56] device replacement you need to be reasonably current on kernel and btrfs-progs (say Feb 2015). Still the best solution IMHO.

Last edited by syg00; 06-28-2015 at 05:34 AM. Reason: Lost some text ...
 
Old 06-28-2015, 08:07 AM   #7
wpeckham
Senior Member
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, Fedora, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, Vsido, tinycore, Q4OS
Posts: 2,294

Rep: Reputation: 929Reputation: 929Reputation: 929Reputation: 929Reputation: 929Reputation: 929Reputation: 929Reputation: 929
Backups

I have been a professional SYSADMIN for over 20 years, a NETADMIN for half that.
If I have learned one thing, it is that you can be good or bad at almost any part of this business and succeed ONLY if you are good at backup and recovery.
We wear a dozen hats, and it is good to be excellent at many or all of our jobs, but this is the ONE thing that is survival: for the businesses we support and for our career.

If you are not a professional, it is less of a priority, but still a good lesson. Every newbie must suffer through learning this at some point: no system/hardware lasts forever.

My first rules of backups (not my invention, any experienced sysadmin might list the same):
1. Don't back up everything, only back up what you do not want to lose.
2. No storage solution takes the place of backups. Notice that is plural: if one failure can kill it, one is not secure.
3. It cannot be called a good backup, until restore is tested.

I hope that you can recover your data, and wish you the best of luck!

Last edited by wpeckham; 06-28-2015 at 08:08 AM.
 
Old 06-28-2015, 10:51 AM   #8
S.Haran
LQ Newbie
 
Registered: Jun 2014
Location: Boston USA
Posts: 15

Rep: Reputation: Disabled
@syg00, To clarify
Quote:
if fsck won't fix it, do a mkfs and restore your data
I expect you mean restore from backup as running mkfs will in no way restore data by itself. As stated the OP has no backup.

As for Photorec one reason to run it here is as a test to validate the RAID is assembled correctly. If for example Photorec recovers a 2MB .jpg file that renders fine then chances are the RAID is in good shape.

For the best chance of recovery the rule is to avoid any operation that writes to the RAID member drives, including fsck. If this is high value data for maximum safety it is best to take drive images and perform recovery work on the images.
 
Old 06-28-2015, 07:46 PM   #9
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 15,805

Rep: Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168Reputation: 2168
Quote:
Originally Posted by S.Haran View Post
As stated the OP has no backup.
Hmmm- maybe I misunderstood. I read that as "I have backup (of data), but not of the array metadata."
My mistake.
Quote:
for maximum safety it is best to take drive images and perform recovery work on the images.
Agreed.
 
Old 06-28-2015, 11:35 PM   #10
millenium
LQ Newbie
 
Registered: Jun 2015
Posts: 3

Original Poster
Rep: Reputation: 0
Thanks for all the answers. I will get a hold of some drives and make drive images and try the suggestions.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Why can't I mount this md0 raid? (mdadm and software raid) cruiserparts Linux - Software 35 01-05-2013 03:35 PM
[SOLVED] Problem with software raid! inactive array, won't mount -Thomas- Linux - Software 9 05-29-2012 04:19 AM
mount existing ntfs SATA RAID 0 on RHEL4 VIA fake RAID tmoble Linux - Hardware 10 11-13-2009 07:49 PM
RAID drives won't mount at boot time dsp444 Slackware 5 01-23-2005 12:43 AM
NFTS sata raid drives - won't mount jamangold Linux - Newbie 0 07-25-2004 06:57 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 03:11 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration