LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 04-17-2007, 11:24 AM   #16
rtspitz
Member
 
Registered: Jan 2005
Location: germany
Distribution: suse, opensuse, debian, others for testing
Posts: 307

Rep: Reputation: 32

Quote:
Originally Posted by hazmatt20
That German thread is exactly what mine is doing. If you don't mind translating the solution, I'd be grateful. I tried
translation of the last part with the "solution":

Code:
at the time I created the raid I must have made a mistake, which showed up right now. 
apparently I had created persistent superblocks on the devices (/dev/sd[a-e]) as well as
on the partitions (/dev/sd[a-e]1). 
after zero-ing the superblocks with "mdadm --zero-superblock /dev/sd[-e]" and rebooting,
the partitions showed up in /proc/partitions again and the raid was operational and could
be mounted without any errors.

this night was not subject to entertainment tax. (:
 
Old 04-17-2007, 02:01 PM   #17
hazmatt20
Member
 
Registered: Jan 2006
Distribution: FC5, Ubuntu
Posts: 126

Original Poster
Rep: Reputation: 15
Well, I decided to give mdadm --create a shot.

Code:
mdadm --create /dev/md0 --verbose --level=5 --raid-devices=6 /dev/sd[d-i]1
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 64K
mdadm: /dev/sdd1 appears to contain an ext2fs file system
    size=1953543680K  mtime=Sun Apr 15 18:40:03 2007
mdadm: /dev/sdd1 appears to be part of a raid array:
    level=raid5 devices=6 ctime=Sun Mar 11 00:22:58 2007
mdadm: /dev/sde1 appears to be part of a raid array:
    level=raid5 devices=6 ctime=Sun Mar 11 00:22:58 2007
mdadm: /dev/sdf1 appears to be part of a raid array:
    level=raid5 devices=6 ctime=Sun Mar 11 00:22:58 2007
mdadm: /dev/sdg1 appears to contain an ext2fs file system
    size=1953543680K  mtime=Sun Apr 15 18:40:02 2007
mdadm: /dev/sdg1 appears to be part of a raid array:
    level=raid5 devices=6 ctime=Sun Mar 11 00:22:58 2007
mdadm: /dev/sdh1 appears to be part of a raid array:
    level=raid5 devices=6 ctime=Sun Mar 11 00:22:58 2007
mdadm: /dev/sdi1 appears to be part of a raid array:
    level=raid5 devices=6 ctime=Sun Mar 11 00:22:58 2007
mdadm: size set to 390708736K
Continue creating array? y
mdadm: array /dev/md0 started.

#cat /proc/mdstat
Personalities : [raid5] [raid4]
md0 : active raid5 sdi1[6] sdh1[4] sdg1[3] sdf1[2] sde1[1] sdd1[0]
      1953543680 blocks level 5, 64k chunk, algorithm 2 [6/5] [UUUUU_]
      [>....................]  recovery =  0.1% (419712/390708736) finish=340.8min speed=19077K/sec

# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Tue Apr 17 14:57:02 2007
     Raid Level : raid5
     Array Size : 1953543680 (1863.04 GiB 2000.43 GB)
    Device Size : 390708736 (372.61 GiB 400.09 GB)
   Raid Devices : 6
  Total Devices : 6
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Tue Apr 17 14:57:41 2007
          State : clean, degraded, recovering
 Active Devices : 5
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 64K

 Rebuild Status : 1% complete

           UUID : ce16308c:c13226e7:126d5cca:b4ac2ebe
         Events : 0.3

    Number   Major   Minor   RaidDevice State
       0       8       49        0      active sync   /dev/sdd1
       1       8       65        1      active sync   /dev/sde1
       2       8       81        2      active sync   /dev/sdf1
       3       8       97        3      active sync   /dev/sdg1
       4       8      113        4      active sync   /dev/sdh1
       6       8      129        5      spare rebuilding   /dev/sdi1
So, I'll let it go for a few hours and check back.
 
Old 04-17-2007, 06:31 PM   #18
hazmatt20
Member
 
Registered: Jan 2006
Distribution: FC5, Ubuntu
Posts: 126

Original Poster
Rep: Reputation: 15
Alright, so it finished resyncing. Now we get

Code:
# mount /dev/md0 md0
mount: wrong fs type, bad option, bad superblock on /dev/md0,
       missing codepage or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

# fsck /dev/md0
fsck 1.39 (29-May-2006)
e2fsck 1.39 (29-May-2006)
Group descriptors look bad... trying backup blocks...
fsck.ext3: Bad magic number in super-block while trying to open /dev/md0

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
Is there anything else we can try, or is it game over?
 
Old 04-17-2007, 06:37 PM   #19
Quakeboy02
Senior Member
 
Registered: Nov 2006
Distribution: Debian Squeeze 2.6.32.9 SMP AMD64
Posts: 3,238

Rep: Reputation: 121Reputation: 121
You've kind of lost me here. I was under the impression that the create option created a new array and threw away anything that previously existed. As far as I was understand, the data was gone when you ran create.
 
Old 04-17-2007, 07:00 PM   #20
hazmatt20
Member
 
Registered: Jan 2006
Distribution: FC5, Ubuntu
Posts: 126

Original Poster
Rep: Reputation: 15
I was going by this article posted earlier. http://kev.coolcavemen.com/2007/03/h...d-superblocks/
 
Old 04-17-2007, 07:26 PM   #21
rtspitz
Member
 
Registered: Jan 2005
Location: germany
Distribution: suse, opensuse, debian, others for testing
Posts: 307

Rep: Reputation: 32
there is a utility called testdisk http://www.cgsecurity.org/wiki/TestDisk which can scan devices with ext2/ext3 for backup superblocks and help recover them.

some hints:

http://www.cgsecurity.org/wiki/Advan...kup_SuperBlock


you could run:

testdisk /dev/md0

then: [PROCEED], [NONE], [Advanced], [Superblock]

if this works you should get some output like this:

superblock 0, blocksize=1024
superblock 8193, blocksize=1024
...
...

with that you can tell fsck.ext3 (or equivalent on your system) to use a backup superblock like e.g.:

/sbin/fsck.ext3 -b 8193 -B 1024 /dev/md0


if that doesn't work I'm at my wits' end.

Last edited by rtspitz; 04-17-2007 at 07:42 PM.
 
Old 04-17-2007, 10:02 PM   #22
hazmatt20
Member
 
Registered: Jan 2006
Distribution: FC5, Ubuntu
Posts: 126

Original Poster
Rep: Reputation: 15
Well, testdisk didn't show any partitions under advanced, so I'm running analyse. It's going to take a good while, but I'm going to start making plans to start reloading data. I'll post an update when it finishes.
 
Old 04-18-2007, 06:11 PM   #23
hazmatt20
Member
 
Registered: Jan 2006
Distribution: FC5, Ubuntu
Posts: 126

Original Poster
Rep: Reputation: 15
Alright, well, it analyse didn't detect stuff correctly and just gave a bunch of garbage, so I'm pretty positive it's gone. So many DVDs to reload! Oh, well. Thanks for your help.

One last thing, what precautions should I take in the future to increase my chances of recovery? I know now to run dist-upgrade install of installing from disk, but other than that and backing up my mdadm.conf, what should I do?
 
Old 04-18-2007, 06:34 PM   #24
Quakeboy02
Senior Member
 
Registered: Nov 2006
Distribution: Debian Squeeze 2.6.32.9 SMP AMD64
Posts: 3,238

Rep: Reputation: 121Reputation: 121
I've been thinking about this today, and I wonder if the problem could have been avoided if you hadn't had your drives connected when you installed mdadm. I mentioned that I installed mdadm once and it created a bunch of junk on the drives I had connected. It may or may not be a problem, but it's something to think about if you have to reinstall for any reason. You might also think about creating a backup of your non-data files. There are a number of good backup systems out there. I just used tar along with a trivial script I wrote. I actually did a restore (from Knoppix) of the boot/non-data image I keep on my data disks recently, and it worked just fine.
 
Old 04-19-2007, 12:52 AM   #25
hazmatt20
Member
 
Registered: Jan 2006
Distribution: FC5, Ubuntu
Posts: 126

Original Poster
Rep: Reputation: 15
Well, I've almost got everything working, but I've got a few snags. Two parts.

First, I want the 6 400GB drives to start as md0 and the 3 500GB drives to start as md1. When I reboot, md0 starts with 2 of the 3 500GB drives and resyncs with the third while md1 starts with 4 of the 6 400GB drives. mdadm.conf is currently

Code:
# cat mdadm.conf
DEVICE partitions
MAILADDR root
ARRAY /dev/md0 level=raid5 num-devices=6 UUID=d295489e:1146f6bf:10e91e6c:42385ae5
ARRAY /dev/md1 level=raid5 num-devices=3 UUID=27088f9d:f6aea8a8:60e614e7:ea4536bf
Secondly, I'm setting up the two arrays on an LVM. I've already done this much.

Code:
pvcreate /dev/md0
pvcreate /dev/md1
vgcreate RAID_GROUP /dev/md0 /dev/md1
modprobe dm-mod
lvcreate -L2.72T -nmedia RAID_GROUP
mkfs.ext3 /dev/RAID_GROUP/media
After that, I could mount /dev/RAID_GROUP/media normally. After it reboots and I get the raid arrays back up, I activate it with

Code:
vgchange -a y RAID_GROUP
If I want it to activate on startup, should I just add that line to rc.local (assuming the raid arrays come up first), or is there a better way to do it before local file systems are mounted?

Last edited by hazmatt20; 04-19-2007 at 06:47 AM.
 
Old 04-19-2007, 08:41 AM   #26
hazmatt20
Member
 
Registered: Jan 2006
Distribution: FC5, Ubuntu
Posts: 126

Original Poster
Rep: Reputation: 15
Well, now on reboot, md0 comes up correctly but as the 3 500GB drives, so I'll just leave that the way it is. md1, however, only comes up with 4 drives on startup. Once I get a console,

Code:
mdadm -A /dev/md1 /dev/sd[d-i]
works, so I could run it under rc.local, but is there a cleaner way?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
cannot rebuild RAID5 array after two disks went offline waimea Linux - General 2 03-22-2007 03:40 AM
RAID 1 Array recovery ? Andrew_OC Linux - Server 6 01-30-2007 02:49 PM
Secure Deletion with RAID5 array neilschelly Linux - Security 2 12-05-2004 06:25 PM
problem in replacing hardisk raid5 array slack66 Linux - Newbie 0 08-13-2003 04:13 AM
Resizing ext3 partitions on RAID5 array greenhornet Linux - General 2 04-09-2002 08:22 AM


All times are GMT -5. The time now is 02:59 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration