LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices

Reply
 
Search this Thread
Old 10-31-2010, 05:00 PM   #1
geekmiki
LQ Newbie
 
Registered: Oct 2010
Posts: 10

Rep: Reputation: 0
RAID5: impossible to create raid properly on partitions


Hi,

I've tried to create a raid 5 array for the past few days and it seems impossible to create it properly on the right partitions.
I'm using ubuntu 10.04 server, 3x 2TB Samsung F4EG (with 4k sectors that need to be properly aligned).

Here's how I proceded:
-I create a Linux raid partition on each drive with Gdisk (fdisk supporting GPT), aligned on sector 2048
-Launch mdadm to create array with:
sudo mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdd1
-I wait until the whole array is created
-I format md0 with:
mkfs.ext4 -b 4096 -E stride=16,stripe-width=32 -O extent -m 0 -v /dev/md0


The problem is that I end up with an array on the drives; not on the partitions:
my /proc/mdstat

Code:
ersonalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid5 sdd[2] sdb[1] sda[0]
      3907026944 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
      
unused devices: <none>
and mdadm --detail

Code:
/dev/md0:
        Version : 00.90
  Creation Time : Sat Oct 30 18:11:20 2010
     Raid Level : raid5
     Array Size : 3907026944 (3726.03 GiB 4000.80 GB)
  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Sun Oct 31 16:16:40 2010
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           UUID : 0551e0d3:44e7a9f1:46479b19:e8b0c6ac (local to host miki-server)
         Events : 0.36

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       2       8       48        2      active sync   /dev/sdd
I am going crazy, I've destroyed the raid 3 times (by erasing superblocks) to start over and every time I end up with a raid on the disks and not on the specified partitions.
Any help is welcome!
 
Old 10-31-2010, 07:06 PM   #2
Soadyheid
Member
 
Registered: Aug 2010
Location: Near Edinburgh, Scotland
Posts: 735

Rep: Reputation: 123Reputation: 123
First of all, I'm not all that clued up on software RAID, I'm more used to HP's SmartArray hardware stuff.
Quote:
every time I end up with a raid on the disks and not on the specified partitions.
Maybe I'm wrong but I thought the unit of storage would be the disk and not a partition. RAID means Redundant Array of Inexpensive (or Individual) Disks, not partitions. I've never heard of RAIDed partitions. What would there be to stop you creating three partitions on a single disk and then RAIDing them? Maybe you can, I dunno, but what's the point? Disks in a hardware RAID have a separate system partition which holds the RAID configuration in case the controller has to be replaced,is there a similar sort of thing in a software RAID to assist in rebuilding swapped out disks? Hmm...

Nope, I think I'll leave it to someone smarter than me to help you.

Play Bonny!
 
Old 11-01-2010, 01:02 AM   #3
mesiol
Member
 
Registered: Nov 2008
Location: Lower Saxony, Germany
Distribution: CentOS, RHEL, Solaris 10, AIX, HP-UX
Posts: 731

Rep: Reputation: 137Reputation: 137
Hi,

The array by itself looks okay. Are you using GPT or MBR partition table? I've never seen such a behavior before.

@Soadyheid: Geekmiki is right, in Linux software RAID the RAID-disks are partitions on physical devices.
 
Old 11-01-2010, 09:38 AM   #4
Soadyheid
Member
 
Registered: Aug 2010
Location: Near Edinburgh, Scotland
Posts: 735

Rep: Reputation: 123Reputation: 123
Quote:
@Soadyheid: Geekmiki is right, in Linux software RAID the RAID-disks are partitions on physical devices.
Cheers mesiol, thanks for the clarification.

Play Bonny!
 
Old 11-01-2010, 11:23 AM   #5
geekmiki
LQ Newbie
 
Registered: Oct 2010
Posts: 10

Original Poster
Rep: Reputation: 0
This is driving me crazy!
@mesiol: I'm using a GPT partition table (created with GPT Fdisk)

4th time I create the array from scratch and 4th time this happens... I might be doing something wrong, but I'm following tutorials just as described.
The raid creation itself goes fine and once finished I do have an array from /dev/sd[abd]1. When I reboot and check /proc/mdstat everything has moved to /dev/sd[abd] from the orginal partitions.

Code:
personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid5 sdd[2] sdb[1] sda[0]
      3907026944 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
      
unused devices: <none>
I'm about to give up...

This my last try, the array is recovering on the right partitions:
Code:
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sdd1[3] sdb1[1] sda1[0]
      3907026944 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]
      [=>...................]  recovery =  6.8% (134665600/1953513472) finish=358.6min speed=84518K/sec

unused devices: <none>

Last edited by geekmiki; 11-01-2010 at 11:27 AM.
 
Old 11-01-2010, 11:50 AM   #6
mesiol
Member
 
Registered: Nov 2008
Location: Lower Saxony, Germany
Distribution: CentOS, RHEL, Solaris 10, AIX, HP-UX
Posts: 731

Rep: Reputation: 137Reputation: 137
Hi,

possibly a problem with GPT style layout. Have you tried normal fdisk to create MBR style partitions? Did this change anything?
 
Old 11-01-2010, 11:53 AM   #7
geekmiki
LQ Newbie
 
Registered: Oct 2010
Posts: 10

Original Poster
Rep: Reputation: 0
Actually for the first array creation attempt, I used fdisk and MBR partitions (also starting at sector 2048), but the behaviour was the same. After first reboot the array didn't stay on the partitions.
That's when I decided to try GPT fdisk...
 
Old 11-01-2010, 06:09 PM   #8
Electro
Guru
 
Registered: Jan 2002
Posts: 6,042

Rep: Reputation: Disabled
After you make the partitions, did you use "hdparm -z" for each hard drive before you add those partitions to the array. Every time you boot or reboot and the partitions that you created are not written, you will lose everything although the data could be there.

I think you have issues of the first sector of the new or 4 kilobytes and the old 512 bytes. You may want to zero out those sectors to have a clean drive to start over...again.

You can do a test to see if Linux recognizes the drive is using 4 kilobyte sector by just using one drive. If it does not, you will have to update the kernel and the disk utilities.

I have not setup RAID or software RAID, but what I know is is best to use a separate none software RAID drive to store the boot data. You can rely on the BIOS on what it knows best and then wait until the kernel loads which will give your computer more support for GPT. I suggest make sure the kernel and mdadm utilities are updated with the versions that support GPT.

Just a note, a 2 terabyte hard drive will take a long time to re-construct, so you will have a higher chance of losing data when any drives fails during re-constructing. You have to hope that the drives does not fail during about 15 hour that it takes for your setup to re-construct.
 
Old 11-01-2010, 06:49 PM   #9
geekmiki
LQ Newbie
 
Registered: Oct 2010
Posts: 10

Original Poster
Rep: Reputation: 0
Just tested with one of the drives alone:
-created a GPT partition with gdisk
-mkfs.ext4
-mounted
-copied a file
-added to fstab
-rebooted

Works like a charm...
 
Old 11-02-2010, 01:42 PM   #10
mesiol
Member
 
Registered: Nov 2008
Location: Lower Saxony, Germany
Distribution: CentOS, RHEL, Solaris 10, AIX, HP-UX
Posts: 731

Rep: Reputation: 137Reputation: 137
Hi,

so i am a little at the end. What is about viewing the mailing list of the programmers?
 
Old 11-02-2010, 01:49 PM   #11
geekmiki
LQ Newbie
 
Registered: Oct 2010
Posts: 10

Original Poster
Rep: Reputation: 0
Problem solved by installing the package mdadm 3.1.14 from Debian sid to replace the 2.6 something included in Ubuntu 10.04!

Thanks all for your help!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Software raid5, is it using 2 or 3 partitions?? pingu Linux - Software 2 10-01-2009 02:21 AM
Multi Layer RAID50 fail (Intel SRCS14L RAID5 + 3ware 9550SX-4LP RAID5)+Linux RAID 0 BaronVonChickenPants Linux - Server 4 09-27-2009 04:06 AM
mdadm, images over loopback, raid5, can't create partitions emat Linux - Software 1 05-03-2007 03:46 PM
Create software RAID partitions first, then create filesystem partitions on top of th stefanlasiewski Linux - Software 1 04-28-2004 04:12 PM
Resizing ext3 partitions on RAID5 array greenhornet Linux - General 2 04-09-2002 08:22 AM


All times are GMT -5. The time now is 01:27 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration