LinuxQuestions.org - mdadm.conf and RAID 1 on Ubuntu

- Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)

- - mdadm.conf and RAID 1 on Ubuntu (https://www.linuxquestions.org/questions/linux-software-2/mdadm-conf-and-raid-1-on-ubuntu-457956/)

mdadm.conf and RAID 1 on Ubuntu

I am a newbie. And have absolutely done a terrible newbie thing. When installing Ubuntu on our servers I set up a RAID 1 on all partitions on two SATA disk drives. Logically, I made all sda partitions devices and all sdb partitions spares. You can image the horror when we disconnected the sda drive to test the sdb drive on boot.

For those of you who don't quite understand my configuration here is the result of the following commands:

Code:

$ cat /proc/mdstat

Personalities : [raid1]

md3 : active raid1 sdb4[1](S) sda4[0]

      136664832 blocks [1/1] [U]



md2 : active raid1 sdb3[1](S) sda3[0]

      9767424 blocks [1/1] [U]



md1 : active raid1 sda2[0] sdb2[1](S)

      9767424 blocks [1/1] [U]



md0 : active raid1 sdb1[1](S) sda1[0]

      48064 blocks [1/1] [U]



unused devices: <none>

Code:

$ sudo mdadm --query --detail /dev/md0

Password:

/dev/md0:

        Version : 00.90.03

  Creation Time : Thu May 11 06:34:52 2006

    Raid Level : raid1

    Array Size : 48064 (46.95 MiB 49.22 MB)

    Device Size : 48064 (46.95 MiB 49.22 MB)

  Raid Devices : 1

  Total Devices : 2

Preferred Minor : 0

    Persistence : Superblock is persistent



    Update Time : Sat Jun 24 13:39:18 2006

          State : clean

 Active Devices : 1

Working Devices : 2

 Failed Devices : 0

  Spare Devices : 1



          UUID : efbff26c:f7e:)f3a3:cd:)cb948:61506d6:) 

        Events : 0.1016



    Number  Major  Minor  RaidDevice State

      0      8        1        0      active sync  /dev/sda1



      1      8      17        -      spare  /dev/sdb1

md1, md2, md3 are similar to md0. So what I want to do is promote my spare to a working device. I want to change the /etc/mdadm/mdadm.conf file for all Arrays and make the following changes:

Code:

devices=1 spare=1

            to

devices=2 spare=0

Of course there is more to the file I just homed in on the blood and guts of the dilemma. Before actually trying this is there something I have to do in order to assure that I don't completely break the servers? Also I haven't adjusted grub but I think I know how to do that to boot on backup if necessary. See you all at the Portland OSCON on the 24th of July. Thanks in advanced.:scratch:

It takes a lot of balls to admit a mistake like that, so I think you deserve some help.

First off, the purpose of mdadm.conf is to tell the OS what to expect. It has nothing to do with how software raids are configured.

But not to despair, the fix is easy! What you need to do is to --grow the raid1s.

Look at the mdadm man page for more info about using -G (--grow).

I set up a --grow example (see below) for you to look at. Modify the --grow commands to match your situation. Be sure to let the rebuild processes finish before you reboot.

When all of the raids have been rebuilt and before you reboot, update mdadm.conf as is shown at the end of the example (Very, Very, Very Important!!!).

Code:

# mdadm -C /dev/md14 -l1 -n1 -x1 /dev/hd[eg]14 --force

mdadm: array /dev/md14 started.



# cat /proc/mdstat

Personalities : [raid1]

md14 : active raid1 hdg14[1](S) hde14[0]

      15735552 blocks [1/1] [U]



unused devices: <none>



# mdadm -D /dev/md14

/dev/md14:

        Version : 00.90.03

  Creation Time : Sat Jun 24 21:39:10 2006

    Raid Level : raid1

    Array Size : 15735552 (15.01 GiB 16.11 GB)

    Device Size : 15735552 (15.01 GiB 16.11 GB)

  Raid Devices : 1

  Total Devices : 2

Preferred Minor : 14

    Persistence : Superblock is persistent



    Update Time : Sat Jun 24 21:39:10 2006

          State : clean

 Active Devices : 1

Working Devices : 2

 Failed Devices : 0

  Spare Devices : 1



          UUID : 6a1f3a58:018cc3c9:7c3b2649:a2422600

        Events : 0.1



    Number  Major  Minor  RaidDevice State

      0      33      14        0      active sync  /dev/hde14



      1      34      14        -      spare  /dev/hdg14



# mdadm -G /dev/md14 -n2



# cat /proc/mdstat

Personalities : [raid1]

md14 : active raid1 hdg14[2] hde14[0]

      15735552 blocks [2/1] [U_]

      [=>...................]  recovery =  6.1% (961792/15735552) finish=7.1min speed=34349K/sec



unused devices: <none>



# # And the clouds parted and the birds began to sing!



# mdadm -D /dev/md14

/dev/md14:

        Version : 00.90.03

  Creation Time : Sat Jun 24 21:39:10 2006

    Raid Level : raid1

    Array Size : 15735552 (15.01 GiB 16.11 GB)

    Device Size : 15735552 (15.01 GiB 16.11 GB)

  Raid Devices : 2

  Total Devices : 2

Preferred Minor : 14

    Persistence : Superblock is persistent



    Update Time : Sat Jun 24 21:59:15 2006

          State : clean

 Active Devices : 2

Working Devices : 2

 Failed Devices : 0

  Spare Devices : 0



          UUID : 6a1f3a58:018cc3c9:7c3b2649:a2422600

        Events : 0.5



    Number  Major  Minor  RaidDevice State

      0      33      14        0      active sync  /dev/hde14

      1      34      14        1      active sync  /dev/hdg14





# # Update Debian-style mdadm.conf



# cd /etc/mdadm

# cp mdadm.conf mdadm.conf.`date +%y%m%d`

# echo "DEVICE partitions" > mdadm.conf

# mdadm -D -s >> mdadm.conf

# cd /root

Good Luck!

Thank you so much for the advice, although on the test server the procedure worked fine, but on the production server when I tried to install grub so that the server could boot from either hardrive on startup for fault-tollerance, I get the following error:

Code:

# grub-install /dev/sda

# grub



grub> device (hd0) /dev/sdb



grub> root (hd0,0)

 Filesystem type is fat, partition type 0x6



grub> setup (hd0)

 Checking if "/boot/grub/stage1" exists... no

 Checking if "/grub/stage1" exists... no



Error 15: File not found



grub> quit

How can Ubuntu run on fat? File /boot/grub/stage1 exists but grub cannot find it. With no success I even tried:

Code:

grub> find /boot/grub/stage1

Something is not right, and I am afraid to reboot the server since we need it functioning at 8:00 am PST today. I thank you in advance.

Stayed up all night trying to configure grub to boot from either disk on start up in case of disk failure :confused:

Sounds like you’re running a Dell with the FAT16-based Dell Utility as the first partition.

Try searching for grub stage1 slightly differently:

Code:

grub> find /grub/stage1

which is used when you have a separate /boot partition.

Sorry that did not work either...

Yes you were right, it is a Dell

Quote:

Sounds like you’re running a Dell with the FAT16-based Dell Utility as the first partition.

Try searching for grub stage1 slightly differently:

Code:
grub> find /grub/stage1

which is used when you have a separate /boot partition.

but neither one of those commands find my stage1 What do I do now?

Code:

~$ ls -l  /boot/grub/stage*

-rw-r--r-- 1 root root    512 2006-06-30 19:22 /boot/grub/stage1

-rw-r--r-- 1 root root 105428 2006-06-30 19:22 /boot/grub/stage2

Doesn't matter what I do..

Code:

grub> find /boot/grub/stage1

Code:

grub> find /grub/stage1

Both come back not found. Thanks in advanced... :scratch:

The Raid 1 Mirror seems to be working

So I disconnected sda primary hardrive and tried to boot from the second hardrive sdb and it won't boot. A message comes up as

Quote:

No loader.

I booted with knoppix to make sure that the RAID 1 replication was working and sdb seems to have all the files duplicated for all partitions as sda. Even the boot partition is duplicated. I have a feeling grub may not be configured correctly, which is the problem I had originally. I though maybe if I connected sdb to the sda connection it would boot, but it still comes up as "no loader". If sdb is a mirror of sda why won't it boot on its own? I cannot afford to lose all the data on sda or to destroy the boot loader, grub, on the system. And I can't seem to get past the find command previously noted to place sdb on the grub boot loader to boot. Thanks in advance, as any help would be greatly appreciated. :Pengy:

“No Loader” sounds like a system BIOS issue.

Make sure the system BIOS boot options for the hard drives are correct.

well we tested that...

We disconnected the good drive and the one with the bad loader wouldn't boot weather it was plugged into SATA-0 or SATA-1 (on the motherboard) as if it was a good drive. It has a mirror images of all four partitions of the "good" drive (this was accomplished after your great advice to Grow the RAID Devices) By default, shouldn't it boot as well? Or as you are saying the BIOS recognizes drives other than by where they are connected? Because the BIOS is configured to boot from CDROM first, and then to any available hard-drive. We don't think that is the problem. What do you reccommend? We appreciate all you help, and we thank you for all your good advice.
:confused:

grub is not connecting to filesystem...

:eek: It appears that grub is not communicating with the filesystem. When I reinstall grub and add device /dev/sdb it will not allow me to

Code:

grub> setup (hd0)

It says that file stage1 could not be found and doesnot exist. So the find command does not connect to filesystem, although on the bash shell prompt the find command works fine and can find /boot/grub/stage1 everytime.

Our system boots fine with sda, although not with sdb. We believe that during duplication something may have gone wrong.

So we would like to remove,reformat sdb and reinstall it to duplicate all over again then maybe grub will be able to communicate with the file system on disk sdb. Do we need to partition the newly formated drive or not? Does Ubuntu software RAID1 take care of it?

We are very grateful for you help, and apprecitate all the time and effort you have given us. Hope to hear from you soon. :scratch:

Starting back at the beginning sounds like a plan.

There should be no problem in reusing the existing disk partition structure, but as you already know, be sure to have the Debian installer reformat the filesystems so that you start with a clean slate.

Which Debian Installer would you recommend?

Won't a debian installer install the OS onto the drive after formatting it? Can't I just fdisk the disk on another machine and format it that way? Although, should I be formatting LinuxRAID or XFS3?

At the step where you select the mount points, you need to specify how you want each partition/logical volume formatted (e.g., leave it alone, ext3, reiserfs, etc.).