LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 06-24-2006, 04:59 PM   #1
jcorrea920
LQ Newbie
 
Registered: Jun 2006
Location: Oregon, USA
Distribution: Ubuntu / Fedora
Posts: 14

Rep: Reputation: 0
mdadm.conf and RAID 1 on Ubuntu


I am a newbie. And have absolutely done a terrible newbie thing. When installing Ubuntu on our servers I set up a RAID 1 on all partitions on two SATA disk drives. Logically, I made all sda partitions devices and all sdb partitions spares. You can image the horror when we disconnected the sda drive to test the sdb drive on boot.

For those of you who don't quite understand my configuration here is the result of the following commands:

Code:
$ cat /proc/mdstat
Personalities : [raid1]
md3 : active raid1 sdb4[1](S) sda4[0]
      136664832 blocks [1/1] [U]

md2 : active raid1 sdb3[1](S) sda3[0]
      9767424 blocks [1/1] [U]

md1 : active raid1 sda2[0] sdb2[1](S)
      9767424 blocks [1/1] [U]

md0 : active raid1 sdb1[1](S) sda1[0]
      48064 blocks [1/1] [U]

unused devices: <none>
Code:
$ sudo mdadm --query --detail /dev/md0
Password:
/dev/md0:
        Version : 00.90.03
  Creation Time : Thu May 11 06:34:52 2006
     Raid Level : raid1
     Array Size : 48064 (46.95 MiB 49.22 MB)
    Device Size : 48064 (46.95 MiB 49.22 MB)
   Raid Devices : 1
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Sat Jun 24 13:39:18 2006
          State : clean
 Active Devices : 1
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 1

           UUID : efbff26c:f7e:)f3a3:cd:)cb948:61506d6:) 
         Events : 0.1016

    Number   Major   Minor   RaidDevice State
       0       8        1        0      active sync   /dev/sda1

       1       8       17        -      spare   /dev/sdb1
md1, md2, md3 are similar to md0. So what I want to do is promote my spare to a working device. I want to change the /etc/mdadm/mdadm.conf file for all Arrays and make the following changes:

Code:
devices=1 spare=1
             to
devices=2 spare=0
Of course there is more to the file I just homed in on the blood and guts of the dilemma. Before actually trying this is there something I have to do in order to assure that I don't completely break the servers? Also I haven't adjusted grub but I think I know how to do that to boot on backup if necessary. See you all at the Portland OSCON on the 24th of July. Thanks in advanced.
 
Old 06-24-2006, 09:27 PM   #2
WhatsHisName
Senior Member
 
Registered: Oct 2003
Location: /earth/usa/nj (UTC-5)
Distribution: RHL9;F1-10; CentOS4-5; DebianSarge-Squeeze
Posts: 1,151

Rep: Reputation: 46
It takes a lot of balls to admit a mistake like that, so I think you deserve some help.

First off, the purpose of mdadm.conf is to tell the OS what to expect. It has nothing to do with how software raids are configured.

But not to despair, the fix is easy! What you need to do is to --grow the raid1s.

Look at the mdadm man page for more info about using -G (--grow).

I set up a --grow example (see below) for you to look at. Modify the --grow commands to match your situation. Be sure to let the rebuild processes finish before you reboot.

When all of the raids have been rebuilt and before you reboot, update mdadm.conf as is shown at the end of the example (Very, Very, Very Important!!!).

Code:
# mdadm -C /dev/md14 -l1 -n1 -x1 /dev/hd[eg]14 --force
mdadm: array /dev/md14 started.

# cat /proc/mdstat
Personalities : [raid1]
md14 : active raid1 hdg14[1](S) hde14[0]
      15735552 blocks [1/1] [U]

unused devices: <none>

# mdadm -D /dev/md14
/dev/md14:
        Version : 00.90.03
  Creation Time : Sat Jun 24 21:39:10 2006
     Raid Level : raid1
     Array Size : 15735552 (15.01 GiB 16.11 GB)
    Device Size : 15735552 (15.01 GiB 16.11 GB)
   Raid Devices : 1
  Total Devices : 2
Preferred Minor : 14
    Persistence : Superblock is persistent

    Update Time : Sat Jun 24 21:39:10 2006
          State : clean
 Active Devices : 1
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 1

           UUID : 6a1f3a58:018cc3c9:7c3b2649:a2422600
         Events : 0.1

    Number   Major   Minor   RaidDevice State
       0      33       14        0      active sync   /dev/hde14

       1      34       14        -      spare   /dev/hdg14

# mdadm -G /dev/md14 -n2

# cat /proc/mdstat
Personalities : [raid1]
md14 : active raid1 hdg14[2] hde14[0]
      15735552 blocks [2/1] [U_]
      [=>...................]  recovery =  6.1% (961792/15735552) finish=7.1min speed=34349K/sec

unused devices: <none>

# # And the clouds parted and the birds began to sing!

# mdadm -D /dev/md14
/dev/md14:
        Version : 00.90.03
  Creation Time : Sat Jun 24 21:39:10 2006
     Raid Level : raid1
     Array Size : 15735552 (15.01 GiB 16.11 GB)
    Device Size : 15735552 (15.01 GiB 16.11 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 14
    Persistence : Superblock is persistent

    Update Time : Sat Jun 24 21:59:15 2006
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : 6a1f3a58:018cc3c9:7c3b2649:a2422600
         Events : 0.5

    Number   Major   Minor   RaidDevice State
       0      33       14        0      active sync   /dev/hde14
       1      34       14        1      active sync   /dev/hdg14


# # Update Debian-style mdadm.conf

# cd /etc/mdadm
# cp mdadm.conf mdadm.conf.`date +%y%m%d`
# echo "DEVICE partitions" > mdadm.conf
# mdadm -D -s >> mdadm.conf
# cd /root
Good Luck!

Last edited by WhatsHisName; 06-24-2006 at 09:36 PM.
 
Old 06-26-2006, 06:26 AM   #3
jcorrea920
LQ Newbie
 
Registered: Jun 2006
Location: Oregon, USA
Distribution: Ubuntu / Fedora
Posts: 14

Original Poster
Rep: Reputation: 0
Exclamation Thank you

Thank you so much for the advice, although on the test server the procedure worked fine, but on the production server when I tried to install grub so that the server could boot from either hardrive on startup for fault-tollerance, I get the following error:

Code:
# grub-install /dev/sda
# grub

grub> device (hd0) /dev/sdb

grub> root (hd0,0)
 Filesystem type is fat, partition type 0x6

grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... no

Error 15: File not found

grub> quit
How can Ubuntu run on fat? File /boot/grub/stage1 exists but grub cannot find it. With no success I even tried:

Code:
grub> find /boot/grub/stage1
Something is not right, and I am afraid to reboot the server since we need it functioning at 8:00 am PST today. I thank you in advance.

Stayed up all night trying to configure grub to boot from either disk on start up in case of disk failure
 
Old 06-26-2006, 09:05 AM   #4
WhatsHisName
Senior Member
 
Registered: Oct 2003
Location: /earth/usa/nj (UTC-5)
Distribution: RHL9;F1-10; CentOS4-5; DebianSarge-Squeeze
Posts: 1,151

Rep: Reputation: 46
Sounds like you’re running a Dell with the FAT16-based Dell Utility as the first partition.

Try searching for grub stage1 slightly differently:
Code:
grub> find /grub/stage1
which is used when you have a separate /boot partition.
 
Old 06-30-2006, 09:32 PM   #5
jcorrea920
LQ Newbie
 
Registered: Jun 2006
Location: Oregon, USA
Distribution: Ubuntu / Fedora
Posts: 14

Original Poster
Rep: Reputation: 0
Thumbs down Sorry that did not work either...

Yes you were right, it is a Dell

Quote:
Sounds like you’re running a Dell with the FAT16-based Dell Utility as the first partition.

Try searching for grub stage1 slightly differently:

Code:
grub> find /grub/stage1

which is used when you have a separate /boot partition.
but neither one of those commands find my stage1 What do I do now?


Code:
~$ ls -l  /boot/grub/stage*
-rw-r--r-- 1 root root    512 2006-06-30 19:22 /boot/grub/stage1
-rw-r--r-- 1 root root 105428 2006-06-30 19:22 /boot/grub/stage2
Doesn't matter what I do..

Code:
grub> find /boot/grub/stage1
or

Code:
grub> find /grub/stage1
Both come back not found. Thanks in advanced...
 
Old 07-03-2006, 12:38 PM   #6
jcorrea920
LQ Newbie
 
Registered: Jun 2006
Location: Oregon, USA
Distribution: Ubuntu / Fedora
Posts: 14

Original Poster
Rep: Reputation: 0
Question The Raid 1 Mirror seems to be working

So I disconnected sda primary hardrive and tried to boot from the second hardrive sdb and it won't boot. A message comes up as

Quote:
No loader.
I booted with knoppix to make sure that the RAID 1 replication was working and sdb seems to have all the files duplicated for all partitions as sda. Even the boot partition is duplicated. I have a feeling grub may not be configured correctly, which is the problem I had originally. I though maybe if I connected sdb to the sda connection it would boot, but it still comes up as "no loader". If sdb is a mirror of sda why won't it boot on its own? I cannot afford to lose all the data on sda or to destroy the boot loader, grub, on the system. And I can't seem to get past the find command previously noted to place sdb on the grub boot loader to boot. Thanks in advance, as any help would be greatly appreciated.
 
Old 07-05-2006, 10:20 AM   #7
WhatsHisName
Senior Member
 
Registered: Oct 2003
Location: /earth/usa/nj (UTC-5)
Distribution: RHL9;F1-10; CentOS4-5; DebianSarge-Squeeze
Posts: 1,151

Rep: Reputation: 46
“No Loader” sounds like a system BIOS issue.

Make sure the system BIOS boot options for the hard drives are correct.
 
Old 07-05-2006, 05:06 PM   #8
jcorrea920
LQ Newbie
 
Registered: Jun 2006
Location: Oregon, USA
Distribution: Ubuntu / Fedora
Posts: 14

Original Poster
Rep: Reputation: 0
Thumbs down well we tested that...

We disconnected the good drive and the one with the bad loader wouldn't boot weather it was plugged into SATA-0 or SATA-1 (on the motherboard) as if it was a good drive. It has a mirror images of all four partitions of the "good" drive (this was accomplished after your great advice to Grow the RAID Devices) By default, shouldn't it boot as well? Or as you are saying the BIOS recognizes drives other than by where they are connected? Because the BIOS is configured to boot from CDROM first, and then to any available hard-drive. We don't think that is the problem. What do you reccommend? We appreciate all you help, and we thank you for all your good advice.
 
Old 07-05-2006, 05:58 PM   #9
jcorrea920
LQ Newbie
 
Registered: Jun 2006
Location: Oregon, USA
Distribution: Ubuntu / Fedora
Posts: 14

Original Poster
Rep: Reputation: 0
Lightbulb grub is not connecting to filesystem...

It appears that grub is not communicating with the filesystem. When I reinstall grub and add device /dev/sdb it will not allow me to

Code:
grub> setup (hd0)
It says that file stage1 could not be found and doesnot exist. So the find command does not connect to filesystem, although on the bash shell prompt the find command works fine and can find /boot/grub/stage1 everytime.

Our system boots fine with sda, although not with sdb. We believe that during duplication something may have gone wrong.

So we would like to remove,reformat sdb and reinstall it to duplicate all over again then maybe grub will be able to communicate with the file system on disk sdb. Do we need to partition the newly formated drive or not? Does Ubuntu software RAID1 take care of it?

We are very grateful for you help, and apprecitate all the time and effort you have given us. Hope to hear from you soon.
 
Old 07-05-2006, 11:16 PM   #10
WhatsHisName
Senior Member
 
Registered: Oct 2003
Location: /earth/usa/nj (UTC-5)
Distribution: RHL9;F1-10; CentOS4-5; DebianSarge-Squeeze
Posts: 1,151

Rep: Reputation: 46
Starting back at the beginning sounds like a plan.

There should be no problem in reusing the existing disk partition structure, but as you already know, be sure to have the Debian installer reformat the filesystems so that you start with a clean slate.
 
Old 07-06-2006, 02:57 PM   #11
jcorrea920
LQ Newbie
 
Registered: Jun 2006
Location: Oregon, USA
Distribution: Ubuntu / Fedora
Posts: 14

Original Poster
Rep: Reputation: 0
Lightbulb Which Debian Installer would you recommend?

Won't a debian installer install the OS onto the drive after formatting it? Can't I just fdisk the disk on another machine and format it that way? Although, should I be formatting LinuxRAID or XFS3?
 
Old 07-06-2006, 03:14 PM   #12
WhatsHisName
Senior Member
 
Registered: Oct 2003
Location: /earth/usa/nj (UTC-5)
Distribution: RHL9;F1-10; CentOS4-5; DebianSarge-Squeeze
Posts: 1,151

Rep: Reputation: 46
At the step where you select the mount points, you need to specify how you want each partition/logical volume formatted (e.g., leave it alone, ext3, reiserfs, etc.).
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Why can't I mount this md0 raid? (mdadm and software raid) cruiserparts Linux - Software 35 01-05-2013 03:35 PM
RAID - mdadm.conf keith ralph Linux - General 4 05-05-2006 09:10 AM
Linux raid with mdadm eyt Linux - Newbie 6 02-07-2006 05:26 AM
Mdadm Raid question stefaandk Linux - General 12 08-16-2005 02:11 AM
Mailing with MDADM RAID rootking Linux - Hardware 1 07-26-2005 04:47 PM


All times are GMT -5. The time now is 04:45 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration