LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 02-26-2011, 07:07 PM   #1
thewird
LQ Newbie
 
Registered: Feb 2011
Posts: 22

Rep: Reputation: 0
How to Mirror the Boot Partition on Existing Raid1


Hi, I have a raid1 setup on a machine. Recently it died and I thought one of the drives had failed as it was shooting errors. So I tried unplugging that drive get it to boot off the mirror but it seems the techs forgot to mirror the boot device so the 2nd drive can't boot on its own. After a while it was realized that the sata cable was in fact bad and replaced so now its working again.

However, this occurrence showed a flaw in the setup where the RAID1 isn't working as its supposed to. I would like to correct this. Can I somehow mirror the boot partition so the 2nd drive will boot independent? I'm not sure how I would go about this. This is a CentOS 5 installation. Thanks.

thewird

Last edited by thewird; 02-26-2011 at 07:09 PM.
 
Old 02-27-2011, 01:45 PM   #2
jefro
Guru
 
Registered: Mar 2008
Posts: 11,969

Rep: Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484
I suspect that you have a faux raid card. Real hardware raid cards work correctly.
 
Old 02-27-2011, 01:58 PM   #3
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
Quote:
Can I somehow mirror the boot partition so the 2nd drive will boot independent?
The short answer is yes. But we need to know how your system is set up now in terms of partitions on both drives and how your RAID is set up. Is it hardware or software (md) raid?

Some background that you are probably aware of. It's common practice to make the boot directory a separate file system (especially on Fedora/Redhat/Centos systems). And for the longest time you couldn't put your boot partition on RAID1. So the practice was to have identical boot partitions kept in sync with rsync, and RAID1 everything else. And even though I think /boot on RAID1 is supported now, I don't know anyone that has done it. See this, though: http://www.linuxquestions.org/questi...1-grub-394494/

I don't fully agree with that post. The key to being able to boot off both systems is to make sure the boot loader is written to both drives and kept up to date. (My guess is that you are on an x86 system and have grub for a boot loader.)

So, what do you have for partitions on the two drives? An 'fdisk -l' would tell us the disk layouts. And 'cat /proc/mdstat' would tell us about your software RAID setup (of course, assuming software RAID).

Last edited by tommylovell; 02-27-2011 at 07:17 PM. Reason: technical error - there is no such thing as /dev/mdstat
 
Old 02-27-2011, 05:45 PM   #4
thewird
LQ Newbie
 
Registered: Feb 2011
Posts: 22

Original Poster
Rep: Reputation: 0
Oddly when I type "cat /dev/mdstat" I get "cat: /dev/mdstat: No such file or directory". However, I am positive that the hard disks are software raid1. What could this be?

Quote:
fdisk -l

Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 19326 155236063+ fd Linux raid autodetect
/dev/sda2 19327 19457 1052257+ 82 Linux swap / Solaris

Disk /dev/sdb: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 19326 155236063+ fd Linux raid autodetect
/dev/sdb2 19327 19457 1052257+ 82 Linux swap / Solaris

Disk /dev/md0: 158.9 GB, 158961631232 bytes
2 heads, 4 sectors/track, 38808992 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

Disk /dev/md0 doesn't contain a valid partition table
thewird
 
Old 02-27-2011, 07:16 PM   #5
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
Oops. My error. And 'cat /dev/mdstat' should be
Code:
cat /proc/mdstat
(I'll correct it in the prior post!)
 
Old 02-27-2011, 07:47 PM   #6
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
But from your 'fdisk -l' output your /dev/sda1 and /dev/sdb1 partitions are paired into a single RAID1 array called /dev/md0.

Your 'cat /proc/mdstat' will look something like this if it is healthy.
Code:
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] 
md0 : active raid1 sda1[0] sdb1[1]
      xxxxxxxx blocks [2/2] [UU]
      
unused devices: <none>
If it doesn't say [2/2] [UU] then it is running degraded and that is (at least) part of your problem.

If it is [UU] then I think you are all set in terms of having everything mirrored and kept in sync and you probably just need the boot loader written to your second drive.

If you do an 'ls /boot' and there is a 'grub' subdirectory you're using grub as a boot loader. And it is probably "grub 0.something".
Code:
[root@athlonz ~]# ls /boot
config-2.6.27.41-170.2.117.fc10.x86_64  grub                                        lost+found                                  vmlinuz-2.6.27.41-170.2.117.fc10.x86_64
efi                                     initrd-2.6.27.41-170.2.117.fc10.x86_64.img  System.map-2.6.27.41-170.2.117.fc10.x86_64

[root@athlonz ~]# rpm -qa | grep grub
grub-0.97-38.fc10.x86_64

[root@athlonz grub]# grub --version
grub (GNU GRUB 0.97)
Please confirm that and I'll research how you would write grub to your 2nd drive.

Sorry about the /dev/mdstat thing.

Last edited by tommylovell; 02-27-2011 at 09:43 PM.
 
Old 02-27-2011, 09:31 PM   #7
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
I found a good article describing what you are looking for. It's http://www.ping.co.il/node/1/
The only difference is that it is for system with boot split out as a separate filesystem so there is an md0 and an md1 array. Yours is just md0.

What you need to do is start at Step 3. You'll be doing a slightly modified version of that. Your hd0 (drive 1) is already set up, so all you need to do is the "device (hd1)" part.

Invoke the grub shell.
Code:
# grub
In the GRUB shell type the following commands to install the boot loader on second drive.
Code:
grub> device (hd1) /dev/sdb
grub> root (hd1,0)
grub> setup (hd1)
You should now be all set.

Look at the subsequent steps in that article. And take that author's advice to heart and take a backup before you muck about with your system. Things can go wrong.

Good luck.
 
Old 02-27-2011, 09:37 PM   #8
thewird
LQ Newbie
 
Registered: Feb 2011
Posts: 22

Original Poster
Rep: Reputation: 0
Oh dear...

Quote:
cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[1]
155235968 blocks [2/1] [_U]

unused devices: <none>
thewird
 
Old 02-27-2011, 10:16 PM   #9
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
I think you can simply re-add the sda1 partition back into the md0 array.

There are two 'mdadm' commands that are useful that you may want to use first. 'mdadm --detail /dev/md0' will display the array "looking down"; 'mdadm --examine /dev/sda2' will display from the component's viewpoint (yours is sda1, not sda2).

Code:
]# mdadm --detail /dev/md0
/dev/md0:
        Version : 0.90
  Creation Time : Tue Mar 10 06:59:43 2009
     Raid Level : raid1
     Array Size : 25005056 (23.85 GiB 25.61 GB)
  Used Dev Size : 25005056 (23.85 GiB 25.61 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Sun Feb 27 21:54:21 2011
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : 397c7395:9c3c48fa:68e69357:c9f2169f
         Events : 0.632582

    Number   Major   Minor   RaidDevice State
       0       8        2        0      active sync   /dev/sda2
       1       8       34        1      active sync   /dev/sdc2
Code:
# mdadm --examine /dev/sda2
/dev/sda2:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 397c7395:9c3c48fa:68e69357:c9f2169f
  Creation Time : Tue Mar 10 06:59:43 2009
     Raid Level : raid1
  Used Dev Size : 25005056 (23.85 GiB 25.61 GB)
     Array Size : 25005056 (23.85 GiB 25.61 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0

    Update Time : Sun Feb 27 21:55:11 2011
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 4a6ee6f4 - correct
         Events : 632582


      Number   Major   Minor   RaidDevice State
this     0       8        2        0      active sync   /dev/sda2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       34        1      active sync   /dev/sdc2
[root@athlonz ~]#
It may be as simple as this to add the failed device.
Code:
mdadm --manage /dev/md0 --add /dev/sda1
In practice, you can fail, remove then add a component (the underlying partition) like this.
Code:
mdadm /dev/md0 -f /dev/sda1 -r /dev/sda1 -a /dev/sda1
-or-
mdadm --manage /dev/md0 --fail /dev/sda1 --remove /dev/sda1 --add /dev/sda1
Or do it as three separate commands.

But basically, you need to fail the component before it can be removed; and remove a component before it can be added.

Once added, you can watch it rebuild using the 'cat /proc/mdstat' command repeatedly. At 160GB, it'll take a few minutes to sync.

Your /dev/sda1 is probably already failed. But it may need to be removed before it is added.

That is, of course, removed logically from the array, not the hardware physically from the server!!!!

Last edited by tommylovell; 02-27-2011 at 10:26 PM.
 
Old 02-27-2011, 11:39 PM   #10
thewird
LQ Newbie
 
Registered: Feb 2011
Posts: 22

Original Poster
Rep: Reputation: 0
I used the add command you suggested and it worked.

Quote:
mdadm --manage /dev/md0 --add /dev/sda1
After syncing it now seems to be ok...

Quote:
cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0] sdb1[1]
155235968 blocks [2/2] [UU]

unused devices: <none>
Quote:
# mdadm --detail /dev/md0
/dev/md0:
Version : 0.90
Creation Time : Sun Jul 19 17:36:28 2009
Raid Level : raid1
Array Size : 155235968 (148.04 GiB 158.96 GB)
Used Dev Size : 155235968 (148.04 GiB 158.96 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Sun Feb 27 22:54:36 2011
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : 14122fa5:f0616d66:fdd160c0:449a36d2
Events : 0.143262

Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
I'm assuming this happened because of the bad sata cable, is there any way to actually test the drive just to double check? Before I proceed with trying to make the 2nd hard drive bootable.

thewird

Last edited by thewird; 02-27-2011 at 11:55 PM.
 
Old 02-28-2011, 12:06 AM   #11
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
If you power down; remove the sata cable from the first drive; power up; that'll prove whether you have a boot loader on the 2nd drive.

If it does boot, power down; replace the cable; reboot; check the md0 array, and re-add it like before, and you're done.

If it doesn't boot, power down; replace the cable; check the md0 array and re-add if necessary (it's probably still good); then use the grub shell to write the boot loader to the 2nd drive.

Invoke the grub shell.
Code:
# grub
In the GRUB shell type the following commands to install the boot loader on the second drive.
Code:
grub> device (hd1) /dev/sdb
grub> root (hd1,0)
grub> setup (hd1)
Then start from the top of this note. You should be ok.

If not, follow the "power down/replace cable/reboot/repair array" routine. Then we'll need to find someone with more expertise than me to figure out the grub stuff.

Make certain that anytime you've caused a drive failure, or discover one, you repair it asap. It's much easier to do it from a running system than from rescue mode.
 
Old 02-28-2011, 12:09 AM   #12
thewird
LQ Newbie
 
Registered: Feb 2011
Posts: 22

Original Poster
Rep: Reputation: 0
What I ment by test was to make sure the drive is indeed fine and the cause the the error which made it fail and not boot originally was in fact the sata cable which was replaced.

thewird
 
Old 02-28-2011, 12:42 AM   #13
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
I suppose there are three things that say the drive is sound.

If the drive has S.M.A.R.T. capability, you can run the smartd daemon.
If the drive has synced (as your 'cat /proc/mdadm' has shown).
And you could always just do a read of the whole drive, like 'dd if=/dev/sda of=/dev/null count=155235968 bs=1k'. (There's nothing that says you can't read from an md component - you just never, ever, want to write directly to it...)
But that's just one pass, so it may not really force the error if it still exists.

S.M.A.R.T. may be a better indicator of pending failure if it is the drive and not motherboard related.

There was at one time the Linux Test Project (LTP), but I don't know if that is still viable.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Missing Drive From RAID1 Mirror carlosinfl Linux - Server 1 10-30-2009 07:47 PM
RAID1 how does it mirror roders22 Linux - Newbie 3 11-06-2008 06:53 AM
input/output error on grub.conf file on raid1 boot partition desimo Linux - Server 2 06-25-2007 09:53 AM
break raid1 mirror hoover93 Linux - Software 0 10-05-2005 11:55 AM
Soft RAID1 (mirror) rebuild GAVollink Linux - Hardware 4 04-25-2003 09:18 AM


All times are GMT -5. The time now is 06:46 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration