LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 06-04-2013, 03:20 PM   #1
swamp-dog
LQ Newbie
 
Registered: Sep 2010
Posts: 6

Rep: Reputation: 0
software raid1, underlying disks


Hi Folks,

Three raid partitions, md0-md2.

md0 boot
md1 OS (centos 5.9)
md2 data

/dev/sda1 * 1 33 265041 fd Linux raid autodetect
/dev/sda2 34 8388 67111537+ fd Linux raid autodetect
/dev/sda3 8389 121601 909383422+ fd Linux raid autodetect
/dev/sdb1 * 1 33 265041 fd Linux raid autodetect
/dev/sdb2 34 8388 67111537+ fd Linux raid autodetect
/dev/sdb3 8389 121601 909383422+ fd Linux raid autodetect

Smartd is reporting pending sectors on /dev/sdb so I bought a pair of 2Tb disks. I have added one of the new disks live (this mobo is hot swappable). This appeared as /dev/sdp. Added it as a spare.

sfdisk -d /dev/sda | sfdisk /dev/sdp
dd if=/dev/sda/ bs=512 count=1 of=/dev/sdp

Failed /dev/sdb[1-3] and raid1 is currently re-syncing to /dev/sdp. Machine is physically difficult to get at so would prefer powering it off only the once.

Question is this: When /dev/sdp is synced. Can I pull the machine to bits, remove both /dev/sda and /dev/sdb disks (scsi 1:0:0:0 & 2:0:0:0 respectively), stick /dev/sdp in their place *plus* the virgin 2Tb (blank) counterpart & expect the machine to boot off of what was /dev/sdp.

Any thoughts?
TIA
 
Old 06-04-2013, 06:54 PM   #2
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Lenny/Squeeze/Wheezy/Sid
Posts: 4,087

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
You recreated the partion table using sfdump. That is good.

However, the second step (dd) I don't understand. There you did copy the first 512 bytes again, overwriting the partition table.

It seems like you wanted to copy the mbr. But in that case you should have copied only the first 446 bytes, and leave the rest (446-512) alone because that was the partition table. I think the dd step is not necessary and damages the partition table.

I did this a long long time ago, but to give you some pointers:
  1. Copy the partition table using sfdump as you did
  2. Take out the failed disk and insert the new one and let them sync
  3. Now very important, you must install the boot loader (see below)
  4. Once syncing is complete and you installed the boot loader, fail the second (formerly) good disk, and follow the same procedure

Read this article.
http://www200.pair.com/mecham/raid/r...aded-etch.html
It is not fully applicable as it start installing an OS on a degraded array, but it points out very well how to make your RAID bootable. Pay attention that both disks are made bootable. If not, the machine might not boot of the disk with the boot loader fails.

Since you are replacing the disks using the re-assemble mechanism of mdadm itself the UUID of the disks should not give problems, but check anyway if mdadm.conf is updated with the UUIDs of the newly assembled array.

jlinkels

PS. You don't want to do this on a production machine. Use some old hardware and make test runs. I think it can even be done in a VM.

Last edited by jlinkels; 06-04-2013 at 06:58 PM.
 
Old 06-06-2013, 03:36 AM   #3
swamp-dog
LQ Newbie
 
Registered: Sep 2010
Posts: 6

Original Poster
Rep: Reputation: 0
Yes. It was just the first 446 bytes I wanted to copy. I think I'd best repeat the whole thing to be safe.

One further question. Atm the new disk appeared as /dev/sdp but when I replace the old /dev/sdb it will become /dev/sdb - will mdadm be able to handle that via UUID's? Also bearing this in mind, how will this affect what I tell grub when I install the bootloader (which I had completely forgotten about!)?

TIA
 
Old 06-06-2013, 06:43 PM   #4
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Lenny/Squeeze/Wheezy/Sid
Posts: 4,087

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
Quote:
Originally Posted by swamp-dog View Post
Yes. It was just the first 446 bytes I wanted to copy. I think I'd best repeat the whole thing to be safe.
No, there is no need for dd.
The sfdump procedure is good and does what is needs to do. There is no need for a byte-for-byte copy of the mbr. It only can (and will confuse) things. Install grub as pointed out in the web page I pointed to.

Quote:
Originally Posted by swamp-dog View Post
One further question. Atm the new disk appeared as /dev/sdp but when I replace the old /dev/sdb it will become /dev/sdb - will mdadm be able to handle that via UUID's? Also bearing this in mind, how will this affect what I tell grub when I install the bootloader (which I had completely forgotten about!)?
TIA
I hope the new disk becomes /dev/sdb. It is highly suspicious that the last disk became /dev/sdp. Very unusual. What happened to /dev/sdc through /dev/sdo?? When you add the new sdb to the array (and there is a procedure for that, check again that web page or the man page) mdadm takes care that the new UUID will be used. If the disk does not become /dev/sdb but something else, you can recreate the array with that something else, but you have to be certain that the device names remain the same through a reboot.

For the boot loader it doesn't matter. Just do a grub install to the real /dev/sdb or /dev/sda ignoring any raiding.

About device names at booting: in case you have different device names after installing new disks, forget about options to change them using udev. udev is only acting after the array is assembled. It won't rename drives before assembling, so the raid will assemble using the device names they get a booting, period.

Again, you should try this beforehand on a test system. mdadm is quite resilient and non-destructive, but Murphy's law is a strong contender when working on live data.

jlinkels
 
Old 06-08-2013, 12:57 PM   #5
swamp-dog
LQ Newbie
 
Registered: Sep 2010
Posts: 6

Original Poster
Rep: Reputation: 0
Thumbs up

With your help I fixed it. :-)

The missing disks are a red herring. There were 7 disks attached, two of which were usb caddies. They can be a bit flakey in that they sometimes reinitialise themselves *plus* they are LUKS encrypted. It being the weekend I can reboot with them turned off. The remaining three are simple ext3 "whole disk" partitions for primary backup purposes (caddies being for offsite). It just so happened that when I plugged in one of the new disks it appeared as /dev/sdp.

The main issue was the bios. When I have it boot off another drive it appears to be mapping that drive to "slot 0" rather than the bios slot listed initially - difficult to be certain as that bios info appears fleetingly. Long & short of it I must have been installing grub onto the wrong disk.

Things improved last night. I ripped the machine to bits and replaced both old disks with the new ones. Wouldn't boot (grub grub grub) until I reattached the old working disk. This appeared as /dev/sde(*) which is what gave me the hint about the bios above. That and mdadm picking /dev/sd[ab] for /dev/md[01] but electing for /dev/sde2 for /dev/md2 had me very confused(**) over which actual disks were being used for what.

Anyway, with the new disks connected to their proper/final sata connectors I successfully installed & verified grub to both new disks.

(*) Dunno why /dev/sde, that should be one of the primary backup disks.

(**) Cockpit error on my part methinks. Very reluctant to tell mdadm to fail /dev/sde3 so methinks mdadm (for some reason) picked that up in preference to /dev/sdb3. I don't understand why but /var/log/messages ..

Jun 7 19:33:00 sdn kernel: md: Autodetecting RAID arrays.
Jun 7 19:33:00 sdn kernel: md: autorun ...
Jun 7 19:33:00 sdn kernel: md: considering sdb1 ...
Jun 7 19:33:00 sdn kernel: md: adding sdb1 ...
Jun 7 19:33:00 sdn kernel: md: sdb2 has different UUID to sdb1
Jun 7 19:33:00 sdn kernel: md: sdb3 has different UUID to sdb1
Jun 7 19:33:00 sdn kernel: md: md0 already running, cannot run sdb1
Jun 7 19:33:00 sdn kernel: md: export_rdev(sdb1)
Jun 7 19:33:00 sdn kernel: md: considering sdb2 ...
Jun 7 19:33:00 sdn kernel: md: adding sdb2 ...
Jun 7 19:33:00 sdn kernel: md: sdb3 has different UUID to sdb2
Jun 7 19:33:00 sdn kernel: md: md1 already running, cannot run sdb2
Jun 7 19:33:00 sdn kernel: md: export_rdev(sdb2)
Jun 7 19:33:00 sdn kernel: md: considering sdb3 ...
Jun 7 19:33:00 sdn kernel: md: adding sdb3 ...
Jun 7 19:33:00 sdn kernel: md: md2 already running, cannot run sdb3
Jun 7 19:33:00 sdn kernel: md: export_rdev(sdb3)
Jun 7 19:33:00 sdn kernel: md: ... autorun DONE.

..gave me a hint I should do exactly that.

On the upside, I've learnt more about mdadm than I ever thought I needed to know.

Cheers!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
New disks (RAID1), not enough space? ScorchPipe Linux - General 1 01-17-2011 01:18 AM
Making bootable both disks of mdadm raid1 chiendarret Linux - Software 0 03-08-2009 05:15 AM
Software RAID1 SBN Linux - General 1 01-21-2009 08:47 AM
Have software raid1, but like to change to raid1+0 or 0+1, how? spaceuser Debian 8 03-17-2008 02:07 PM
Change raid1-mdadm disks Debian Suhy Linux - Software 1 07-03-2006 03:48 PM


All times are GMT -5. The time now is 08:58 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration