How do I get mdadm to assemble a raid using local and nbd devices
 

Hello all,

My goal is to get mdadm to assemble a raid using a nbd and a local disk.

I have an init.d startup script to do this:

modprobe nbd
/usr/sbin/nbd-client 192.168.1.3 2000 /dev/nbd0
mdadm --assemble /dev/md2 /dev/sdb6 /dev/nbd0
lvm vgchange -a y
mount -t reiserfs /dev/vg00/home /home


The mdadm --assemble line returns
"/dev/md2 has been started with 1 drive (out of 2)"

Either of the following two commands run with the same result:
mdadm --assemble /dev/md2 --run /dev/nbd0
mdadm --assemble /dev/md2 --run /dev/sdb6

i.e. both sdb6 and nbd0 are accessible. But for some reason
mdadm insists on using just one of them when given both.

This creates problems for me. If I add the devices sequentially
to the array, e.g.:

mdadm --assemble /dev/md2 --run /dev/sdb6
mdadm --manage /dev/md2 --write-mostly --re-add /dev/nbd0


If the nbd0 is actually more up to date, the changes get lost in the resync. What I need to do is assemble them both at the same time so that raid can determine which is the more up to date and do the resynch in the right direction. But mdadm refuses to add them to the array at the same time.

I'm stuck.

Can anybody help?


Thanks in advance,

Adrian Stephens - Cambridge UK.

Probably this is a simple minded question, but if you DO add them sequentially, and let them resync fully, then shutdown and restart the system, does mdadm still refuse to add them?
BTW, I'm guessing the only reason they'd get out of sync is that /dev/nbd0 would miss writes if the network goes down, so how would it get more up to date?

To answer the last question: nbd0 can be more up to date if there is a failure of the local disk. I think I've got an unreliable SATA connector, but I've certainly seen failures of the local hard disk component. When this happens nbd0 continues to support the raid component (as it's supposed to), and then the local disk is out of date.

Nothing I found works automatically. I can force the correct update by adding nbd0 first with --run, and then sdb6 with --re-add.

I think the moral of this story is: 1. make sure the raid is started after init 2 processing. After a RAID failure, to boot into single mode (init 2 I think), use mdadm --examine to determine which is the more up to date component, manually assemble --run the more up to date component, then --re-add the out of date one. Wait for the sync and reboot into normal operation.

By the way, I had a hang about once per day. Although I've not localized it, I suspect the raid/nbd combination, and I'm going to remove the nbd raid component, and instead do a periodic unison backup to the remote server. Less high tech, but I'm guessing I'll have less trouble with it.

Well, yeah, but I meant under normal circumstances. If failure of the disk is a regular worry, then you have bigger problems. RAID 1 is a bit like harddisk insurance; there in the unlikely event that you need it - but NOT as a substitute for regular backups. So you are probably much better off with your current plan to do periodic backups. But if you have a faulty SATA connector then (a) replace it (b) don't use it (c) use local RAID (ie. another local harddisk)as insurance in addition to your backups.