[SOLVED] recreate raid-5 array w/o loosing data

BinWondrin · 12-11-2010, 02:18 PM

I rebuilt a server and am now trying to recover my large data arrays.
The server was ubuntu 10.04lts before.
I decided to rebuild it with CentOS simply because I am more familiar with it.
I had 2 raid-5 arrays on the old server:

4 x 1tb -> md0
5 x 2tb -> md1

The newly built server does not know about these arrays yet.
How can I reassemble the arrays without loosing my data?
I know the data can still be accessed because booting the server with a live-cd mounts and shows the arrays just fine.
Should I boot with a live cd and copy the mdadm config file?

Thanks in advance.

BinWondrin
;-)

stress_junkie · 12-11-2010, 02:24 PM

The live CD probably just runs dmraid -ay.

Code:

dmraid -ay

Regarding the "...without losing data..." part I recommend ensuring that you have an up to date backup before doing anything else. Mr. Murphy is always looking for an opportunity to mess you up. Since the live CD will assemble the arrays properly you could use that to make your backups if you don't already have them.

BinWondrin · 12-11-2010, 02:32 PM

Thank you. I gave it a shot and here is what I get:

These are the devices:

Quote:

Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
Disk /dev/sdc: 2000.3 GB, 2000398934016 bytes
Disk /dev/sdd: 2000.3 GB, 2000398934016 bytes
Disk /dev/sde: 2000.3 GB, 2000398934016 bytes
Disk /dev/sdf: 2000.3 GB, 2000398934016 bytes
Disk /dev/sdg: 1000.2 GB, 1000204886016 bytes
Disk /dev/sdh: 1000.2 GB, 1000204886016 bytes
Disk /dev/sdi: 1000.2 GB, 1000204886016 bytes
Disk /dev/sdj: 1000.2 GB, 1000204886016 bytes
Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
Disk /dev/sdc: 2000.3 GB, 2000398934016 bytes
Disk /dev/sdd: 2000.3 GB, 2000398934016 bytes
Disk /dev/sde: 2000.3 GB, 2000398934016 bytes
Disk /dev/sdf: 2000.3 GB, 2000398934016 bytes
Disk /dev/sdg: 1000.2 GB, 1000204886016 bytes
Disk /dev/sdh: 1000.2 GB, 1000204886016 bytes
Disk /dev/sdi: 1000.2 GB, 1000204886016 bytes
Disk /dev/sdj: 1000.2 GB, 1000204886016 bytes

Here is the return from dmraid -ay:

Quote:

dmraid -ay
ddf1: virtual drives with CRC 466DCCE2, expected 866C0109 on /dev/sda
ddf1: physical drives with CRC EA982444, expected 8A877121 on /dev/sda
ddf1: VD CFG with CRC F4AB4282, expected 981107F1 on /dev/sda
ddf1: VD CFG with CRC EA90F2, expected ECF124FA on /dev/sda
ERROR: ddf1: cannot find virtual drive record on /dev/sda
ERROR: ddf1: wrong # of devices in RAID set "ddf1_SYSTEM" [1/4] on /dev/sda
RAID set "ddf1_SYSTEM" already active
The dynamic shared library "libdmraid-events-ddf1.so" could not be loaded:
libdmraid-events-ddf1.so: cannot open shared object file: No such file or directory
ERROR: dos: reading /dev/mapper/ddf1_SYSTEM[No such device or address]

stress_junkie · 12-11-2010, 02:34 PM

That's unfortunate.

You should definitely ensure that you have a good backup of the data before proceeding.

Do you have a backup of the mdadm config file from the original system?

BinWondrin · 12-11-2010, 03:20 PM

O.K. - I figured it out:

mdadm will do it if supplied with the correct original raid device name followed by the devices that form ther array:

Quote:

mdadm --assemble /dev/md0 /dev/sdg /dev/sdh /dev/sdi /dev/sdj
mdadm: /dev/md0 has been started with 4 drives.
[root@megastorix ~]# mdadm --assemble /dev/md1 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf
mdadm: /dev/md1 has been started with 5 drives.
[root@megastorix ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4]
md1 : active raid5 sdb[0] sdf[5] sde[3] sdd[2] sdc[1]
7814057472 blocks super 1.2 level 5, 128k chunk, algorithm 2 [5/5] [UUUUU]

md0 : active raid5 sdg[0] sdj[4] sdi[2] sdh[1]
2930287104 blocks super 1.2 level 5, 128k chunk, algorithm 2 [4/4] [UUUU]

unused devices: <none>

stress_junkie · 12-11-2010, 03:42 PM

Thank you for posting the answer and for marking the thread "solved".

mwtzzz · 09-10-2012, 08:09 PM

Quote:

Originally Posted by BinWondrin

ERROR: ddf1: cannot find virtual drive record on /dev/sda

Before doing anything, you gotta make sure all previous RAID metadata is erased from disk. "dmraid remove_all" does not erase it.

The DDF1 array stores metadata at the end of the hard drive. One way to erase it is to zero out the entire hard drive, but that could take too much time depending on size.

Another way, is to find out where the metadata is stored. You can do this in Linux by running "strace dmraid -c -s > /tmp/out 2>&1". Look at the output file for "lseek" which will give you the byte offset of the metadata. The metadata is stored in a bunch of different sectors, so "lseek" will appear a bunch of times, just find the smallest byte offset and use "dd" to zero the data from that point forward.