LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 03-03-2011, 08:02 PM   #1
dkeith
LQ Newbie
 
Registered: Mar 2011
Posts: 5

Rep: Reputation: 0
RAID 1 Partition devices disappear after upgrade


I found a workaround of sorts. It looks like this is related to a 9.04 bug (https://bugs.launchpad.net/ubuntu/+s...nux/+bug/27037) and the loopback workaround brings back the array. It is not clear how I will handle this long term.

Note: before using this technique, I used gparted to tag the partitions as "raid". They disappeared again on reboot, so I had to do it again. I am not sure how this is going to work out long-term

Note: I suspect some of this is related to the embedded "HOMEHOST" that is written into the RAID metadata on the paritions. The server was misnamed when first built and the name was changed later (cerebus -> cerberus) and the old name has surfaced in the name of a phanton device reported by gparted - /dev/mapper/jmicron_cerebus_root

----------------


Good day, experts!

I am taking a stab that the Server forum is correct for this post.

I have a mythbuntu 9.10 system that I have upgraded from 8.10 to 9.04 to 9.10 in the last 2 days. I am on my way to 10.x, but need to make sure it works after every step.

The basic problem is that in its current incarnation, it is not recognizing the underlying partitions for one of the RAID devices, and therefore not happy.

As a 8.10 system
I had 2 raid devices:
/dev/md16 -> /dev/sda5 and /dev/sdb5
/dev/md21 -> /dev/sdc1 and /dev/sdd1

/etc/fstab looked like this (in part):
/dev/md16 /var/lib xfs defaults 0 2
/dev/md21 /u02 xfsi defaults 0 2

After upgrading to 9.04
- both raid devices failed to load, so I enetually figured out run this command:
mdadm --examine --scan >> /etc/mdadm/mdadm.conf

- this added these lines to /etc/mdadm/mdadm.conf
ARRAY /dev/md16 level=raid1 num-devices=2 UUID=1664eda9:d2695350:7635e7b7:75a625cd
ARRAY /dev/md21 level=raid1 num-devices=2 UUID=8709c430:7a9eb6d3:7635e7b7:75a625cd

- a reboot later and they were back in business
- /dev/sd16 was half busted, but it rebuilt fine when I ran:
mdadm /dev/md16 -a /dev/sdb6
- note - the file system type worked with xfs, but did not recognize xfsi under 9.04

After the upgrade to 9.10:
- /dev/md16 is very happy
- /dev/md21 is nowhere to be seen

>> here are the partitions that were part of the original RAID 1:

root@cerberus:~# fdisk -l /dev/sdc /dev/sdd

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0001b2c7

Device Boot Start End Blocks Id System
/dev/sdc1 1 121601 976760001 83 Linux

Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0007f7aa

Device Boot Start End Blocks Id System
/dev/sdd1 1 121601 976760001 83 Linux


>> Here are the devices - or lack thereof:
root@cerberus:~# ls -l /dev/sd[cd]*
brw-rw---- 1 root disk 8, 32 2011-03-03 20:21 /dev/sdc
brw-rw---- 1 root disk 8, 48 2011-03-03 20:21 /dev/sdd


>> I tried creating the devices manually with mknod, but they did not work as file system handles, and were wiped out on reboot:
mknod --mode=660 /dev/sdc1 b 8 33
mknod --mode=660 /dev/sdd1 b 8 49

>> I tried using mdadm to build a 1-way RAID 1 with the disk device. (I was working with the single disk only in case it scragged the drive - this way I had an untouched copy)
root@cerberus:~# mdadm --create /dev/md21 --force --level=1 --raid-devices=1 /dev/sdc
mdadm: Cannot open /dev/sdc: Device or resource busy
mdadm: create aborted

note - this drive is not in use as swap or otherwise mounted that I can tell

>> I am a little fuzzy on when I did this, but adding these lines to /etc/initramfs-tools/modules and rebuilding (update-initramfs -u) helped get things at some point. I *think* this was required to get RAID to work at all after upgrading to 9.10. Here are the lines:
raid1
raid456
the raid456 line does not seem to do anything - I took it out with no change

>> running gparted lists the partition, but I get a warning triangle with an info box that says "fatal error - couldn't initialize XFS library"

>> a few more data points:
root@cerberus:~# mdadm --examine --scan
ARRAY /dev/md16 level=raid1 num-devices=2 UUID=1664eda9:d2695350:7635e7b7:75a625cd

root@cerberus:~# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md16 : active raid1 sdb6[1] sda6[0]
964044480 blocks [2/2] [UU]

unused devices: <none>

root@cerberus:~# mdadm --examine /dev/sdc1
mdadm: cannot open /dev/sdc1: No such file or directory
root@cerberus:~# mdadm --examine /dev/sdc
mdadm: No md superblock detected on /dev/sdc.

root@cerberus:~# cat /proc/partitions
major minor #blocks name

8 0 976762584 sda
8 1 11719386 sda1
8 2 1 sda2
8 5 995998 sda5
8 6 964044553 sda6
8 16 976762584 sdb
8 17 11719386 sdb1
8 18 1 sdb2
8 21 995998 sdb5
8 22 964044553 sdb6
8 32 976762584 sdc
8 48 976762584 sdd
9 16 964044480 md16
252 1 976584704 dm-1

I am a bit puzzled by this last line here; the size lines up with my lost md device, but there is no /dev/md-1 (it was md-0 earlier). There is a device withthis major/minor here:
brw-rw---- 1 root disk 252, 1 2011-03-03 20:21 /dev/mapper/jmicron_cerebus_root
... and it shows up in gparted as an unallocated partition of pretty-close-to-drive size

>> the HD controller is built-in to the mother board - some sort of jmicron thing. There are 4 drives - all 1 TB SATA IIRC.



I don't *really* want to repartition the drive as there is a small amount of data loss between recent backups and what is on the drive, plus it would take me 2 days to move the data back.

Many thanks,
dKeith

Last edited by dkeith; 03-04-2011 at 03:35 PM. Reason: more info
 
Old 03-04-2011, 06:38 PM   #2
dkeith
LQ Newbie
 
Registered: Mar 2011
Posts: 5

Original Poster
Rep: Reputation: 0
More detail on the workaround... (see top of updated original post)

I write a quickie script that does the necessary incantation on boot, and plugged in into the "start)" block in /etc/init.d/mdadm.

For some reason, /dev/sd[cd]1 are wiped out on reboot, but they show up again when you write the disk partition tables. The sleep 3 is there because there seems to be a timing issue with issuing the mdadm too fast.

The script in its entirety is:

>< snip ><

#! /bin/bash

for DEV in sdc sdd ; do
fdisk /dev/${DEV} << EOF
w
quit
EOF
done

losetup /dev/loop0 /dev/sdc1
losetup /dev/loop1 /dev/sdd1
sleep 3
mdadm --assemble /dev/md21 /dev/loop0 /dev/loop1

>< snip ><

I would still like to find a proper resolution instead of this hack.

In the meantime, on to 10.04 and eventually 10.10.

Best,
dKeith
 
Old 03-05-2011, 06:41 AM   #3
dkeith
LQ Newbie
 
Registered: Mar 2011
Posts: 5

Original Poster
Rep: Reputation: 0
Update #2

I upgraded from 9.10 -> 10.04 LTS.

The device ID's shuffled around, like the below. Note that a&b are a pair and c&d are a matched pair, so I am not sure how they changed pairwise, but the pairs swapped place.
sda/b -> sdc/d
sdc/d -> sda/b

after sorting out the mdadm.conf file and about 4 reboots later, the devices came back on their own, and loopback trick was unnecessary.

All is happy again in mythtvland, so I guess it is time to break it again
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
SW RAID - Raid devices larger than Total Devices tkrin Linux - Server 2 12-10-2010 11:01 AM
USB devices disappear AndyCanfield Linux - Hardware 2 02-22-2008 07:07 PM
saving a raid+lvm partition on upgrade mort67 Linux - Software 3 10-02-2007 10:03 AM
Devices Disappear IpowerOne Fedora 1 12-05-2006 03:38 PM
Manually Created Devices Disappear - Slackware 10 Kernel 2.6.9 rovitotv Slackware 1 01-01-2005 04:53 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 12:28 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration