mdadm and raid5
Hi:
I'm new/old at linux. The good thing about Linux is, once it's setup there is not much the user have to do to maintain it. The problem with it is once it's time to redo the system you've forgotten what you originally did. As in my case, I'm trying to migrate all my machines into one unit. Anyways: I have an issue with my system. I'm running Debian 3.1.5 kernel 2.6.8.3.x and using mdadm for my raid setup. I have various drives setup /dev/hda /dev/hdb /dev/hdc /dev/sda /dev/sdb /dev/sdc /dev/sdd The most important are the /dev/sd[abcd] drives. I'm trying to setup RAID5 on these drives. I have read and followed various instructions on this and other forums and have already configured/rebuild multiple times only to end up with the same result. The problem is mdadm doesn't appear to be reading /etc/mdadm/mdadm.conf on boot, and because it doesn't get read on boot I always get this message "mdadm: no devices found for /dev/md0". There are also a bunch of other messages that followed, but suffice to say if mdadm can't find any devices belonging to /dev/md0 array it will error out and will not create the array. The reason I believe mdadm is not reading mdadm.conf is because I can logon and execute the command to assemble the array: #mdadm -A /dev/md0 mdadm: /dev/md0 has been started with 4 drives. I also test /etc/fstab by: #mount -a #df -h shows the mounted file system /dev/md0 101G 33M 96G 1% /raid5 This is a pain especially if I can't rely on the system to complete it's boot-up without pausing at "mdadm: no devices found for /dev/md0" , give root password for maintenance or CONTROL-D to continue. Here's my /etc/mdadm/mdadm.conf DEVICE partitions DEVICE /dev/sda /dev/sdb /dev/sdc /dev/sdd ARRAY /dev/md0 level=raid5 num-devices=4 UUID=23ff6bb9:eb8d753f:2682b942:7ae393e7 devices=/dev/sda,/dev/sdb,/dev/sdc,/dev/sdd It doesn't matter if I remove the first line "DEVICE partitions" it still pauses Here's my /etc/fstab # /etc/fstab: static file system information. # # <file system> <mount point> <type> <options> <dump> <pass> proc /proc proc defaults 0 0 /dev/hda3 / ext3 defaults,errors=remount-ro 0 1 /dev/hda1 /boot ext3 defaults 0 2 /dev/hdc /done_downloading ext3 defaults 0 2 /dev/hdb1 /downloading ext3 defaults 0 2 /dev/md0 /raid5 ext3 defaults 0 2 /dev/hda2 none swap sw 0 0 /dev/hdd /media/cdrom0 iso9660 ro,user,noauto 0 0 /dev/fd0 /media/floppy0 auto rw,user,noauto 0 0 I've also checked /etc/rd2.d to make sure the link to mdadm is proper... I've also checked /etc/init.d mdadm and mdadm-raid file, from what I can tell they look proper... Is there anything else I'm missing? If some linux/mdadm guru or someone experience them me (and I know there are plenty out there) who can give me some info on this problem, I will be extremely greatful. I know it's a long read but please understand my delimma. :( |
Try changing the partition type to 'FD' using a partition editor. This tells the kernel to try and puzzle out the RAID type ad participants. This way you don't even NEED mdadm.conf. I never use it.
Also, make sure your initrd file (in /boot) has RAID5 support if your system partition is RAID5. |
type FD
Quote:
Thanks for the response. I did change the partition type to FD. Through my debugging I tried using DRIVE and partitions: /dev/sda or /dev/sda1... Neither method worked. My system does have RAID5 support because I selected it during Debian installation. This is why I don't understand this issue I'm experiencing. This is a base install of Debian 3.1.5, with RAID-5 for the 4 SCSI drives. Yet everytime I reboot I get the same message "mdadm: no devices found for /dev/md0" Any other hints I can use? |
If you changed the partition type AFTER mucking with the mdadm.conf file, I suggest you rename the mdadm.conf file to mdadm.bkup
Is there any information you are trying to preserve on the target drives? Have you read through the Software RAID HowTo? (one of the best HowTo's out there) Could you post the following in separate replies: mdadm -E /dev/sda1 mdadm -D /dev/md0 fdisk -l cat /proc/mdstat grep kernel /boot/grub/grub.conf pax -z < /boot/initrd*$(uname -r)* | grep kernel That should do it for me to figure out what's going on. |
mdadm -E /dev/sda1
Quote:
Though I've still got the same error message, I'm using the below method to somewhat automate the (dirty) Raid mounting. I still have to press CTRL+D to bypass the error message. ############################################################################################## !/bin/sh echo echo Starting RAID-5 mdadm -A /dev/md0 mount -a echo Done with Manual Raid setup echo ################################################## Save it at /etc/init.d/mdadm.mine Make it executible "chmod +x /etc/init.d/mdadm.mine" Add it to startup menu "update-rc.d /etc/init.d/mdadm.mine defaults" ################################################################################################ Here's the result from mdadm -E /dev/sda1 mdadm -E /dev/sda1 /dev/sda1: Magic : a92b4efc Version : 00.90.00 UUID : 44e75c85:91fc9a9f:34b2c1b4:4834c282 Creation Time : Tue Apr 10 18:26:45 2007 Raid Level : raid5 Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Update Time : Wed Apr 11 20:08:33 2007 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Checksum : 8b54834b - correct Events : 0.82 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 0 8 1 0 active sync /dev/sda1 0 0 8 1 0 active sync /dev/sda1 1 1 8 17 1 active sync /dev/sdb1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 49 3 active sync /dev/sdd1 |
mdadm -D /dev/md0
mdadm -D /dev/md0
/dev/md0: Version : 00.90.01 Creation Time : Tue Apr 10 18:26:45 2007 Raid Level : raid5 Array Size : 107581056 (102.60 GiB 110.16 GB) Device Size : 35860352 (34.20 GiB 36.72 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Wed Apr 11 20:08:33 2007 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K UUID : 44e75c85:91fc9a9f:34b2c1b4:4834c282 Events : 0.82 Number Major Minor RaidDevice State 0 8 1 0 active sync /dev/sda1 1 8 17 1 active sync /dev/sdb1 2 8 33 2 active sync /dev/sdc1 3 8 49 3 active sync /dev/sdd1 |
fdisk -l
fdisk -l
Disk /dev/hda: 80.0 GB, 80026361856 bytes 255 heads, 63 sectors/track, 9729 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/hda1 * 1 12 96358+ 83 Linux /dev/hda2 13 377 2931862+ 82 Linux swap / Solaris /dev/hda3 378 9729 75119940 83 Linux Disk /dev/hdb: 82.3 GB, 82348277760 bytes 255 heads, 63 sectors/track, 10011 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/hdb1 1 10011 80413326 83 Linux Disk /dev/hdc: 200.0 GB, 200049647616 bytes 255 heads, 63 sectors/track, 24321 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/hdc doesn't contain a valid partition table Disk /dev/md0: 110.1 GB, 110163001344 bytes 2 heads, 4 sectors/track, 26895264 cylinders Units = cylinders of 8 * 512 = 4096 bytes Device Boot Start End Blocks Id System /dev/md0p1 1 26895264 107581054 83 Linux Disk /dev/sda: 36.7 GB, 36746412032 bytes 64 heads, 32 sectors/track, 35044 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/sda1 1 35044 35885040 fd Linux raid autodetect Disk /dev/sdb: 36.7 GB, 36746412032 bytes 64 heads, 32 sectors/track, 35044 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 35044 35885040 fd Linux raid autodetect Disk /dev/sdc: 36.7 GB, 36778545152 bytes 64 heads, 32 sectors/track, 35074 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/sdc1 1 35074 35915760 fd Linux raid autodetect Disk /dev/sdd: 36.7 GB, 36722061312 bytes 64 heads, 32 sectors/track, 35020 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/sdd1 1 35020 35860464 fd Linux raid autodetect kaos:/etc/init.d# |
cat /proc/mdstat
cat /proc/mdstat
Personalities : [raid5] md0 : active raid5 sda1[0] sdd1[3] sdc1[2] sdb1[1] 107581056 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU] unused devices: <none> I have no /boot/grub/grub.conf Pax is not installed on this machine. Can be installed if need be. |
mdadm
This is a fresh build of Debian 3.1.5. I have a drive /dev/hdc which has data on it, though I could careless about it. There is no data on the drives I'm trying to create the RAID. Wouldn't matter anyways since my effort have wiped and rewiped all data on them.
|
Did update-initramfs get run when you installed mdadm? Have you tried running it to see if it fixes it? I run a dmraid and I'm pretty sure that it needs to be run for that.
Added: Not that it strictly matters for this problem, but you should really consider upgrading to Etch. It's now officially stable. |
Switching between whole disk RAID devices, disk partition RAID devices, and partitions of RAID devices as RAID devices (/dev/md0p1) has me, you, the kernel, and everyone on this board confused. :)
Unmount everything that has to do with the SCSI drives. Code:
umount /dev/md0p1 Code:
mdadm /dev/md0p1 --stop Code:
cat /proc/mdstat (This writes all zeros to the beginning of each scsi drive. Be careful.) Code:
for i in a b c d; do Code:
for i in a b c d; do Code:
mdadm --create /dev/md0 --level=RAID5 --raid-devices=4 /dev/sd[abcd]1 Code:
mkfs.ext3 /dev/md0 Code:
tune2fs -c0 -m0 -i0 /dev/md0 Code:
mkdir -p /raid Code:
/dev/md0 /raid ext3 defaults 0 5 Code:
mount /raid |
Did I read the OP wrong? I thought the problem was only that it was booting without the array assembled, not that he couldn't assemble the array once the system is booted.
|
He wants the kernel autodetection to work, which is facilitated by partition id FD. But while trying to figure out why it wouldn't work he delved into the black art of RAID partitioning (/dev/md0p1) which now has things confused.
|
Oh. Gotcha.
|
Raid
Quakerboy: even though you though upgrading to Etch is unrelated. However some of the most bizzar suggestions are useful and in this case solved my problem. So thumbs up for your suggestion. I've spent the last couple days installing and configuring Etch (multiple times to test various builds). With this new version I can build raids to my harts content. /dev/hda or /dev/sda it doesn't matter, RAID 0, 1, 5 what ever...
This project started with a download of the most current version of Debian. At the time it was 3.1.5, I'm glad 4.0r0 solved my issues. Thank You for the suggestion. Dgar: I'm not leaving you out of this either. Thank you for all your troubleshooting effort. I got some valuable information on the process. You have been helping me from the start of this delima. Though as an Engineer I really would have liked to know exactly why the RAID would not stick on reboot... Oh well, on a bright note, it's working now!!! :D Now to the main part of the project "XEN", any of you's have experience on this topic? Moderators: This is the best forum I'm happily to be a part of. The number of members is enormous, topics are constantly posted and answered. Thank you for a great forum. |
All times are GMT -5. The time now is 12:47 AM. |