mdadm reports no superblock trying to rebuild failed RAID 5
 

Pertinent system information:

Ubuntu Feisty Fawn
Software RAID 5 with 5 disks
/dev/sda1 60 MB (/)
/dev/sda3 120 MB (raid)
/dev/sda5 5 MB (swap)

/dev/sdb1 120MB (raid)
/dev/sdc1 120MB (raid)
/dev/sdd1 120MB (raid)
/dev/sde1 120MB (raid)

I had a power outage and somehow the partition table on my SDA drive was mucked up (seems like I lost no data on SDA1 lucky me!!!). I had to use a live cd of GPartEd to rebuild the partition on SDA3. I discovered the partition table was messed up because my RAID wouldn't mount.

next step is I did mdadm --add /dev/md0 /dev/sda3

and then if I do mdadm --detail /dev/md0 I get the following:

shad@server:~$ sudo mdadm --detail /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Sun May 6 22:58:40 2007
Raid Level : raid5
Device Size : 117218176 (111.79 GiB 120.03 GB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Fri Aug 17 21:17:41 2007
State : active, degraded, Not Started
Active Devices : 4
Working Devices : 5
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 64K

UUID : 106d7379:5e589387:1b3c85f6:630c4603
Events : 0.207719

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 33 1 active sync /dev/sdc1
2 8 49 2 active sync /dev/sdd1
3 8 65 3 active sync /dev/sde1
4 0 0 4 removed

5 8 3 - spare /dev/sda3

The drive listed as removed happened automatically and I assume it's the entry for the previous partition that went missing. What I've read on the forums is that when you issue the mdadm --add command that the raid is supposed to start rebuilding itself. You're supposed to be able to watch in by issuing cat /proc/mdstat and I get the following:

Personalities : [raid6] [raid5] [raid4]
md0 : inactive sdb1[0] sda3[5](S) sde1[3] sdd1[2] sdc1[1]
591906432 blocks

unused devices: <none>

Looks to me like it's not rebuilding. I don't know how to activate the spare. I also tried taking the RAID offline using mdadm --manage --stop
which gave me a perfectly normal looking "stopped the RAID" message but when I went to mdadm --assemble --scan I get

mdadm: WARNING /dev/mapper/sda3 and /dev/mapper/sdb1 appear to have very similar superblocks.
If they are really different, please --zero the superblock on one
If they are the same or overlap, please remove one from the
DEVICE list in mdadm.conf.

Otherwise I just get "the device is already active and you can't assemble it" Incidentally my mdadm.config file looks like this:

# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays
ARRAY /dev/md0 level=raid5 num-devices=5 UUID=106d7379:5e589387:1b3c85f6:630c4603

# This file was auto-generated on Wed, 27 Jun 2007 09:54:38 -0500
# by mkconf $Id: mkconf 261 2006-11-09 13:32:35Z madduck $

How can I get my raid to rebuild itself?

Sorry if this seems terse. I've been pounding on this problem for about 8 hours now and I can't seem to make any progress. Everyone wants to use raidtools which I've read is supposed to be replaced by mdadm.

Thanks in advance to anyone out there who takes pity on me and can solve this!