This is a new issue that I am having with a raid group. It started with the raid (raid 1) degrading... for some reason the physical drive would change device ids (sdf would become sdh). I could re-add the partition to the array and all would be fine for a week then it would happen again.

The last however is that the raid degraded... I rebooted the system and now I cannot bring the raid partitions back on line even in degraded form.

When I try to create:
mdadm /dev/md7 --assemble -u mdadm --assemble /dev/md7 /dev/sdf7 /dev/sdg7
mdadm: cannot open device /dev/sdf7: No such file or directory
mdadm: /dev/sdf7 has no superblock - assembly aborted

Smart says the drives are fine and the bios is reporting the drives as present. The drives are attached to the system via an adaptec raid card.

uname -a
Linux hostserver.home 2.6.27.41-170.2.117.fc10.x86_64 #1 SMP Thu Dec 10 10:36:29 EST 2009 x86_64 x86_64 x86_64 GNU/Linux


fdisk -l
Device Boot Start End #cyls #blocks Id System
/dev/sdf1 0+ 95295 95296- 765465088+ fd Linux raid autodetect
/dev/sdf2 95296 100779 5484 44050230 fd Linux raid autodetect
/dev/sdf3 100780 106263 5484 44050230 fd Linux raid autodetect
/dev/sdf4 106264 121600 15337 123194452+ 5 Extended
/dev/sdf5 106264+ 110572 4309- 34612011 fd Linux raid autodetect
/dev/sdf6 110573+ 114881 4309- 34612011 fd Linux raid autodetect
/dev/sdf7 114882+ 121600 6719- 53970336 fd Linux raid autodetect

Disk /dev/sdg: 121601 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End #cyls #blocks Id System
/dev/sdg1 0+ 95295 95296- 765465088+ fd Linux raid autodetect
/dev/sdg2 95296 100779 5484 44050230 fd Linux raid autodetect
/dev/sdg3 100780 106263 5484 44050230 fd Linux raid autodetect
/dev/sdg4 106264 121600 15337 123194452+ 5 Extended
/dev/sdg5 106264+ 110572 4309- 34612011 fd Linux raid autodetect
/dev/sdg6 110573+ 114881 4309- 34612011 fd Linux raid autodetect
/dev/sdg7 114882+ 121600 6719- 53970336 fd Linux raid autodetect

#cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1]
md4 : active raid1 sdf[0] sdg[1]
976762496 blocks [2/2] [UU]

md0 : active raid5 sdb2[0] sde2[3](S) sdd2[2] sdc2[1]
691228544 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md3 : active raid5 sdb1[0] sde1[3](S) sdd1[2] sdc1[1]
90188672 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

# cat /etc/mdadm.conf
ARRAY /dev/md3 level=raid5 num-devices=3 UUID=c2efe398:421bff8a:7f390ee3:00f99222
spares=1 devices=/dev/sde1,/dev/sdd1,/dev/sdc1,/dev/sdb1
ARRAY /dev/md0 level=raid5 num-devices=3 UUID=1c9cb177:369b13f9:3f2980bf:cca6d225
spares=1 devices=/dev/sde2,/dev/sdd2,/dev/sdc2,/dev/sdb2
ARRAY /dev/md4 level=raid1 num-devices=2 UUID=00995434:87aa912a:0c865975:ac14234a
devices=/dev/sdg,/dev/sdf
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=47fb4407:dc78c3c2:bce1d340:0455b8dd
spares=1 devices=/dev/sdg1,/dev/sdf1
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=f71c8f20:11314495:cd9daaa5:2c2c2293
spares=1 devices=/dev/sdg2,/dev/sdf2
ARRAY /dev/md4 level=raid1 num-devices=2 UUID=f0b0a4c5:3a43384b:96096ffb:f1f0340a
spares=1 devices=/dev/sdg3,/dev/sdf3
ARRAY /dev/md5 level=raid1 num-devices=2 UUID=6407e1ce:290122a9:c04d42e0:d30b21e4
spares=1 devices=/dev/sdg5,/dev/sdf5
ARRAY /dev/md6 level=raid1 num-devices=2 UUID=9ee26d49:263ee9f5:7d9073fa:c2bddeb3
spares=1 devices=/dev/sdg6,/dev/sdf6
ARRAY /dev/md7 level=raid1 num-devices=2 UUID=11473b68:246c464f:20f8c5e4:65039625
spares=1 devices=/dev/sdg7,/dev/sdf7

# dmidecode
# dmidecode 2.9
SMBIOS 2.5 present.
49 structures occupying 2021 bytes.
Table at 0x0009F400.

Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
Vendor: American Megatrends Inc.
Version: 080015
Release Date: 09/01/2008
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 1024 kB
Characteristics:
ISA is supported
PCI is supported
PNP is supported
APM is supported
BIOS is upgradeable
BIOS shadowing is allowed
ESCD support is available
Boot from CD is supported
Selectable boot is supported
BIOS ROM is socketed
EDD is supported
5.25"/1.2 MB floppy services are supported (int 13h)
3.5"/720 KB floppy services are supported (int 13h)
3.5"/2.88 MB floppy services are supported (int 13h)
Print screen service is supported (int 5h)
8042 keyboard services are supported (int 9h)
Serial services are supported (int 14h)
Printer services are supported (int 17h)
CGA/mono video services are supported (int 10h)
ACPI is supported
USB legacy is supported
LS-120 boot is supported
ATAPI Zip drive boot is supported
BIOS boot specification is supported
Targeted content distribution is supported
BIOS Revision: 8.15

Handle 0x0001, DMI type 1, 27 bytes
System Information
Manufacturer: To Be Filled By O.E.M.
Product Name: To Be Filled By O.E.M.
Version: To Be Filled By O.E.M.
Serial Number: To Be Filled By O.E.M.
UUID: 087EE2C1-1879-DD11-8A76-52CC977B599B
Wake-up Type: Power Switch
SKU Number: To Be Filled By O.E.M.
Family: To Be Filled By O.E.M.

Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: XFX
Product Name: MI-A78S-8209
Version: Ver1.1
Serial Number: To be filled by O.E.M.
Asset Tag: To Be Filled By O.E.M.
Features:
Board is a hosting board
Board is replaceable
Location In Chassis: To Be Filled By O.E.M.
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0

Handle 0x0003, DMI type 3, 21 bytes
Chassis Information
Manufacturer: To Be Filled By O.E.M.
Type: Desktop
Lock: Not Present
Version: To Be Filled By O.E.M.
Serial Number: To Be Filled By O.E.M.
Asset Tag: To Be Filled By O.E.M.
Boot-up State: Safe
Power Supply State: Safe
Thermal State: Safe
Security Status: None
OEM Information: 0x00000000
Height: Unspecified
Number Of Power Cords: 1
Contained Elements: 0

Handle 0x0004, DMI type 4, 40 bytes
Processor Information
Socket Designation: CPU 1
Type: Central Processor
Family: Athlon 64
Manufacturer: AMD
ID: F3 0F 05 00 FF FB 8B 07
Signature: Family 15, Model 95, Stepping 3
Flags:
FPU (Floating-point unit on-chip)
VME (Virtual mode extension)
DE (Debugging extension)
PSE (Page size extension)
TSC (Time stamp counter)
MSR (Model specific registers)
PAE (Physical address extension)
MCE (Machine check exception)
CX8 (CMPXCHG8 instruction supported)
APIC (On-chip APIC hardware supported)
SEP (Fast system call)
MTRR (Memory type range registers)
PGE (Page global enable)
MCA (Machine check architecture)
CMOV (Conditional move instruction supported)
PAT (Page attribute table)
PSE-36 (36-bit page size extension)
CLFSH (CLFLUSH instruction supported)
MMX (MMX technology supported)
FXSR (Fast floating-point save and restore)
SSE (Streaming SIMD extensions)
SSE2 (Streaming SIMD extensions 2)
Version: AMD Athlon(tm) 64 Processor 3500+
Voltage: 1.5 V
External Clock: 200 MHz
Max Speed: 2200 MHz
Current Speed: 2212 MHz
Status: Populated, Enabled
Upgrade: Other
L1 Cache Handle: 0x0005
L2 Cache Handle: 0x0006
L3 Cache Handle: 0x0007
Serial Number: To Be Filled By O.E.M.
Asset Tag: To Be Filled By O.E.M.
Part Number: To Be Filled By O.E.M.
Core Count: 1
Core Enabled: 1
Characteristics:
64-bit capable

Handle 0x0005, DMI type 7, 19 bytes
Cache Information
Socket Designation: L1-Cache
Configuration: Enabled, Not Socketed, Level 1
Operational Mode: Varies With Memory Address
Location: Internal
Installed Size: 128 KB
Maximum Size: 128 KB
Supported SRAM Types:
Pipeline Burst
Installed SRAM Type: Pipeline Burst
Speed: Unknown
Error Correction Type: Single-bit ECC
System Type: Data
Associativity: 4-way Set-associative

Handle 0x0006, DMI type 7, 19 bytes
Cache Information
Socket Designation: L2-Cache
Configuration: Enabled, Not Socketed, Level 2
Operational Mode: Varies With Memory Address
Location: Internal
Installed Size: 512 KB
Maximum Size: 512 KB
Supported SRAM Types:
Pipeline Burst
Installed SRAM Type: Pipeline Burst
Speed: Unknown
Error Correction Type: Single-bit ECC
System Type: Unified
Associativity: 4-way Set-associative

Handle 0x0007, DMI type 7, 19 bytes
Cache Information
Socket Designation: L3-Cache
Configuration: Disabled, Not Socketed, Level 3
Operational Mode: Unknown
Location: Internal
Installed Size: 0 KB
Maximum Size: 0 KB
Supported SRAM Types:
Unknown
Installed SRAM Type: Unknown
Speed: Unknown
Error Correction Type: Unknown
System Type: Unknown
Associativity: Unknown

Handle 0x0008, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J1A1
Internal Connector Type: None
External Reference Designator: PS2Mouse
External Connector Type: PS/2
Port Type: Mouse Port

Handle 0x0009, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J1A1
Internal Connector Type: None
External Reference Designator: Keyboard
External Connector Type: PS/2
Port Type: Keyboard Port

Handle 0x000A, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2A2
Internal Connector Type: None
External Reference Designator: USB1
External Connector Type: Access Bus (USB)
Port Type: USB

Handle 0x000B, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2A2
Internal Connector Type: None
External Reference Designator: USB2
External Connector Type: Access Bus (USB)
Port Type: USB

Handle 0x000C, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J4A1
Internal Connector Type: None
External Reference Designator: LPT 1
External Connector Type: DB-25 male
Port Type: Parallel Port ECP/EPP

Handle 0x000D, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2A1
Internal Connector Type: None
External Reference Designator: COM A
External Connector Type: DB-9 male
Port Type: Serial Port 16550A Compatible

Handle 0x000E, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6A1
Internal Connector Type: None
External Reference Designator: Audio Mic In
External Connector Type: Mini Jack (headphones)
Port Type: Audio Port

Handle 0x000F, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6A1
Internal Connector Type: None
External Reference Designator: Audio Line In
External Connector Type: Mini Jack (headphones)
Port Type: Audio Port

Handle 0x0010, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6B1 - AUX IN
Internal Connector Type: On Board Sound Input From CD-ROM
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Audio Port

Handle 0x0011, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6B2 - CDIN
Internal Connector Type: On Board Sound Input From CD-ROM
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Audio Port

Handle 0x0012, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6J2 - PRI IDE
Internal Connector Type: On Board IDE
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x0013, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6J1 - SEC IDE
Internal Connector Type: On Board IDE
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x0014, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J4J1 - FLOPPY
Internal Connector Type: On Board Floppy
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x0015, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9H1 - FRONT PNL
Internal Connector Type: 9 Pin Dual Inline (pin 10 cut)
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x0016, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J1B1 - CHASSIS REAR FAN
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x0017, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2F1 - CPU FAN
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x0018, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J8B4 - FRONT FAN
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x0019, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9G2 - FNT USB
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x001A, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6C3 - FP AUD
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x001B, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9G1 - CONFIG
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x001C, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J8C1 - SCSI LED
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x001D, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9J2 - INTRUDER
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x001E, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9G4 - ITP
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x001F, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2H1 - MAIN POWER
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other

Handle 0x0020, DMI type 9, 13 bytes
System Slot Information
Designation: AGP
Type: 32-bit AGP 4x
Current Usage: In Use
Length: Short
ID: 0
Characteristics:
3.3 V is provided
Opening is shared
PME signal is supported

Handle 0x0021, DMI type 9, 13 bytes
System Slot Information
Designation: PCI1
Type: 32-bit PCI
Current Usage: Available
Length: Short
ID: 1
Characteristics:
3.3 V is provided
Opening is shared
PME signal is supported

Handle 0x0022, DMI type 10, 6 bytes
On Board Device Information
Type: Video
Status: Enabled
Description: To Be Filled By O.E.M.

Handle 0x0023, DMI type 13, 22 bytes
BIOS Language Information
Installable Languages: 1
en|US|iso8859-1
Currently Installed Language: en|US|iso8859-1

Handle 0x0024, DMI type 15, 55 bytes
System Event Log
Area Length: 1008 bytes
Header Start Offset: 0x0810
Data Start Offset: 0x0810
Access Method: General-purpose non-volatile data functions
Access Address: 0x0001
Status: Valid, Not Full
Change Token: 0x00000000
Header Format: No Header
Supported Log Type Descriptors: 9
Descriptor 1: Single-bit ECC memory error
Data Format 1: Multiple-event handle
Descriptor 2: Multi-bit ECC memory error
Data Format 2: Multiple-event handle
Descriptor 3: Parity memory error
Data Format 3: Multiple-event
Descriptor 4: I/O channel block
Data Format 4: Multiple-event
Descriptor 5: POST error
Data Format 5: POST results bitmap
Descriptor 6: PCI parity error
Data Format 6: Multiple-event handle
Descriptor 7: PCI system error
Data Format 7: Multiple-event handle
Descriptor 8: System limit exceeded
Data Format 8: Multiple-event system management
Descriptor 9: OEM-specific
Data Format 9: POST results bitmap

Handle 0x0025, DMI type 16, 15 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: None
Maximum Capacity: 8 GB
Error Information Handle: Not Provided
Number Of Devices: 4

Handle 0x0026, DMI type 19, 15 bytes
Memory Array Mapped Address
Starting Address: 0x00000000000
Ending Address: 0x0021FFFFFFF
Range Size: 8704 MB
Physical Array Handle: 0x0025
Partition Width: 0

Handle 0x0027, DMI type 17, 27 bytes
Memory Device
Array Handle: 0x0025
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 72 bits
Size: 2048 MB
Form Factor: DIMM
Set: None
Locator: DIMM0
Bank Locator: BANK0
Type: DDR2
Type Detail: Synchronous
Speed: 333 MHz (3.0 ns)
Manufacturer: Manufacturer00
Serial Number: 00000000
Asset Tag: AssetTagNum0
Part Number: OCZ2N800SR2G

Handle 0x0028, DMI type 20, 19 bytes
Memory Device Mapped Address
Starting Address: 0x00000000000
Ending Address: 0x0007FFFFFFF
Range Size: 2 GB
Physical Device Handle: 0x0027
Memory Array Mapped Address Handle: 0x0026
Partition Row Position: 1

Handle 0x0029, DMI type 17, 27 bytes
Memory Device
Array Handle: 0x0025
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 72 bits
Size: 2048 MB
Form Factor: DIMM
Set: None
Locator: DIMM1
Bank Locator: BANK1
Type: DDR2
Type Detail: Synchronous
Speed: 333 MHz (3.0 ns)
Manufacturer: Manufacturer01
Serial Number: 00000000
Asset Tag: AssetTagNum1
Part Number: OCZ2N800SR2G

Handle 0x002A, DMI type 20, 19 bytes
Memory Device Mapped Address
Starting Address: 0x00080000000
Ending Address: 0x000FFFFFFFF
Range Size: 2 GB
Physical Device Handle: 0x0029
Memory Array Mapped Address Handle: 0x0026
Partition Row Position: 1

Handle 0x002B, DMI type 17, 27 bytes
Memory Device
Array Handle: 0x0025
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 72 bits
Size: 2048 MB
Form Factor: DIMM
Set: None
Locator: DIMM2
Bank Locator: BANK2
Type: DDR2
Type Detail: Synchronous
Speed: 333 MHz (3.0 ns)
Manufacturer: Corsair
Serial Number: 00000000
Asset Tag: AssetTagNum2
Part Number: CM2X2048-6400C5

Handle 0x002C, DMI type 20, 19 bytes
Memory Device Mapped Address
Starting Address: 0x00100000000
Ending Address: 0x0017FFFFFFF
Range Size: 2 GB
Physical Device Handle: 0x002B
Memory Array Mapped Address Handle: 0x0026
Partition Row Position: 1

Handle 0x002D, DMI type 17, 27 bytes
Memory Device
Array Handle: 0x0025
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 72 bits
Size: 2048 MB
Form Factor: DIMM
Set: None
Locator: DIMM3
Bank Locator: BANK3
Type: DDR2
Type Detail: Synchronous
Speed: 333 MHz (3.0 ns)
Manufacturer: Corsair
Serial Number: 00000000
Asset Tag: AssetTagNum3
Part Number: CM2X2048-6400C5

Handle 0x002E, DMI type 20, 19 bytes
Memory Device Mapped Address
Starting Address: 0x00180000000
Ending Address: 0x001FFFFFFFF
Range Size: 2 GB
Physical Device Handle: 0x002D
Memory Array Mapped Address Handle: 0x0026
Partition Row Position: 1

Handle 0x002F, DMI type 32, 20 bytes
System Boot Information
Status: No errors detected

Handle 0x0030, DMI type 127, 4 bytes
End Of Table