LVM, raid1 and scsi failure
I'm running a file server with FC6. OS is installed on hda. The data is stored on two raid-1 arrays of 2 scsi disks each, md0 and md1, combined into a logical volume VolGroup01-LogVol00. The volume is mounted on /data. I started getting IO errors on /data and checked mdadm --detail /dev/md0 and md1 and saw that one of the disks had failed and was removed. I planned on replacing it but needed to get it back up so did a mdadm re-add and that seemed to work, ...for a while.
The short story, now I can't read anyting on /data. My plan was to take down the LV, remove the bad disk from md0, install a new disk, rebuild the raid and recreate the volume. I unmounted the LV, but lvremove wil not work. I get a series of "read failure" errors on all the devices that make up the group.
Can I salvage any data here? Any suggestions on steps to take would be greatly appreciated.