cannot mount after failed reiserfsck --rebuild-tree
 

Anyone know of a way to get a FS to mount after a reiserfsck --rebuild-tree has failed on it? I am running Linux 5.2. I have read a bit about ddrescue, not sure if it can be run with a raid 10.

If "--rebuild-tree" has failed, you must be looking at some pretty serious file corruption. What messages does resierfsck show when you run it? Do you have any idea of what might have caused the corruption in the first place?

The most important question, and the one that more often than not is a "no", do you have any backups of the filesystem from before the corruption, or even before you ran "--rebuild-tree"?

Pass 1 (will try to insert 2275459 leaves):
Looking for allocable blocks .. finished
0%
The problem has occurred looks like a hardware problem (perhaps
memory). Send us the bug report only if the second run dies at
the same place with the same block number.

This is not the entire error log of course. I rolled the logs each time I ran them but since my management has hired a third party to take a look at this since I told them it is a data loss they have managed to blow up the OS and the host is no longer bootable. The rebuild tree was run about 7 times, dying at a different block each time. The hw was checked out. Initially there was a controller issue that was making 4 drives report as bad. After that was resolved we ran a fsck -check as our procedures state. That suggested a rebuild tree. We have no backup of the data. At some point management deemed it too costly to have duplicates if this data. Any experience with ddrescue on raid?

(It would help to list events verbosely and in a timeline because that avoids having to interprete things. Right now I can't see for instance if those 4 drives were actually OK or if it the RAID ran degraded after replacing the RAID controller and then which disks were b0rken.) The main problem with recovery at this point is the RAID was allowed to function and that destructive ops have altered disk contents. If I were to do a backup of this RAID I'd save all config details, label all disks physically, break the RAID, hook up each drive separately and dd_rescue each of them to a suitable separate disk pending recovery option research. But then again I'm no RAID expert.

thanks for the input

I had the exact same problem on my raid-1 array

Apparently, Reiserfsck cannot properly handle FS corruptions on raid arrays and gets confused by the loss of sync between disks (which sounds logical if you think about it).

Here is how I solved this :

Let /dev/md0 be the array, standing for /dev/sda4 and /dev/sdb5

do not run reiserfsck on the /dev/md0 array ; instead run it on the individual disks :

reiserfsck --logfile sda4.log --rebuild-tree /dev/sda4
reiserfsck --logfile sdb5.log --rebuild-tree /dev/sdb5

then determine from the stats output by reiserfsck which disk is less damaged (let it be /dev/sdb5), and mark the other one as failed - then remove and re-add it to trigger rebuilding:

mdadm /dev/md0 --fail /dev/sda4
mdadm /dev/md0 --remove /dev/sda4
mdadm /dev/md0 --add /dev/sda4

watch the progress as the array gets rebuilt :

cat /proc/mdstat

I wish I hadn't run with --rebuild-tree at all on md0 on the first place when my distro dropped me to an emergency shell; that was probably a mistake that just made things worse.

Pitty that reiserfsck didn't warn me about letting it hit a raid device.

I suspect running with just --fix-fixable on the individual disks would have been enough (and then rebuilding the array for sync) and I wouldn't have all those lost+found entries to deal with ;-/