After recovered LVM from Too Small for Target, XFS does not mount

tecsun · 12-12-2020, 12:13 AM

Hello everyone. I've encountered the same issue as discussed in this original thread:
https://www.linuxquestions.org/quest...et-4175620516/

However, my filesystem cannot be mount. Appreciate if anyone could help.

Here's the description of my problem.

Quote:

Hi rknichols & JWolberg,

This thread is very helpful! I'm facing the exact problem, the server suddenly cannot see the volume when I was copying data over SMB into the volume. All disks health are good. Not using any thin-provisioning. I have no clue at all why it happened.

I've managed to recover my volume following the steps discussed above. However, after creating /dev/zzbigdev, I cannot mount it.

Code:

mount: Structure needs cleaning

I'm using:

Ubuntu 14.04
LVM2 on a mdadm RAID-6
The volume formatted with XFS

I googled and people are suggesting to do xfs_metadump to a file to test it. But whenever I run xfs_metadump, it either give an error (see below) or just hang there (the server still responding to putty but cannot login new session, it displays nothing after password is enter. GUI screen also halted). I've restarted the server multiple times but still cannot. Any clue?

Code:

root@xe5:~# xfs_metadump /dev/xe5-2tr6/lfcold /media/lfxe3/8TB-backup/recover/output.dump
xfs_metadump: cannot init perag data (117)
xfs_metadump: duplicate name for inode 0 in dir inode 6500231229

*** Error in `xfs_db': free(): invalid next size (normal): 0x0000000000ca8000 ***

berndbausch · 12-12-2020, 08:21 PM

The storage device contains a filesystem with inconsistent data that, it seems, can't be repaired automatically (when necessary, XFS performs automatic repairs at mount time). I guess that human intervention is required to decide whether some data can be deleted.

I would run xfs_repair and carefully analyze the output to see what kind of data would be overwritten to restore consistency. xfs_db could be used to recover data without mounting.

Also search the internet for "structure needs cleaning". I found this very detailed discussion of the risks and issues involved in xfs_repair: https://serverfault.com/questions/85...ing-xfs-errors.

tecsun · 12-13-2020, 08:15 AM

I managed to ddrescue to an image file, and run xfs_repair -L to the image file. However, the recovered files are mostly in lost+found, losing directory structure and filename. =( Too much manual work need to be done, so I'm going to try another approach.

I'm wondering if I grow the size of the /dev/md0 (my raid device，which claim by LVM that it is too small... it is kinda weird that it happened suddenly though)。 I will post the result here.

Thanks for the link.

syg00 · 12-13-2020, 04:51 PM

It's probably too late for that - the errors are in the filesystem structure itself. Adding space to the array is pretty safe, but I'm never keen to do recovery on the real data - work on another image. Create another vg (non-RAID is fine) and screw around with a copy of the data over there.

Note my sigline.