LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Broken LUKS LVM Debian install (https://www.linuxquestions.org/questions/linux-newbie-8/broken-luks-lvm-debian-install-4175692737/)

JClant 03-27-2021 11:51 AM

Broken LUKS LVM Debian install
 
Hi Everybody! Hope you are having a good weekend :)

I had a rough one so far.. As an intro, I had Debian install running on LUKS encrypted LVM.

This morning, I wanted to install OS on external disk to be used on another host, but mistakenly I chose the wrong disk... Few seconds in I noticed that I have chosen the wrong one, so right away I stopped the process, but partition table already got written.

To recover I booted system rescue CD, and with the help of testdisk I recovered the partitions. However, I still couldn't boot my old setup, so I started looking for a way to recover the files at least. I ran rescue disk again, and I did the following:

Code:

# fdisk -l

Disk /dev/nvme0n1: 232.89 GiB, 250059350016 bytes, 488397168 sectors
Disk model: Samsung SSD 980 PRO 250GB             
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 6653A4F0-8EEA-11EB-8B8C-1C697A6E9DFB

Device          Start      End  Sectors  Size Type
/dev/nvme0n1p1 1050624  2050047    999424  488M Linux filesystem
/dev/nvme0n1p2 2050048 446297482 444247435 211.8G Linux filesystem

Code:

# cryptsetup open /dev/nvme0n1p2 recoveryx --type luks

WARNING: Locking directory /run/cryptsetup is missing!
Enter passphrase for /dev/nvme0n1p2:

Code:

# ls /dev/mapper/

control  recoveryx

Code:

# vgscan

  WARNING: Device /dev/mapper/recoveryx has size of 444214667 sectors which is smaller than corresponding PV size of 486313984 sectors. Was device resized?
  WARNING: One or more devices used as PVs in VG worker1-vg have changed sizes.
  Found volume group "worker1-vg" using metadata type lvm2

Code:

# lvdisplay

  WARNING: Device /dev/mapper/recoveryx has size of 444214667 sectors which is smaller than corresponding PV size of 486313984 sectors. Was device resized?
  WARNING: One or more devices used as PVs in VG worker1-vg have changed sizes.
  --- Logical volume ---
  LV Path                /dev/worker1-vg/root
  LV Name                root
  VG Name                worker1-vg
  LV UUID                ZnO9e1-PRbO-TuDA-mfiV-uQXw-dzNT-OWunvP
  LV Write Access        read/write
  LV Creation host, time worker1, 2021-01-02 13:28:48 +0000
  LV Status              NOT available
  LV Size                230.93 GiB
  Current LE            59119
  Segments              1
  Allocation            inherit
  Read ahead sectors    auto
 
  --- Logical volume ---
  LV Path                /dev/worker1-vg/swap_1
  LV Name                swap_1
  VG Name                worker1-vg
  LV UUID                T4UxqE-swKh-ivwW-JGwF-7vNU-eax7-hiy7dc
  LV Write Access        read/write
  LV Creation host, time worker1, 2021-01-02 13:28:49 +0000
  LV Status              NOT available
  LV Size                980.00 MiB
  Current LE            245
  Segments              1
  Allocation            inherit
  Read ahead sectors    auto

Code:

# vgchange -v -ay worker1-vg

  WARNING: Device /dev/mapper/recoveryx has size of 444214667 sectors which is smaller than corresponding PV size of 486313984 sectors. Was device resized?
  WARNING: One or more devices used as PVs in VG worker1-vg have changed sizes.
  Activating logical volume worker1-vg/root.
  activation/volume_list configuration setting not defined: Checking only host tags for worker1-vg/root.
  Creating worker1--vg-root
  Loading table for worker1--vg-root (254:1).
  device-mapper: reload ioctl on  (254:1) failed: Invalid argument
  Removing worker1--vg-root (254:1)
  Activating logical volume worker1-vg/swap_1.
  activation/volume_list configuration setting not defined: Checking only host tags for worker1-vg/swap_1.
  Creating worker1--vg-swap_1
  Loading table for worker1--vg-swap_1 (254:1).
  device-mapper: reload ioctl on  (254:1) failed: Invalid argument
  Removing worker1--vg-swap_1 (254:1)
  Activated 0 logical volumes in volume group worker1-vg.
  0 logical volume(s) in volume group "worker1-vg" now active

Luckily the VG is detected, and I still have the LUKS header, but as can be seen in the above code blocks, activation fails due to "device-mapper: reload ioctl on (254:1) failed: Invalid argument"

Any ideas how to solve this issue?

rknichols 03-27-2021 10:18 PM

The LUKS header contains no information about the size of the volume, and thus testdisk cannot determine the correct size of the partition. You need to use a partitioning tool to extend partition 2 to the end of the disk. Just do not use any tool that attempts to reformat the partition. That would overwrite your LUKS header and make your data forever unrecoverable. To be safe, you should first use "cryptsetup luksHeaderBackup ..." to back up the LUKS header to a file on a separate device. See the cryptsetup manpage for details.

JClant 03-30-2021 08:44 PM

Thanks for the hint rknichols!

I extended root fs to end of disk, which solved the mount issue!!

However it took me a while to reconstruct the boot partition.. I chrooted root + boot partitions, and ran update-initramfs, but following boot failed with:

Code:

/dev/mapper/worker1--vg-root does not exist. Dropping to a shell!
A tip for anyone facing similar problem, when decrypting fs with cryptsetup luksOpen, device name has to correspond with root fs definition in /etc/crypttab before updating initramfs.. This finally let me completely recover the system :)

Thanks all again, and wish you good health and all the best in these turbulent times..

Cheers! 🍺🍺


All times are GMT -5. The time now is 03:45 PM.