Activating encrypted LV from previous Linux installation

rknichols · 01-01-2016, 02:19 PM

I shouldn't matter whether you plug that PV into a new system or the old one. It should still look like a PV with part of its volume group missing.

I just set up a volume group with one of its 2 PVs on a USB flash drive, shut the volume group down with "vgchange -an testvg1", and plugged that flash drive into a different system. Running pvs there shows that device and another "unknown device" as making up a VG with that name. I can activate that partial VG just fine. I don't know why you're seeing something different. Did you perhaps take some action to remove the VG when shutting down the old system?

linuxbawks · 01-01-2016, 03:52 PM

If we consider the VG, I initially used the same VG name for both the setups on pv0 and pv1. I set up an encrypted container of two logical volumes on pv0. Then I created another encrypted container of one logical volume on pv1 and then used vgextend to add the container on pv1 to the initial VG. Thus, one volume group spanning two physical volumes and three logical volumes.

One other thing I don't understand is why isn't the container on pv1 identifying itself with some sort of encryption using Luks or otherwise? Because it was encrypted.

rknichols · 01-01-2016, 05:05 PM

What I fear has happened is that something has set up /dev/sda3 as an LVM PV. That could have been done by either pvcreate or vgcreate since the latter will perform an default pvcreate if told to use a device that is not already set up as a PV. If that's the case, I'm sorry to say your data is lost forever unless you happen to have a backup of that LUKS header. With that header overwritten, data essential to generating the master key has been destroyed and there is no recovery.

You can try using hexedit (or another hex editor) on the whole drive and search for a 512-byte sector beginning with the ASCII characters "LUKS" followed by the hex bytes 0xBA and 0xBE. (That whole thing is the hex sequence "4c 55 4b 53 ba be".) If you find that in a reasonable location (somewhere near the beginning of the disk), then there is a chance for recovery. Without that, it's "Game Over".

rknichols · 01-01-2016, 05:20 PM

Quote:

Originally Posted by linuxbawks

If we consider the VG, I initially used the same VG name for both the setups on pv0 and pv1. I set up an encrypted container of two logical volumes on pv0. Then I created another encrypted container of one logical volume on pv1 and then used vgextend to add the container on pv1 to the initial VG. Thus, one volume group spanning two physical volumes and three logical volumes.

Your language here is extremely confusing and inconsistent with the "device" hint in that LVM backup file.

Code:

device = "/dev/mapper/sda3_crypt"

That suggests that the partition was first decrypted to create the /dev/mapper/sda3_crypt device (Else, where did that device come from?), and then the PV was set up inside that device.

When you say "I created another encrypted container of one logical volume on pv1", that is saying you had a PV and created an encrypted container inside it. No, you didn't, and that sort of inattention to detail can be fatal to encrypted data.

linuxbawks · 01-03-2016, 02:09 PM

I apologize for any confusion caused. I've been trying a few different things over the course of the thread and have mixed the communications up a little.

1. After initially trying to open the device with cryptsetup luksOpen it reported not a Luks device.
Though, I did specifically LUKS encrypt the entire partition first (/dev/sda3) and then set up one logical volume inside it in ext4 format.

2. After trying to set this encrypted partition up again and it having reported that it's not a LUKS device, I proceeded to recreated the encrypted partition but this time without using the luksFormat option in cryptsetup. I then proceeded to recreate the single logical volume inside it consisting of the entire container.

The data in the encrypted volume is still there. But the partition header is now showing this:

Code:

# sudo hexdump -C -n 8192 /dev/sda3
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  4c 41 42 45 4c 4f 4e 45  01 00 00 00 00 00 00 00  |LABELONE........|
00000210  bc f8 27 07 20 00 00 00  4c 56 4d 32 20 30 30 31  |..'. ...LVM2 001|
00000220  61 4a 6a 4e 36 43 4f 32  78 44 50 4c 70 66 50 32  |aJjN6CO2xDPLpfP2|
00000230  56 43 53 76 48 30 41 61  59 39 72 59 54 4a 6d 49  |VCSvH0AaY9rYTJmI|
00000240  00 60 40 41 62 00 00 00  00 00 10 00 00 00 00 00  |.`@Ab...........|
00000250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000260  00 00 00 00 00 00 00 00  00 10 00 00 00 00 00 00  |................|
00000270  00 f0 0f 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000280  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00000290  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  41 80 16 a5 20 4c 56 4d  32 20 78 5b 35 41 25 72  |A... LVM2 x[5Ar|
00001010  30 4e 2a 3e 01 00 00 00  00 10 00 00 00 00 00 00  |0N*>............|
00001020  00 f0 0f 00 00 00 00 00  00 02 00 00 00 00 00 00  |................|
00001030  cb 02 00 00 00 00 00 00  5a f0 84 74 00 00 00 00  |........Z..t....|
00001040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001200  73 64 61 33 5f 63 72 79  70 74 20 7b 0a 69 64 20  |sda3_crypt {.id |
00001210  3d 20 22 6f 63 32 53 33  61 2d 52 4f 55 52 2d 47  |= "oc2S3a-ROUR-G|
00001220  4c 59 45 2d 31 65 43 44  2d 38 79 45 62 2d 67 67  |LYE-1eCD-8yEb-gg|
00001230  49 79 2d 50 71 58 30 59  76 22 0a 73 65 71 6e 6f  |Iy-PqX0Yv".seqno|
00001240  20 3d 20 31 0a 66 6f 72  6d 61 74 20 3d 20 22 6c  | = 1.format = "l|
00001250  76 6d 32 22 0a 73 74 61  74 75 73 20 3d 20 5b 22  |vm2".status = ["|
00001260  52 45 53 49 5a 45 41 42  4c 45 22 2c 20 22 52 45  |RESIZEABLE", "RE|
00001270  41 44 22 2c 20 22 57 52  49 54 45 22 5d 0a 66 6c  |AD", "WRITE"].fl|
00001280  61 67 73 20 3d 20 5b 5d  0a 65 78 74 65 6e 74 5f  |ags = [].extent_|
00001290  73 69 7a 65 20 3d 20 38  31 39 32 0a 6d 61 78 5f  |size = 8192.max_|
000012a0  6c 76 20 3d 20 30 0a 6d  61 78 5f 70 76 20 3d 20  |lv = 0.max_pv = |
000012b0  30 0a 6d 65 74 61 64 61  74 61 5f 63 6f 70 69 65  |0.metadata_copie|
000012c0  73 20 3d 20 30 0a 0a 70  68 79 73 69 63 61 6c 5f  |s = 0..physical_|
000012d0  76 6f 6c 75 6d 65 73 20  7b 0a 0a 70 76 30 20 7b  |volumes {..pv0 {|
000012e0  0a 69 64 20 3d 20 22 61  4a 6a 4e 36 43 2d 4f 32  |.id = "aJjN6C-O2|
000012f0  78 44 2d 50 4c 70 66 2d  50 32 56 43 2d 53 76 48  |xD-PLpf-P2VC-SvH|
00001300  30 2d 41 61 59 39 2d 72  59 54 4a 6d 49 22 0a 64  |0-AaY9-rYTJmI".d|
00001310  65 76 69 63 65 20 3d 20  22 2f 64 65 76 2f 73 64  |evice = "/dev/sd|
00001320  61 33 22 0a 0a 73 74 61  74 75 73 20 3d 20 5b 22  |a3"..status = ["|
00001330  41 4c 4c 4f 43 41 54 41  42 4c 45 22 5d 0a 66 6c  |ALLOCATABLE"].fl|
00001340  61 67 73 20 3d 20 5b 5d  0a 64 65 76 5f 73 69 7a  |ags = [].dev_siz|
00001350  65 20 3d 20 38 32 34 32  32 31 37 34 34 0a 70 65  |e = 824221744.pe|
00001360  5f 73 74 61 72 74 20 3d  20 32 30 34 38 0a 70 65  |_start = 2048.pe|
00001370  5f 63 6f 75 6e 74 20 3d  20 31 30 30 36 31 32 0a  |_count = 100612.|
00001380  7d 0a 7d 0a 0a 7d 0a 23  20 47 65 6e 65 72 61 74  |}.}..}.# Generat|
00001390  65 64 20 62 79 20 4c 56  4d 32 20 76 65 72 73 69  |ed by LVM2 versi|
000013a0  6f 6e 20 32 2e 30 32 2e  31 32 32 28 32 29 20 28  |on 2.02.122(2) (|
000013b0  32 30 31 35 2d 30 36 2d  32 30 29 3a 20 46 72 69  |2015-06-20): Fri|
000013c0  20 4a 61 6e 20 20 31 20  30 30 3a 32 38 3a 30 35  | Jan  1 00:28:05|
000013d0  20 32 30 31 36 0a 0a 63  6f 6e 74 65 6e 74 73 20  | 2016..contents |
000013e0  3d 20 22 54 65 78 74 20  46 6f 72 6d 61 74 20 56  |= "Text Format V|
000013f0  6f 6c 75 6d 65 20 47 72  6f 75 70 22 0a 76 65 72  |olume Group".ver|
00001400  73 69 6f 6e 20 3d 20 31  0a 0a 64 65 73 63 72 69  |sion = 1..descri|
00001410  70 74 69 6f 6e 20 3d 20  22 22 0a 0a 63 72 65 61  |ption = ""..crea|
00001420  74 69 6f 6e 5f 68 6f 73  74 20 3d 20 22 75 62 75  |tion_host = "ubu|
00001430  6e 74 75 22 09 23 20 4c  69 6e 75 78 20 75 62 75  |ntu".# Linux ubu|
00001440  6e 74 75 20 34 2e 32 2e  30 2d 32 32 2d 6c 6f 77  |ntu 4.2.0-22-low|
00001450  6c 61 74 65 6e 63 79 20  23 32 37 2d 55 62 75 6e  |latency #27-Ubun|
00001460  74 75 20 53 4d 50 20 50  52 45 45 4d 50 54 20 46  |tu SMP PREEMPT F|
00001470  72 69 20 44 65 63 20 31  38 20 30 30 3a 30 36 3a  |ri Dec 18 00:06:|
00001480  34 38 20 55 54 43 20 32  30 31 35 20 78 38 36 5f  |48 UTC 2015 x86_|
00001490  36 34 0a 63 72 65 61 74  69 6f 6e 5f 74 69 6d 65  |64.creation_time|
000014a0  20 3d 20 31 34 35 31 36  30 38 30 38 35 09 23 20  | = 1451608085.# |
000014b0  46 72 69 20 4a 61 6e 20  20 31 20 30 30 3a 32 38  |Fri Jan  1 00:28|
000014c0  3a 30 35 20 32 30 31 36  0a 0a 00 00 00 00 00 00  |:05 2016........|
000014d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000

I am attempting to search other ares of the partition for any semblance of a LUKS string.

An important point to note where I may have gone wrong in the initial setup of the encrypted volume on pv1. Mind, it was all working well in the Debian system that it was initially setup in until I changed the system to Ubuntu and then back again to Debian. I initially made a mistake to add the logical volume in the encrypted partition to the same VG as the others on pv0. Thereafter the logical volumes on pv0 were recreated and pv1 got lost. It's not clear to me why this was originally a mistake and why the disk was not showing up with a LUKS structure to begin with. Without understanding this I fear to make the same mistake again.

rknichols · 01-03-2016, 02:39 PM

Quote:

Originally Posted by linuxbawks

I apologize for any confusion caused. I've been trying a few different things over the course of the thread and have mixed the communications up a little.

1. After initially trying to open the device with cryptsetup luksOpen it reported not a Luks device.
Though, I did specifically LUKS encrypt the entire partition first (/dev/sda3) and then set up one logical volume inside it in ext4 format.

If you had the entire sda3 partition set up as a LUKS container ("cryptsetup luksFormat /dev/sda3"), then the start of that partition (offset 00000000) would contain the ASCII string "LUKS" followed by the bytes 0xBA and 0xBE. Since those are no longer present, any data that was stored in that container is now permanently unrecoverable, and there is no point in proceeding further.

Quote:

The data in the encrypted volume is still there. But the partition header is now showing this:

Code:

# sudo hexdump -C -n 8192 /dev/sda3
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200  4c 41 42 45 4c 4f 4e 45  01 00 00 00 00 00 00 00  |LABELONE........|
00000210  bc f8 27 07 20 00 00 00  4c 56 4d 32 20 30 30 31  |..'. ...LVM2 001|
00000220  61 4a 6a 4e 36 43 4f 32  78 44 50 4c 70 66 50 32  |aJjN6CO2xDPLpfP2|
00000230  56 43 53 76 48 30 41 61  59 39 72 59 54 4a 6d 49  |VCSvH0AaY9rYTJmI|
00000240  00 60 40 41 62 00 00 00  00 00 10 00 00 00 00 00  |.`@Ab...........|
00000250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000260  00 00 00 00 00 00 00 00  00 10 00 00 00 00 00 00  |................|
00000270  00 f0 0f 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000280  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00000290  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

There's nothing in your description thus far that I can point to as the culprit, but if at any point you did a "pvcreate ... /dev/sda3" or "vgcreate ... /dev/sda3" it would have replaced that crtitcal LUKS header with the LVM header you see now.

Yes, the data is still there, but it's encrypted, and you have destroyed the key. There is no way to fix that. The way LUKS works is by generating a random key (completely unrelated to your passphrase) and then storing that key in a safe that is unlocked by your passphrase. You have obliterated that safe and its contents. While you can of course create a new safe that is unlocked by that same passphrase, it won't contain that original key, and it won't be able to decrypt your data.

Without that LUKS header, it's "Game over, thank you for playing." I'm sure it's little consolation right now, but anyone who has played with encryption has at some point lost data to it. Welcome to the club.

linuxbawks · 01-03-2016, 02:46 PM

Quote:

Originally Posted by rknichols

There's nothing in your description thus far that I can point to as the cuplrit, but if at any point you did a "pvcreate ... /dev/sda3" or "vgcreate ... /dev/sda3" it would have replaced that crtitcal LUKS header with the LVM header you see now.

This is precisely what I did.

linuxbawks · 01-03-2016, 03:12 PM

This is what I have at the moment:

Code:

# lsblk
NAME                  MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                     8:0    0 465.8G  0 disk  
\├─sda1                  8:1    0   100M  0 part  
\├─sda2                  8:2    0  72.7G  0 part  
\└─sda3                  8:3    0   393G  0 part  
sdb                     8:16   0 232.9G  0 disk  
\├─sdb1                  8:17   0    28G  0 part  /
\├─sdb2                  8:18   0     1K  0 part  
\└─sdb5                  8:21   0   205G  0 part  
  \└─sdb5_crypt        254:0    0   205G  0 crypt 
    \├─debian--vg-swap 254:1    0  29.8G  0 lvm   [SWAP]
    \├─debian--vg-var  254:2    0  23.3G  0 lvm   /var
    \└─debian--vg-home 254:3    0 151.9G  0 lvm   /home

I want to encrypt /dev/sda3 to create a container (sda3_crypt say) for one logical volume using all of the available space. Preferably: I would like to add this volume to the same volume group as the others (debian-vg) for simplicity. I would like to use the same passphrase to unlock all volumes since the data will essentially be mirrored to both volumes. I would like to issue the passphrase once to unlock both volumes at once.

What additional precautions do I need to take or be aware of should one of the two physical volumes fail or get compromised?

rknichols · 01-03-2016, 03:47 PM

Basically do what you did originally, but I recommend not using the same volume group name. There is some added complexity to activating a VG when one volume is missing, and you really don't want to have any LV that spans both volumes if you want to use it with one volume missing.

Code:

cryptsetup luksFormat /dev/sda3
cryptsetup open --type luks /dev/sda3 sda3_crypt
vgcreate otherGroup /dev/sda3_crypt
lvcreate -n someName --extents 100%FREE otherGroup
mkfs.ext4 /dev/otherGroup/someName

You might consider using some name other than "sda3_crypt" there. It's just too easy to leave off the "_crypt" part, and that can be fatal, as you've seen. Something like "crypt_a3" should keep you thinking in the right direction.

In many (most??) systems, any passphrase you enter at boot time will be tried for any LUKS volumes that are present.