Is it possible to recover a LUKS partiton after deleting it with gparted

rknichols · 10-29-2013, 11:35 AM

Quote:

Originally Posted by cyberpatrol

Why don't you stay with the facts? LVM was brought into by the OP, not by me.

Please indicate which message from the OP mentioned LVM.

rknichols · 10-29-2013, 11:42 AM

Quote:

Originally Posted by cyberpatrol

WThere is no LUKS partition header!

You might want to read the LUKS on-disk-specification, especially section 3, "The Partition Header".

cyberpatrol · 10-29-2013, 12:47 PM

Quote:

Originally Posted by rknichols

Please indicate which message from the OP mentioned LVM.

I thought I had read LVM in his last post.

cyberpatrol · 10-29-2013, 12:59 PM

Quote:

Originally Posted by rknichols

You might want to read the LUKS on-disk-specification, especially section 3, "The Partition Header".

They call it "LUKS partition header", but it's actually a LUKS header, not a partition header. Say, they use the colloquial, not the technical term there.

Do you know this one: http://www.brainstorming.co.uk/puzzles/ninedotsnj.html?

rknichols · 10-29-2013, 03:17 PM

I try to use the terms as presented in that document, where, except for one footnote describing detached header, the partition header (PHDR) and key material are always mentioned individually. Note that the "key material" is entirely separate from the "key slots" ("key slot descriptors" would be more accurate) that are part of the PHDR. The cryptsetup manpage totally confuses this by saying "LUKS header and keyslot areas" when it really means "LUKS PHDR and key material".

And yes, I've seen that puzzle before. Still can never figure out just how to do it, even remembering the "out of the box" nature of the solution.

rknichols · 10-29-2013, 04:01 PM

Quote:

Originally Posted by cyberpatrol

I thought I had read LVM in his last post.

OK, I see that he did mention "LV" there, which is why I didn't see it looking for "LVM". From his description, it appears that the volume is/was inside the encrypted container ("an encrypted LV"), which shouldn't affect decrypting that container.

Grasping at straws a bit here, another possibility would be that at one time sdb2 was an encrypted partition (which might explain the LUKS header at the beginning), but was later converted to an (unencrypted) LVM physical volume, with an LV for an encrypted /home inside it. That changes the problem into one of linking the PV into the system's LVM structure, at which time the LV should become available. But that should happen automatically either at boot time or when the device is plugged in. Then, lvscan should show the LV as "inactive" and needing just a "lvchange -ay volgroupname/lvname" to activate it. There are a lot of holes in that scenario, not the least of which are that the pvcreate should have wiped out the old LUKS header, and testdisk should at least have spotted the LVM PV header and, depending on the options settings, the LUKS header inside that. No, that really doesn't seem likely either.

cyberpatrol · 10-29-2013, 05:26 PM

Quote:

Originally Posted by rknichols

I try to use the terms as presented in that document, where, except for one footnote describing detached header, the partition header (PHDR) and key material are always mentioned individually. Note that the "key material" is entirely separate from the "key slots" ("key slot descriptors" would be more accurate) that are part of the PHDR. The cryptsetup manpage totally confuses this by saying "LUKS header and keyslot areas" when it really means "LUKS PHDR and key material".

Actually, it doesn't matter if you count the keys to the header or not. If something is overwritten you can't recover it, if it's not overwritten you can recover it. That simple.

Quote:

Originally Posted by rknichols

And yes, I've seen that puzzle before. Still can never figure out just how to do it, even remembering the "out of the box" nature of the solution.

That's probably your problem. In the figurative sense you're staying inside the grid of the 9 dots. And actually I find this case a lot simpler than the 9 dot puzzle. I already gave you quite a lot of hints. And you already mentioned yourself the point where to start.

Just do me and yourself a favor and forget testdisk for this case. LVM is also not in question, it just would make the analysis of testdisk rather more unreliable, which is unreliable because of LUKS anyway.

cavig5 · 11-02-2013, 10:07 AM

It doesn't look like there's another LUKS header on the disk. I ran testdisk after changing the options to
Cylinder boundary : No
and also ran the "Deeper Scan" and nothing new came up.

The suggested hex dump doesn't have any structure I can see. Nothing in the ASCII interpretation and nothing regular looking in the hex version. Short snip below.

------------------------
00000000 a7 60 ac cf 8e 16 e3 d1 81 b3 20 c3 db 02 9f 02 |.`........ .....|
00000010 9c 60 5b 1b e1 35 93 78 5b 41 cc 73 85 9e e5 96 |.`[..5.x[A.s....|
00000020 aa f0 e2 49 61 98 97 37 4f 87 70 e6 88 41 df 05 |...Ia..7O.p..A..|
00000030 2f b6 b4 c2 c2 c6 b5 6c 3a e3 37 0d 0d 50 fd 65 |/......l:.7..P.e|
00000040 a1 1d 53 cc 00 4e a7 a2 fc 16 ca d8 54 d8 e3 a4 |..S..N......T...|
00000050 ad 3f ad 85 de 4f ac 02 04 29 2d b3 87 83 ae 8a |.?...O...)-.....|
00000060 b0 66 30 24 eb 21 ff 7c 0c 19 c6 aa 7a bb df 37 |.f0$.!.|....z..7|
00000070 88 bc b1 13 48 ae 48 34 2a 33 77 b8 e2 74 ba ae |....H.H4*3w..t..|
00000080 b6 a7 74 14 83 ad 1e 6c 7e 69 17 66 7d c6 71 99 |..t....l~i.f}.q.|
00000090 c3 2b 02 2e 2c 40 46 cd 65 02 1a 02 fa bc be c8 |.+..,@F.e.......|
000000a0 9a ba 88 fb ac 9a 3a 8d 51 ea d3 d7 bf 96 61 3d |......:.Q.....a=|
000000b0 02 92 20 79 78 77 48 71 3c eb 2c ff 07 0f 62 b1 |.. yxwHq<.,...b.|
000000c0 17 96 67 1f 68 ae 19 ee cf f0 6b e0 d1 1c 05 2e |..g.h.....k.....|
000000d0 92 57 96 02 56 e9 0e d6 6b b8 27 75 1b f5 a3 f5 |.W..V...k.'u....|
000000e0 cb 03 c4 83 42 e0 d3 ca 76 85 a7 49 53 b9 23 da |....B...v..IS.#.|
000000f0 92 5a 9f cf 3c 07 95 c2 62 e7 2b ca 12 fb 0f bf |.Z..<...b.+.....|
------------------------

I hate to admit it but I'm not sure exactly what I did. It was late and I was trying to install a new disk on the system. I was using gparted and looking at the wrong disk. For some reason I deleted the home partition, realized I'd goofed, and created a new partition in the space that had been freed up.

From everything I've seen so far, this shouldn't have destroyed the data. It seems that the only the location and size of the partition could be wrong. The LUKS header is in the right place so that leaves me wondering if the size matters.

Does the partition size in the partition table have to match the original partition size for cyrptsetup to be able to open it?

Thanks again for all the help!

cavig5 · 11-02-2013, 04:08 PM

Quote:

Originally Posted by rknichols

Grasping at straws a bit here, another possibility would be that at one time sdb2 was an encrypted partition (which might explain the LUKS header at the beginning), but was later converted to an (unencrypted) LVM physical volume, with an LV for an encrypted /home inside it. That changes the problem into one of linking the PV into the system's LVM structure, at which time the LV should become available.

I was reading through your comments again and this seems like a possibility. I'm not sure whether the system was set up with a LV in an encrypted partition or the other way round. When I first started trying to recover, I saw the LUKS header and assumed the LV was in an encrypted partition. However, I remember having trouble setting up the encrypted system and trying different approaches so that could explain a stray LUKS header. It could well be that the encrypted partiton was in a LV.

Could the partition deletion/recreation have destroyed whatever the LVM needs to link the PV?

cyberpatrol · 11-02-2013, 04:21 PM

Quote:

Originally Posted by cavig5

It doesn't look like there's another LUKS header on the disk. I ran testdisk after changing the options to
Cylinder boundary : No
and also ran the "Deeper Scan" and nothing new came up.

The suggested hex dump doesn't have any structure I can see. Nothing in the ASCII interpretation and nothing regular looking in the hex version. Short snip below.

Like I said several times before, don't trust testdisk in such a case (LUKS)! Use a hexeditor for searching for a LUKS header.

Quote:

Originally Posted by cavig5

Does the partition size in the partition table have to match the original partition size for cyrptsetup to be able to open it?

That's what I said to rknichols before. He's still in the 9 dot grid. There is no LUKS partition (even if this is how it's usually called). There's only a LUKS container.

One other hint: You don't need to write anything onto this harddisk, you just need a second harddisk to copy the data.

rknichols · 11-02-2013, 04:42 PM

Quote:

Originally Posted by cavig5

Does the partition size in the partition table have to match the original partition size for cyrptsetup to be able to open it?

No, the size does not have to match. The size just needs to be at least as big as the LUKS PHDR plus key material, which would be just over 1 Megabyte for the LUKS header you showed earlier (payload offset = 2056 512-bit sectors).

Here is a last resort to keep testdisk detractors happy:

Code:

dd if=/dev/sdb skip=498015 | hexdump -C | grep '^[0-9a-f]*[02468ace]00 .* |LUKS'

That will display the start of any sector (byte offset that is a multiple of 0x200) that begins with the ASCII characters "LUKS". The "skip=498015" will make dd start just after the known first partition, so you would need to add 598015*512 to the displayed offset from hexdump. Running that on a 1TB disk would take about halfway to forever (the CPU time for hexdump + grep is limiting), so you might want to include "count=2097152" to the dd arguments to limit it to the first Gigabyte.

Or, here's a quick and dirty C program to do the search:

Code:

#define _FILE_OFFSET_BITS 64
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main (int argc, char *argv[]) {
    int fd, n;
    off_t offset;
    char buf[64];
    char *cmd;

    if((cmd = strrchr(argv[0], '/')) > 0)  ++cmd;
    else  cmd = argv[0];
    
    if(argc != 3) {
	fprintf(stderr, "Usage: %s device offset\n", cmd);
	return 1;
    }
    offset = strtol(argv[2], NULL, 0);
    if((offset%512) != 0) {
	fprintf(stderr, "ERROR [%s]: Offset not a multiple of 512\n", cmd);
	return 1;
    }
    printf("Starting at byte offset %ld\n", offset);
    if((fd = open(argv[1], O_RDONLY)) < 0) {
	perror("open()");
	return 1;
    }

    while(lseek(fd, offset, SEEK_SET) >= 0) {
	if((n = read(fd, buf, 16)) <= 0) {
	    if(n == 0) {
		printf("EOF at offset %ld\n", offset);
		return 0;
	    }
	    perror("read()");
	    return 1;
	}
	if(strncmp(buf, "LUKS", 4) == 0) {
	    printf("Match at offset %ld\n", offset);
	}
	offset += 512;
    }
    sprintf(buf, "At offset=%ld: ", offset);
    perror(buf);
    return 0;
}

cyberpatrol · 11-02-2013, 07:37 PM

Quote:

Originally Posted by rknichols

Or, here's a quick and dirty C program to do the search:

Again: Have you heard about hexeditors? Do you know that they have a search function? Do you know how to type "LUKS" into this search field?

rknichols · 11-02-2013, 11:00 PM

Quote:

Originally Posted by cavig5

I was reading through your comments again and this seems like a possibility. I'm not sure whether the system was set up with a LV in an encrypted partition or the other way round. When I first started trying to recover, I saw the LUKS header and assumed the LV was in an encrypted partition. However, I remember having trouble setting up the encrypted system and trying different approaches so that could explain a stray LUKS header. It could well be that the encrypted partiton was in a LV.

Could the partition deletion/recreation have destroyed whatever the LVM needs to link the PV?

Regardless of whether the LV was inside the encrypted container or the other way around, there would still be a LUKS header somewhere to be found. Even if the encryption were inside the LV (and you would still have to reassemble the LVM structure before you could actually mount the filesystem), the default extent size (the allocation unit for the LV, typically 4MB) would be large enough to include the entire LUKS header with the key material within one extent, so you should still be able to unlock the master key via that header even though actual decryption of the content could yield garbage (what LUKS was trying to decrypt as sector N might be getting mapped to a different sector on the disk).

I believe you still have your root filesystem intact. You should be able to see the most recent state of your LVM structure in the text files in the /etc/lvm/backup directory. There will be a file in there for each volume group showing exactly how everything was allocated. Look for the "device =" lines for the physical volume(s). If it's showing physical partitions (e.g., "/dev/sdb2"), then the LVM structure is on the outside with encrypted (or not) LVs within. If it's showing "/dev/mapper/xxxx", then the whole physical volume is within an encrypted container.

rknichols · 11-03-2013, 12:34 AM

I'm afraid I have some really bad news. I just took a USB flash drive, used gparted to make one partition, /dev/sdc1, with the default start and end. I then ran "cryptsetup luksFormat /dev/sdc1" to make that partition an encrypted container. Then I tried what you probably did, which was to use gparted to delete that partition and then create a new partition, unformatted, in that same place. The LUKS container there could no longer be opened.

I had done a luksHeaderBackup of the container in its pre-deletion state and compared that file with a luksHeaderBackup done from the re-created partition. There was massive corruption beginning at offset 0x10000, which is within the region for the key material for the first keyslot. The LUKS PHDR itself was, like your, undamaged. If you look at /dev/sdb2 beginning at offset 10000 you will see the ASCII text that gparted dumped there for reasons known only to its authors.

Barring the miraculous discovery of a LUKS header somewhere else on the drive, your data is unrecoverable.

cyberpatrol · 11-03-2013, 05:55 AM

Quote:

Originally Posted by rknichols

I'm afraid I have some really bad news. I just took a USB flash drive, used gparted to make one partition, /dev/sdc1, with the default start and end. I then ran "cryptsetup luksFormat /dev/sdc1" to make that partition an encrypted container. Then I tried what you probably did, which was to use gparted to delete that partition and then create a new partition, unformatted, in that same place. The LUKS container there could no longer be opened.

I had done a luksHeaderBackup of the container in its pre-deletion state and compared that file with a luksHeaderBackup done from the re-created partition. There was massive corruption beginning at offset 0x10000, which is within the region for the key material for the first keyslot. The LUKS PHDR itself was, like your, undamaged. If you look at /dev/sdb2 beginning at offset 10000 you will see the ASCII text that gparted dumped there for reasons known only to its authors.

Barring the miraculous discovery of a LUKS header somewhere else on the drive, your data is unrecoverable.

Please! When do you stop spreading such a FUD? IF the partition was NOT overwritten, the encrypted data in the LUKS container CAN be recovered!

Unless you have seen the OP's harddrive you can't tell, if HIS data can or cannot be recovered.

Once again: Use a hexeditor and search for the ASCII string "LUKS". IF you find the LUKS header (the string "LUKS" can be on the harddrive more than once, not only at the beginning of the header) then you can access the LUKS container and recover the data.

And forget the nonsense about "header" and "keymaterial"! IF none of them are overwritten, it IS still there. And you don't know, what gparted did on the OP's harddrive unless you haven't seen and searched HIS harddrive.