LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 09-05-2017, 08:15 PM   #1
supermistershell
LQ Newbie
 
Registered: Feb 2017
Posts: 3

Rep: Reputation: Disabled
Need help to recover luks partition


Unfortunately, I have an infamous motherboard which has HPA and was responsible for this situation.

I didn't knew about the HPA curse until now, because of this, and learnt the hard way.

Anyone interested on HPA curse can read more here in detail:
https://forums.lime-technology.com/t...fferent-sizes/


So, to make a long story short, I have a 3TB disk which was partitioned like this:
parted /dev/sdx
mklabel gpt
unit s
mkpart primary ext3 2048s -1s
Warning: You requested a partition from 2048s to 5860533167s.
The closest location we can manage is 2048s to 5860533134s.
Is this still acceptable to you?
Yes/No? y

then:
cryptsetup luksFormat --cipher aes-cbc-essiv:sha256 --verify-passphrase --key-size 256 /dev/sdx1
cryptsetup luksOpen /dev/sdx1 sdx1_crypt
mkfs.ext3 /dev/mapper/sdx1_crypt

Afterwards, I started using this disk for storing ebooks, movies, pictures, etc, although most of the time it was just in a drawer.

Everything went fine during many months, until one day it stopped working because of the HPA curse I wrote about above, when I connected it to my main computer and found there was nothing in it; no LUKS partition, nothing.

I remembered one time I had a similar situation and it was solved by creating an exactly equal new partition and so I did it, but it didnt work this time:
parted /dev/sdx
mklabel gpt
unit s
mkpart primary ext3 2048s -1s
Warning: You requested a partition from 2048s to 5860533167s.
The closest location we can manage is 2048s to 5860531021.
Is this still acceptable to you?
Yes/No? y

But this time I noticed that the last sector had changed and the total number was less 2113 sectors!

At that time, I didn't knew why this 2113 sectors were missing; now I know is because of the HPA curse.

So, I stopped and made a clone of this disk using DD to another disk of same make, model and capacity that fortunately I had around.

Then I copied the partition table of a similar disk I had (also partitioned and formatted the same way and of same make, model and capacity)to this clone:
dd if=/dev/zero of=/dev/sdb bs=512 count=1
but I couldn't recover my luks partition

So, then I've ran testdisk twice:
one for EFI or GPT partition structure and a second time for intel partition structure just to see the differences.

Here's the output of the GPT partition structure (if the second is also needed, please say so and I'll post it):

TestDisk 6.13, Data Recovery Utility, November 2011
Christophe GRENIER <grenier@cgsecurity.org>
http://www.cgsecurity.org

Disk /dev/sdb - 3000 GB / 2794 GiB - CHS 364801 255 63

The harddisk (3000 GB / 2794 GiB) seems too small! (< 14785030 TB / 13446906 TiB)
Check the harddisk size: HD jumpers settings, BIOS detection...

The following partition can't be recovered:
Partition Start End Size in sectors
> MS Data 642191996 28877012508677537 28877011866485541


[ Continue ]
JFS 1381173050, 14785030 TB / 13446906 TiB

TestDisk 6.13, Data Recovery Utility, November 2011
Christophe GRENIER <grenier@cgsecurity.org>
http://www.cgsecurity.org

Disk /dev/sdb - 3000 GB / 2794 GiB - CHS 364801 255 63
Partition Start End Size in sectors
>P MS Data 3298905435 3302322120 3416686 [~UM-X D ^X^\ M-3Cm]

Structure: Ok. Use Up/Down Arrow keys to select partition.
Use Left/Right Arrow keys to CHANGE partition characteristics:
P=Primary D=Deleted
Keys A: add partition, L: load backup, T: change type,
Enter: to continue
cramfs, 1749 MB / 1668 MiB

Testdisk.log

Tue Sep 5 14:26:21 2017
Command line: TestDisk
Code:
TestDisk 6.13, Data Recovery Utility, November 2011
Christophe GRENIER <grenier@cgsecurity.org>
http://www.cgsecurity.org
OS: Linux, kernel 3.2.0-4-amd64 (#1 SMP Debian 3.2.81-2) x86_64
Compiler: GCC 4.6
Compilation date: 2012-01-17T14:04:23
ext2fs lib: 1.42.5, ntfs lib: 10:0:0, reiserfs lib: none, ewf lib: none
/dev/sdb: LBA, HPA, LBA48, DCO support
/dev/sdb: size       5860533168 sectors
/dev/sdb: user_max   5860533168 sectors
/dev/sdb: native_max 5860533168 sectors
Hard disk list
Disk /dev/sdb - 3000 GB / 2794 GiB - CHS 364801 255 63, sector size=512 - ST3000DM001-1CH166, S/N:Z1F0TKXK, FW:CC43

Partition table type (auto): EFI GPT
Disk /dev/sdb - 3000 GB / 2794 GiB - ST3000DM001-1CH166
Partition table type: EFI GPT

Analyse Disk /dev/sdb - 3000 GB / 2794 GiB - CHS 364801 255 63
hdr_size=92
hdr_lba_self=1
hdr_lba_alt=5860533167 (expected 5860533167)
hdr_lba_start=34
hdr_lba_end=5860533134
hdr_lba_table=2
hdr_entries=128
hdr_entsz=128
check_part_gpt failed for partition
 1 P MS Data                     2048 5860533134 5860531087 [3TBbk]
Current partition structure:
No FAT, NTFS, ext2, JFS, Reiser, cramfs or XFS marker
 1 P MS Data                     2048 5860533134 5860531087 [3TBbk]
 1 P MS Data                     2048 5860533134 5860531087 [3TBbk]

search_part()
Disk /dev/sdb - 3000 GB / 2794 GiB - CHS 364801 255 63

recover_JFS: s_blocksize=424374330
recover_JFS: s_size 8234812537959046209
recover_JFS: s_fsckpxd.len:13554014
recover_JFS: s_logpxd.len:4479763
recover_JFS: part_size 28877011866485541
     MS Data                642191996 28877012508677537 28877011866485541
     JFS 1381173050, 14785030 TB / 13446906 TiB
This partition ends after the disk limits. (start=642191996, size=28877011866485541, end=28877012508677537, disk end=5860533168)

SYSV4 Marker at 41949/41/20

recover_sysv4
     Unknown                673913287 2171750072774 2171076159488 [B'’ƒf]
     SysV4, 1111 TB / 1010 TiB
Partition not added.
check_FAT: Bad jump in FAT partition

cramfs Marker at 205347/93/22

recover_cramfs
     MS Data               3298905435 3302322120    3416685 [•SD{NCm]
     cramfs, 1749 MB / 1668 MiB
check_FAT: Bad jump in FAT partition

LVM magic value at 239851/51/9
check_FAT: Bad number of sectors per cluster

LVM magic value at 290039/129/30
Disk /dev/sdb - 3000 GB / 2794 GiB - CHS 364801 255 63
Check the harddisk size: HD jumpers settings, BIOS detection...
The harddisk (3000 GB / 2794 GiB) seems too small! (< 14785030 TB / 13446906 TiB)
The following partition can't be recovered:
     MS Data                642191996 28877012508677537 28877011866485541
     JFS 1381173050, 14785030 TB / 13446906 TiB

Results
   P MS Data               3298905435 3302322120    3416686 [•SD{NCm]
     cramfs, 1749 MB / 1668 MiB
SIGINT detected! TestDisk has been killed.
Maybe this is also useful:
Code:
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001c0  01 00 ee fe ff ff 01 00  00 00 ff ff ff ff 00 00  |................|
000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
00000200  45 46 49 20 50 41 52 54  00 00 01 00 5c 00 00 00  |EFI PART....\...|
00000210  6a 10 1f 57 00 00 00 00  01 00 00 00 00 00 00 00  |j..W............|
00000220  4d 93 50 5d 01 00 00 00  22 00 00 00 00 00 00 00  |M.P]....".......|
00000230  2c 93 50 5d 01 00 00 00  c5 c7 67 95 55 51 cb 49  |,.P]......g.UQ.I|
00000240  ae 6b 5e 65 13 28 19 69  02 00 00 00 00 00 00 00  |.k^e.(.i........|
00000250  80 00 00 00 80 00 00 00  86 d2 54 ab 00 00 00 00  |..........T.....|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002800

Is it still possible to recover this luks partition?

Last edited by supermistershell; 09-06-2017 at 04:34 AM. Reason: replace typo HDA (incorrect) with HPA (correct)
 
Old 09-05-2017, 10:19 PM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 16,001

Rep: Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219
Argggh !.
I'm thinking you have lost the lot - have a read of this post. Hopefully rknichols will drop by and offer an opinion.
 
1 members found this post helpful.
Old 09-05-2017, 11:22 PM   #3
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,624

Rep: Reputation: 1578Reputation: 1578Reputation: 1578Reputation: 1578Reputation: 1578Reputation: 1578Reputation: 1578Reputation: 1578Reputation: 1578Reputation: 1578Reputation: 1578
Hello

The sad news is that LUKS stores the encrypted master key in a header at the beginning of the container (partition/disk/LVM volume/...). That header is 1 to 2 MB in size (depends on the cipher's key size). Once even one sector of that key material is overwritten, the master key is forever unrecoverable.

Overwriting the start of the disk with a copy of the BIOS would certainly do that to a partition in that region. But even had the BIOS not done that, running "mkpart primary ext3 2048s -1s" would also have written an ext3 super block and group descriptor there, overwriting the LUKS header in that partition. You would have to tell parted to create an empty partition, not a filesystem, in order to preserve the LUKS header.

You can tell fairly quickly whether there is any chance of recovery. Run "hexedit -s /dev/sdx" and search for the hex string "4C 55 4B 53 BA BE" at the start of a sector. (That's the ASCII string "LUKS" followed by the hex bytes 0xBA and 0xBE.) If you don't find that within the first few megabytes of the disk, the LUKS header is gone. Without a backup of that header there is no chance of recovering any data. (Running luksFormat with the same passphrase will not help. The actual master key is a random number unrelated to the passphrase.)

Welcome to the club. Almost everyone who has used encryption has at some point lost access to some of their data. (Yes, me too. Why did I think the index to a collection of naughty magazines was so private when the mags themselves were right over there in a box?) It's like the lock on a vault, deliberately designed to be as unforgiving as possible if everything is not exactly right.
 
1 members found this post helpful.
Old 09-05-2017, 11:45 PM   #4
IsaacKuo
Senior Member
 
Registered: Apr 2004
Location: Baton Rouge, Louisiana, USA
Distribution: Debian 9 Stretch
Posts: 2,299
Blog Entries: 8

Rep: Reputation: 368Reputation: 368Reputation: 368Reputation: 368
Should "HDA" be "HPA"? Anyway, reading around about it, I think the end of the disc gets overwritten, not the start. So, if the important stuff is stored near the start, then there may be hope?
 
1 members found this post helpful.
Old 09-06-2017, 05:21 AM   #5
supermistershell
LQ Newbie
 
Registered: Feb 2017
Posts: 3

Original Poster
Rep: Reputation: Disabled
Unhappy

Quote:
Originally Posted by syg00 View Post
Argggh !.
I'm thinking you have lost the lot - have a read of this post
Ok, thanks. I've read it and understood it



Quote:
Originally Posted by rknichols View Post
The sad news is that LUKS stores the encrypted master key in a header at the beginning of the container (partition/disk/LVM volume/...). That header is 1 to 2 MB in size (depends on the cipher's key size). Once even one sector of that key material is overwritten, the master key is forever unrecoverable.

Overwriting the start of the disk with a copy of the BIOS would certainly do that to a partition in that region. But even had the BIOS not done that, running "mkpart primary ext3 2048s -1s" would also have written an ext3 super block and group descriptor there, overwriting the LUKS header in that partition. You would have to tell parted to create an empty partition, not a filesystem, in order to preserve the LUKS header.

You can tell fairly quickly whether there is any chance of recovery. Run "hexedit -s /dev/sdx" and search for the hex string "4C 55 4B 53 BA BE" at the start of a sector. (That's the ASCII string "LUKS" followed by the hex bytes 0xBA and 0xBE.) If you don't find that within the first few megabytes of the disk, the LUKS header is gone. Without a backup of that header there is no chance of recovering any data. (Running luksFormat with the same passphrase will not help. The actual master key is a random number unrelated to the passphrase.)
Thank you for this very clear explanation.
I've searched the hex string but unfortunately it just wasn't there



Quote:
Originally Posted by IsaacKuo View Post
Should "HDA" be "HPA"?
Yes, HPA. Sorry, my bad.
Thanks for pointing that out. I've edited the OP.



Quote:
Originally Posted by IsaacKuo View Post
reading around about it, I think the end of the disc gets overwritten, not the start. So, if the important stuff is stored near the start, then there may be hope?
Unfortunately no because the start also got a litle overwritten, as rknichols explained here:
Quote:
Originally Posted by rknichols View Post
Overwriting the start of the disk with a copy of the BIOS would certainly do that to a partition in that region. But even had the BIOS not done that, running "mkpart primary ext3 2048s -1s" would also have written an ext3 super block and group descriptor there, overwriting the LUKS header in that partition. You would have to tell parted to create an empty partition, not a filesystem, in order to preserve the LUKS header.


I guess I'm just going to accept defeat and live with the last backup I have.
I've lost about 10% of data from the time it was made.
Could be a lot worst

Thank you all for helping me!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] recover from deleted luks encryption partition unixedway Linux - General 12 01-16-2017 07:33 AM
dd'ed on an external HDD Luks partition, recover possible? nr765 Linux - General 8 12-22-2015 10:16 AM
[SOLVED] Lvm luks partition recover naguera Linux - Hardware 8 12-06-2013 10:31 AM
recover lost luks partition ehsdav Linux - General 10 07-09-2013 05:50 AM
Recover encrypted LUKS partition itinlopez Linux - General 3 11-30-2008 03:20 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 10:54 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration