So while checking out some threads I came across Testdisk, unfortunately, I didn't really know what I was doing and wrote some random partitions to the disk. The LUKS headers are still intact but when I mount them there's nothing that was in my original disk. I'm wondering if by writing the random partitions to the disk that I might have overwritten my files that I need.

When my disk was still working I came across an error while updating to Ubuntu 18.04.1 and used gparted to change my sdx5 partition to swap, which was originally my LUKS partition. I'm afraid that might've screwed up my files as well.

Anyway, I'm able to mount my disk, but it's only 83gb (originally 113gb). Now in Testdisk I'm not really sure how to continue. Please help?

PS. This is on Ubuntu. The original partition table was set up when I installed it using FDE, so SDA1 would've been boot, which I have, SDA2, which was my FS, and SDA5, which was LUKS. With testdisk I'm able to see SDA2, which is 113gb, which I would assume to be my FS, but it won't fit with my boot and LUKS partitions.

If you change your partition scheme, you lose your data.

While you can increase the size of a partition (provided there is free space available), and you can shrink the size of a partition (providing you have unused space you can discard), it is very unlikely that a random partition change that was an error or done with no particular knowledge about what the risks were, will put you in a position where you could recover anything.

What can possibly be done is a forensic recovery where you take the disk offline immediately, get a full image of the disk, and then use tools or hire someone to help you recover files.

There are no tools I can recommend. There is a photo recovery software which people use, I tried it once, it found ... something, a very many something's ... but not all that I needed. I had re-imaged a disk and wanted to try and reconstruct too. What I found was that the effort I might put into learning, and also not getting every part of most files, was that it was far too much trouble for my worth, for also day to day data. The loss of any data is bad. Some or most of it is entirely non-recoverable, but some or most of it is sometimes able to be reconstructed. In other words, code I had written ... I had to write it again. Code I had backed up on our server, was my starting point. So I lost a couple of days work. Live and learn.

Sorry. Just my opinion. Others may suggest some data recovery techniques. I'm just saying the one time I tried, it was too much effort for too little return.

If you only changed the partition table without modifying any data, and you remember/backed up/wrote down your original partition details, you just recreate the partition table.

If you also overwrote some data, you are likely to be out of luck, although it depends, of course, which data you overwrote.

Your description is not quite clear, but you mention "swap", which could mean overwritten data, and "sda5", which indicates an extended partition that might occupy blocks anywhere on the disk. You may have overwritten data by modifying your logical partitions.

What I would do, and it seems you have done that partially: If you see the LUKS header, you should be able to figure out where the LUKS partition starts (ask Google, not me). Then create a partition that starts at that location and has the original size of the LUKS partition, or larger. After that, cryptsetup, fsck, mount, and recover as much as you can. Or cryptsetup, then a suitable filesystem debugger.

Of course, this way you could lose other potentially valuable data on that disk. Therefore, first clone the disk to a file or another disk so that you can repeatedly perform destructive tests.

Not true - simply (immediately) reverting to the old structure works fine. It is subsequent actions that puts data at risk.
I would expect you to lose everything (including the LUKS headers), so count yourself lucky. Your entire post is vague and disjointed and impossible (for me) to determine what you did, and when. Was the swap assigned as part of the upgrade, or simply changed when the system was up and running ?.Again this is just confusing the issue - what is "FS" ?. What do the sizes refer to ?. From a liveCD run this and post all the output (change device as appropriate in need)

Edit: ... too slow typing again

Hi, thanks for the answers guys. I have not physically written anything nor done anything other than change sda5 to swap as I encountered an error while updating and googled a "solution" that was supposed to fix it. I've only changed around the partition tables so far.

@syg00: FS = filesystem.

@berndbausch: I read not to resize the LUKS partition to the left as it might overwrite data, is that correct? Also, unfortunately, I have not saved the original partition table. Just one more thing to remember to do when having your system running well.

@syg00: ubuntu-mate@ubuntu-mate:~$ sudo lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
loop0
squash /rofs
loop1
squash /snap/core
loop2
squash /snap/puls
loop3
squash /snap/soft
loop4
squash /snap/ubun
sda
├─sda1
│ ext2 78569fbf-ea75-4fa0-b3d2-248360e84ec2
└─sda2
crypto 7d2c9f84-813c-4da4-a62f-5cfac9b88b67
sdb
sr0 iso966 Ubuntu-MATE 18.04.1 LTS amd64
2018-07-25-03-37-28-00 /cdrom

Really hoping I can restore my filesystem. I can actually boot into grub and go into recovery mode. Thank you guys very much for the help, and I will research some more...

ubuntu-mate@ubuntu-mate:~$ sudo parted /dev/sda "print free"
Model: SanDisk Cruzer Glide 3.0 (scsi)
Disk /dev/sda: 125GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number Start End Size Type File system Flags
32.3kB 1049kB 1016kB Free Space
1 1049kB 512MB 511MB primary ext2
512MB 513MB 1049kB Free Space
2 513MB 515MB 2097kB primary
515MB 125GB 125GB Free Space

I've attached an image of what my sda disk looks like in gparted, really hoping it can be restored.

Attached Thumbnails

 

Your problem is the size of your LUKS partition. When scanning for partitions, testdisk has no way to determine the size of a LUKS partition (there is no size field in the LUKS header), and thus always assumes the minimum possible size of 2 MB, just enough for the LUKS header itself.

Step 1: Make a backup of the LUKS header. If that header is overwritten, your data is lost forever.

Step 2: Enlarge the LUKS partition so that it extends to the end of the disk. I would not trust gparted to do that. Older versions will simply refuse. New versions will "support" that operation by digging through to the underlying filesystem (which currently does not exist) and try to resize that filesystem to match the new partition size. You really, really don't want it to do that. Use a basic partitioning tool like fdisk to delete the existing partition and create a new one starting in the same place.

Ok, thanks for the info. I'll try that when my dd copy is done. I will backup the LUKS header and try what you've suggested.

Now when you say to delete the existing partition which one are you referring to specifically? The LUKS partition or the unnallocated partition? I'm assuming the LUKS partition? So first find the start of the LUKS header, using hexdump I'm assuming, then extend it to the end of the disk.

That unallocated space is just that - not an unallocated "partition", so just ignore it - delete the LUKS partition /dev/sda2. With luck you should be able to just allow fdisk to allocate the new /dev/sda2 using defaults - it should get the start right and will default to the end of the disk. You'll soon know when you try to mount it.

I don't know what you mean by "resizing on the left". I meant modifying the partition table the way svg00 is suggesting - create a partition that starts at the LUKS partition and has the expected size.

It turns out that the LUKS partition is at the end of the disk, at least it seems to be. This means there is no data after the LUKS partition, and you don't risk anything extending the partition until the end.

Remember: By changing primary partitions, all you do is modify the partition table. In the case of a DOS partition table, this is a 64 bytes area at the end of the first 512 bytes block. You don't change anything else on the disk.

I don't think you actually wrote new partitions to your disk. I think you changed a partition from LUKS to swap. If you can open that partition without errors, you can use foremost to carve out any salvageable files. Making a swap partition does not entail much writing. So, the files should still be there.

You just want to open the LUKS volume. You don't need to mount it. The partition device file will be in '/dev/mapper'. You'll need to know that for foremost. Foremost needs to know the header bytes of every file type you want it to search for and recover.

There is a file called called foremost.conf. It requires an entry for each file type you want the program to look for, i.e. jpg, mp4, odt, txt, etc. The format of the file is listed at the end of foremost man page. Make sure you read and understand what you are doing before you start, including taking the disk in question of service, imaging it to another disk (not to a file), and recovering your files from the image disk.

Foremost was written by two special agents of the US Air Force special investigations division. It is very effective, fast and accurate. But it won't preserve file names. Every recovered file will have a number for the name, with the proper extension.

To get the proper header and footer bytes, you can look at any file of a certain type with The first 4 sets of 2 are the header. For the footer, you do the same thing without the count=1. The last 4 sets of 2 are the footer.

But some file types have no unique footer, so you always have to check the footer of several files of that type to determine of the footer bytes are the same. If they're not, then leave them out of the foremost.conf entry for that file type. Footer bytes are optional in the config file. But if a file type has a unique footer, you should include it.

As long as you didn't write to anything except the partition table, there should not be any need to use foremost or any other recovery tool. Once you extend the partition and unlock the LUKS volume, whatever filesysten was there before should be back, intact.

Agree - data recovery is a PITA even when it works.

@syg00: Just to make a confirmation - use fdisk to delete the LUKS partition, and extend it to the end using the default values? When I try mounting it I'm assuming it'll still prompt for a password?

@berndbausch: By resizing to the left I mean resizing to the the beginning of the disk, instead of the end. I read that resizing toward the beginning of the disk/partition can cause overwriting of data.

@AwesomeMachine: Yes, I'm sure I wrote new partitions to the disk, or at least previous partitions that remained from the last time I formatted the disk.

@rknichols: So writing random partitions to my disk using testdisk shouldn't damage any data right?

So so far I've done: sudo fdisk -W auto /dev/sda2, deleted sda2, created a new partition sda2 using defaults, now have sda2 partition till the end of the disk. How do I mount the partition though? When I try to mount it it says bad filesystem type, bad superblock, other error, etc.

I looked up how to mount partitions but I could not find an alternative superblock, and fsck comes up with the same error of it not describing a valid extX filesystem.

If you made only primary partitions (1 through 4), then the only thing written is the primary partition table in the MBR. The rest of the disk is untouched.

What does "file -s /dev/sda*" report now? If you created partition 2 in the same location as before it should be seen as "LUKS encrypted file".

What is the output from "fdisk -l /dev/sda" ?

I'd say this is true, however I fear "subsequent actions". Because I'm seeing a lot about moving left and extending to end. That's all graphical and non-precise because of sliding with the mouse in gparted.

My example/background would be fdisk where I printed the partition data, so it was retained in my command prompt. And just because, not due to some protective process, but because I want to see what's there to confirm I'm on the right disk, and then proceed to delete the partitions and create new ones.

After which I'd write the new table, exit, and then use mkfs and eventually figure out that I'd screwed myself once I tried to use the disk.

Say I detected my error just after exiting fdisk. Well, in my example, I'd have my old partition table, and barring any other possible action, "yes", I'd try to make the partition table re-match what it had before, and cross my fingers. It does sound as if it would work.

1. I've just never been in a position where I detected the error early enough, and doubt I'll ever be.
2. OP here does not have their original partition table exactly, unless they saved a screenshot of the old table.
3. Is there a way to derive a former partition table in any way?