Hello all

I've had many situations of possible loss of all my data, and I've always been very impatient, complicating things and finally giving up and erasing everything. This time I can't afford that.

This is what happened: I had debian lenny installed on my amd64 desktop, with a separate ext3 /home partition. It was not encrypted.

Now I have installed ubuntu 9.10 amd64 next to it, from the live cd. Here is where I went wrong. I told it to use my debian /home partition as ext4, and keep all data. This partition is in reality ext3. On top of that, I told it to encrypt it.

Now I cannot mount that partition on debian, and so I cannot log in normally. Also, the new ubuntu system logs in to a terminal, not to gnome. When I look at the /home partion from there I only see three folders, my ubuntu user folder, lost+found, and ".encryptfs".

It seems as though ubuntu did format the partition, against my direct orders! Makes sense, because you cannot use an ext3 partition as ex4 without formatting. Or can you?

Now can I somehow undo the changes ubuntu did to the partition, and make debian recognize it as ext3? Even if just to copy my important files.

Or do I need to use a data recovery software that gets the files from the old ext3 partition? If so, any recommendations?

Any help is appreciated.

Photorec (http://linux.softpedia.com/get/Syste...ec-12060.shtml) is a pretty good piece of software for recovering smaller files from hard drives (up to about 2M or so). I have used it on several systems and have been very surprised at what it can recover.

It works by scanning the disk. It doesn't look at directory entries. This means that it cannot name your files, they just end up as number.extension. However, it can also recover superblocks which might be handy.

Worth a try maybe.

Then, I suggest you to stop, take some tea. Watch a movie, and after that, start doing a backup. Use dd to dump the offending partition into an image. This is really necessary if you want to ensure that nothing silly that you do will completely trash your date (if it hasn't been trashed already).

Now, some theory:

You can mount an ext2/3 volume as ext4 without a problem. The thing goes this way: ext2 and ext3 are compatible, both forward and backward. This means that you can mount an ext2 volume as ext3, and you can also mount an ext3 volume as ext2.

From now on, I will talk about ext2 for the sake of simplicity, but bear in mind that each time I say "ext2" I am really talking about either ext2 or ext3, since they are really the same fs.

Ext4 is compatible with ext2 in the sense that you can mount an ext2 volume as ext4. *BUT*, some of the features that ext4 offers are not compatible with ext2. When you mount an ext2 volume as ext4 it continues to be an ext2 volume until the very moment you write something to disk. In that moment some of the files on that disk are no longer compatible with ext2, and hence, the ext2 layout is broken. From that moment, the volume will only be mountable as ext4.

It's possible to mount an ext2/3 volume as ext4 while still retaining the compatibility (probably) by disabling the extents feature, and probably some other(s). Or, by mounting it as read-only so you make sure you don't write any ext4 structures to the disk.

So, as you see now, it's perfectly possible that the partition has changed the format without having to be re-formatted.

I have no idea. But I can tell you what would I do instead: update your Debian with a kernel that has support for ext4. That way you will not need to toy around with your partition.

I assume that the partition remains untouched, and that it would still be accessible from your Ubuntu installation (if it's still there).

Thanks. That cleared up a lot.

However, it doesn't explain why I can't see my old files in the /home partition.

And what about the encryption? The partition wasn't encrypted from the start but I told ubuntu to encrypt it. Is it possible that the system is expecting all files to be encrypted, and so doesn't see the old files that are still stored as unencrypted ext3 files?

I'll leave recovering individual files as a last resort, but photorec seems to be for jpg and mpeg files only. That wouldn't help much.

I won't be using that pc until I know what I'm going to do with that partition.

I am not sure I understand the question. But if you can't mount the drive, then you can't see its contents...

I have no idea what did you encrypt of how did you encrypt it, so I can't be sure. In any case, whatever you encrypted, encrypted is, and as such, you can't access it without the encryption key. That's the whole point of encryption, right? If it was as easy as installing a new distro and going straight into the partition then encryption would give you no security at all.

The way that on-disk encryption works is via a kernel module, which maps the encrypted drive into a device under /dev/mapper/<whatever>, you need to access the drive though that device and not the usual /dev/<whatever>, but only after having setup the partition properly with cryptsetup so the contents appears de-crypted to the rest of the system.

Maybe your only problem is that the partition is encrypted and you are not setting it up correctly. Tell us more about the encryption. If the partition is *truly* encrypted and you don't have the key, you can already say "bye bye".

I can't be sure at all about what the problem is, I only can point out possibilities, the picture is not clear to me at all.

Photorec, regardless of the name, can recover many kind of files, but it will be useless in an encrypted drive, unless you can decrypt it first.

First, and before you do anything else, do as i92guboj said: Make a binary image of the partition with the dd command (used very carefully!).

An example (as root)

This makes one (big) file with all the binary data on that partition.

Then you can mount that file to a directory, as though it was a disk partition, using the -o loop option to the mount command. Eg (and I don't use ext4, or encryption, so check the syntax:

This means that you can "play" with recovery options for that partition without actually touching the original partition at all. If you mess up, just dd the partition to a file again, remount that file, and start again.

This is MUCH safer than trying to rescue the original partition.

I have the feeling that you will be able to recover your data, but you'll need to find that encryption key and read up on how to unencrypt that "disk partition" (it's really a file, but it doesn't matter, it behaves as a disk partition). Without that key, all is lost, and that is why encryption is so effective.

Good luck.

I can't mount the partition on debian. I can however mount it on ubuntu, but when I do, I only see three directories that were created by ubuntu. I don't see my debian user directory which contains my old files.

/home is 393.2 GB, with 20.2 GB used.
It contains three folders, the sum of their sizes is 8.1 KB. 20 GB sounds right for my old files. The fact that the space is still reserved for them gives me hope that I can still fully recover them.

Once again, here is what I did to the partition: It was ext3 unencrypted in debian, and I used the ubuntu 9.10 live cd to install ubuntu, told it to use that partition as ext4, and encrypt it. My guess is, ubuntu assumed that the partition was already encrypted, and simply used it. It did not encrypt the existing files. correct me if I'm wrong, but encrypting 20 GB of files takes a long time. I don't think ubuntu did that.

So, my believe is that the partition now contains both encrypted files (new ones written by ubuntu) and unencrypted files (old debian files). Since the partition is mounted as encrypted, it will try to decrypt the unencrypted files too, which produces random results that the system cannot recognize.

Does this make sense?

Concerning the backup, will I need space as big as the whole partition? Or just the used space in it? If it is the first, I might prefer not to do it. I don't have anywhere near that much of free disk space.

Thanks for all the help. I usually struggle with computer problems alone, googling, reading, trying things, and pulling my hair when they don't work. I probably won't be doing that this time.

Please, if your data is important to you, don't attempt to mount the partition, only mount an image as above, before you make things a lot worse (and potentially, completely unrecoverable).

That's because you encrypted the partition, so this is essentially "normal behaviour".

That is (hopefully) because they have been encrypted (along with any newer files) when you asked for this.

With any luck, all your files will be in .encryptfs but you cannot see that until you have worked out how to unencrypt that directory. As I said, I don't use encryption, and I cannot help you with this. Others here on LQ will help you though. The really important thing is not to modify the real encrypted partition until you know what you are doing.

i92guboj gave you a very rational and readable post at #5. Please re-read it, and do some searching on anything you don't understand.

Please don't "guess" or "assume" anything when you are trying to recover from potential data loss. Investigate and analyse logically and carefully, making sure you are never in a position where the original, raw data might be compromised, overwritten, or edited badly. That's why we are asking you to take a dd image of the partition that is now encrypted.

The above means "Do NOT mount your encrypted partition. But please do mount the image you have taken of your encrypted partition". If you don't understand the implications of this statement, please say so, and someone will explain further.

Yes, of course you will. We have no idea how the filesystem allocated the files. So a dd image is the safer way to go.

If you do care about your data, then you had better buy, beg or borrow an external HDD that is big enough.

If you do not care about your data, why are you asking us to help you?

OK, I won't mount it anymore, but I did manage to mount it before. When I typed the decryption phrase wrong it refused to mount. But it did mount it when I typed the correct one. Doesn't that mean it is being decrypted correctly? I did that graphically from another ubuntu install, which means it should have set all the mount options correctly, since ubuntu itself encrypted that partition.


I happen to have a 500 GB external hard drive that is now unreadable for unknown reasons. I was trying to leave it as it is until I know what the problem is, but I guess the current problem is more likely to be solved. I'll just format that one and use it.

And, the files in that partition ARE important to me. I will have that tee that i92guboj talked about, and watch that movie, and make the dd copy.

Back.

The drive I talked about decided to work, now that I was about to wipe it out. I had to delete 250 GB of files, and then run dd which took over 16 hours to make the copy. During that time, ntfs-3g was using over 90% of my dual core cpu, leaving very little room for anything else.

Now I have the 429 GB copy. I tried to mount it with the commands:

sudo modprobe cryptoloop && sudo modprobe aes
sudo mount -o loop,encryption=aes ./Cryptimage.img /mnt/baddrive/

After typing the password, I get the error: Wrong file system. Does this mean that the password is wrong? Or is there some other problem?

When I mount it without the "encryption=aes", it mounts and I see the new files only. Ubuntu info says "some contents unreadable".