[SOLVED] How to carve for (undelete) mistakenly removed files?

The situation (I blame lack of sleep :P ):

system ~ % sudo ls -lAi /media/Common/BACKUP/DEFT/
total 0
system ~ % sudo rm -rv /media/Common/
removed directory: `/media/Common/lost+found'
removed `/media/Common/etc/apparmor.d/usr.lib.dovecot.imap'
removed `/media/Common/etc/apparmor.d/lightdm-guest-session'
»»» snipped: ~7500 files & directories removed up to: «««
removed directory: `/media/Common/opt/sweets/calculate/locale/wa'
^Csystem ~ % ^C

I meant to delete /media/Common/BACKUP, not /media/Common ...

I've put that list of deleted files & directories into a file like so:

/media/Common/lost+found
/media/Common/etc/apparmor.d/usr.lib.dovecot.imap
/media/Common/etc/apparmor.d/lightdm-guest-session
/media/Common/etc/apparmor.d/usr.lib.dovecot.dovecot-auth
/media/Common/etc/apparmor.d/usr.sbin.nscd
/media/Common/etc/apparmor.d/force-complain
»»» Snip: 7488 files & directories in total ««««

So I'm trying to undelete those files and directories, so to speak. But I'm not really used to it.
I'm trying to use scalpel or foremost but so far to no success (due to mal-formed command line I think).
I understand those programs seem to work on a single file, like a dd dump. But shouldn't they work just as well on a mounted filesystem? I presume so, and it's likely just how I'm trying to feed what I want to carve that's getting me no results. I think...

I couldn't get foremost to work at all because of the apparently malformed "input feeding":

system ~ % foremost -d -v -k 6000 -i evidence/Ficheiros_apagados_POR_ENGANO_de_mediaCommon-FIN.lst -o carved/media/Common 
Foremost version 1.5.7 by Jesse Kornblum, Kris Kendall, and Nick Mikus
Audit File

Foremost started at Wed Aug 21 00:32:16 2013
Invocation: foremost -d -v -k 6000 -i evidence/Ficheiros_apagados_POR_ENGANO_de_mediaCommon-FIN.lst -o carved/media/Common 
Output directory: /home/user/carved/media/Common
Configuration file: /etc/foremost.conf
Processing: evidence/Ficheiros_apagados_POR_ENGANO_de_mediaCommon-FIN.lst
|------------------------------------------------------------------
File: evidence/Ficheiros_apagados_POR_ENGANO_de_mediaCommon-FIN.lst
Start: Wed Aug 21 00:32:16 2013
Length: 568 KB (582595 bytes)
 
Num	 Name (bs=512)	       Size	 File Offset	 Comment 

*|
Finish: Wed Aug 21 00:32:16 2013

0 FILES EXTRACTED
	
------------------------------------------------------------------

Foremost finished at Wed Aug 21 00:32:16 2013

As for scalpel, here's what I got:

system ~ % scalpel -i evidence/Ficheiros_apagados_POR_ENGANO_de_mediaCommon.lst -o ~/carved/media/Common -O
Scalpel version 1.60
Written by Golden G. Richard III, based on Foremost 0.69.

Opening target "/media/Common/lost+found'"

ERROR: Couldn't open input file: /media/Common/lost+found' -- No such file or directory
Scalpel was unable to open the image file: /media/Common/lost+found'
Skipping...


Opening target "/media/Common/etc"

ERROR: Couldn't open input file: /media/Common/etc/apparmor.d/usr.lib.dovecot.imap' -- No such file or directory
Scalpel was unable to open the image file: /media/Common/etc/apparmor.d/usr.lib.dovecot.imap'
Skipping...


Opening target "/media/Common/etc"

ERROR: Couldn't open input file: /media/Common/etc/apparmor.d/lightdm-guest-session' -- No such file or directory
Scalpel was unable to open the image file: /media/Common/etc/apparmor.d/lightdm-guest-session'
Skipping...


Opening target "/media/Common/etc"

ERROR: Couldn't open input file: /media/Common/etc/apparmor.d/usr.lib.dovecot.dovecot-auth' -- No such file or directory
Scalpel was unable to open the image file: /media/Common/etc/apparmor.d/usr.lib.dovecot.dovecot-auth'
Skipping...
««« And so on ... «««

So can someone please help me undelete the files from my /media/Common partition (mounted or not) with scalpel, foremost or whatever?
I haven't been using the partition since (and only got it autro-mounted once by mistake, likely nothing at all was written to it anyway).

Thanks in advance.

Originally Posted by jdackle (..) I'm trying to use scalpel or foremost but so far to no success (due to mal-formed command line I think). I understand those programs seem to work on a single file, like a dd dump. But shouldn't they work just as well on a mounted filesystem? I presume so, and it's likely just how I'm trying to feed what I want to carve that's getting me no results. I think...

First law of forensics (or so I'd call it): don't ever let your "evidence" be tainted so even auto-mounting a partition once (mistake or not doesn't matter) is bad and saying "likely nothing at all was written to it anyway" means nothing unless you know about journal replaying and ro, norecovery and noload mount flags. Working on a Live file system should only be done if no other option is available and then the first priority would be to create a 'dd' image of the drive.

IIRC (it's been a while since I used Scalpel or Foremost) the "-i file" switch in both foremost and scalpel mean a list of 'dd' images to examine and not a list of flies to recover. The reason why that wouldn't work is that by deleting files and directory structures file contents get severed from their meta data like names.

Originally Posted by jdackle So can someone please help me undelete the files from my /media/Common partition (mounted or not) with scalpel, foremost or whatever?

You could try 'extundelete --restore-all /dev/devicename' ('man extundelete' for more), boot a Live CD containing testdisk to see if you can walk the directory structure or try photorec. See http://www.cgsecurity.org/wiki/Data_Recovery_Examples, http://www.cgsecurity.org/wiki/TestDisk_Step_By_Step, http://www.cgsecurity.org/wiki/PhotoRec_Step_By_Step. YMMV(VM).

Originally Posted by unSpawn First law of forensics (or so I'd call it): don't ever let your "evidence" be tainted so even auto-mounting a partition once (mistake or not doesn't matter) is bad and saying "likely nothing at all was written to it anyway" means nothing unless you know about journal replaying and ro, norecovery and noload mount flags.

Ok. ***-whipping deserved and taken.
I do realise now how presumptuous that statement of mine was.
And that may very well be the reason why extundelete got me zero results (tried both with the --recover-all and --recover-files options).

Originally Posted by unSpawn Working on a Live file system should only be done if no other option is available and then the first priority would be to create a 'dd' image of the drive.

Well, doing that now. I've been using computers for years, Linux included, but I'm really just your regular no-fuss-please end-user kind of guy.

Originally Posted by unSpawn IIRC (it's been a while since I used Scalpel or Foremost) the "-i file" switch in both foremost and scalpel mean a list of 'dd' images to examine and not a list of flies to recover. The reason why that wouldn't work is that by deleting files and directory structures file contents get severed from their meta data like names.

Although that does make perfect sense, extundelete on the other hand specifically gives you the option to look for and undelete files by their filenames so...

Originally Posted by unSpawn You could try 'extundelete --restore-all /dev/devicename' ('man extundelete' for more), boot a Live CD containing testdisk to see if you can walk the directory structure or try photorec. See http://www.cgsecurity.org/wiki/Data_Recovery_Examples, http://www.cgsecurity.org/wiki/TestDisk_Step_By_Step, http://www.cgsecurity.org/wiki/PhotoRec_Step_By_Step. YMMV(VM).

Thanks for the extra tips. Years ago I did use a Photorec live cd and did manage to restore some lost pictures. I wasn't too sure it would work with other types of files too. For the moment, it's still one of my options.

For now, I'm waiting on dd (ddrescue actually) and then will try foremost and/or scalpel on the output file.

Originally Posted by jdackle Ok. ***-whipping deserved and taken.

Never intended that way.

Originally Posted by jdackle Although that does make perfect sense, extundelete on the other hand specifically gives you the option to look for and undelete files by their filenames so...

Heh, I'm not saying it couldn't have any use under the right circumstances...

Originally Posted by jdackle Thanks for the extra tips. Years ago I did use a Photorec live cd and did manage to restore some lost pictures. I wasn't too sure it would work with other types of files too. For the moment, it's still one of my options. For now, I'm waiting on dd (ddrescue actually) and then will try foremost and/or scalpel on the output file.

The problem with file carvers like Photorec, Scalpel and Foremost is they need the files header and footer to work with (as in 'man magic') so they may well miss a file boundary (if the file doesn't have a footer signature), mistake contents of another file as part of it (due to indirect block allocation) or just fail to recover a file if there aren't any signatures. So apart from differences in recovery techniques I'd say the best maintained application with the largest file signature database should offer the best chance of recovery. But even if you manage to recover files there is no guarantee they're the files you're looking for or if they're still usable.
In short: YMMV(VM).

Originally Posted by unSpawn Never intended that way.

Not saying you meant it that way. But your comment made it clear to me how mis-funded my presumptions were... No offense taken anyway.

Originally Posted by unSpawn The problem with file carvers like Photorec, Scalpel and Foremost is they need the files header and footer to work with (as in 'man magic') so they may well miss a file boundary (if the file doesn't have a footer signature), mistake contents of another file as part of it (due to indirect block allocation) or just fail to recover a file if there aren't any signatures. So apart from differences in recovery techniques I'd say the best maintained application with the largest file signature database should offer the best chance of recovery. But even if you manage to recover files there is no guarantee they're the files you're looking for or if they're still usable. In short: YMMV(VM).

Well, Foremost mainly recovered my Firefox cache - pretty useless to me.
Scalpel went berserk. the original partition and dd_rescue dump I made of it were 16GB long. I ended up with an 82GB folder and it would have gone on but stopped because of lack of space on the output directory/partition. I don't really know why but something else I tried might explain it:

It is possible to recover multiple copies of some of your files if you deleted that file more than once; you'll need to decide which is the one you want to keep.

(taken from: http://www.datarecoverypros.com/reco...commander.html )
I remembered Midnight Commander had that functionality so I tried it too. But I could not cd to undel://sdb2 nor undel///dev/sdb2 (no such folder exists). This may be because

First, this particular undelete trick only works for ext2 partitions.

(from the same article). I wasn't too sure that meant ext2 only and no ext3/ext4 or whether it might include those enhancements on the ext2 filesystem. Undeletion through Midnight Commander probably only supports ext2 - not ext3 nor ext4 - as it did not use journaling which was only implemented on ext3.

I could try Testdisk but my past tries and reads on this subject leads me to think I would likely just end up with a bunch of files I would have to look into one by one to try and get (only) some of them right.
So I guess I'll go for the saner way of getting my somewhat out of date backup of that data + resintall some default settings from the program packages (most of what was on thet partition were setttings and configurations) + retweak those settings where needed + try and use the method below to recover the few files I really need restored (first thing to do):
Recovering Deleted Files in UNIX on non-ext2 partitions: http://www.datarecoverypros.com/non-ext2-recovery.html
I'll edit this post and report back when I'm done.

In the meantime, the original question about getting Foremost and Scalpel to run did get solved. I didn't get the results I was hoping for but, as unSpawn pointed out, that was mostly my fault on one side and the limitations of those tools on the other.
But anyway, I did get those programs working so I'm marking this thread as solved.
Thanks for the tips, unSpawn!


EDIT: I tested the grep method mentioned in the link posted above with a couple simple text files (shell scripts and apparmor profiles). The outpur was way too garbled and large for it to be practical, if at all useful, for my situation.
So I'm left with the much more traditional aproach of reinstall, restore backups, reconfigure, redo some work. Seems it will be the easiest and most effective way for me after all...

Originally Posted by jdackle Thanks for the tips, unSpawn!

You're welcome.

Originally Posted by jdackle EDIT: I tested the grep method mentioned in the link posted above with a couple simple text files (shell scripts and apparmor profiles). The outpur was way too garbled and large for it to be practical, if at all useful, for my situation.

I'm sorry to see you wasted time with that crap (pardon my French). Just because a domain name reads "datarecoverypros" doesn't necessarily mean they are and their "tutorials" are, to put it politely, way outdated.

Thanks for caring, unSpawn.