Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Hi all. I have been using a lot of forensic tools lately. I recently formatted a 32GB ext4 flash drive, and was able to recover about 80-90% of the files and even other ones I had deleted before using some of the linux tools out there. I notice that the imaging is just mostly disk-intensive or "transfer-intensive" meaning I don't know if it uses alot of CPU, but extraction of data does. I have been using my laptop with a drive bay via USB for the source drive (external hard drives) and another external hard drive to put the image onto, and then extract onto a different folder. I know its a horrible setup, but I am in the process of deciding which parts to get for a build as I do mostly forensic work so I need a machine devoted to just that.
Do tools like ddrescue and other dd-related tools use alot of CPU or GPU? Also if anyone is in the know, do most serious forensic experts use Windows tools like EnCase for extraction? I was hoping I wouldn't have to resort to any of those tools to do my work since I can't really afford $20k for EnCase and other software. But besides photorec, most of the tools destroy any evidence of the timestamp (when the file was actually created)
If anyone can let me know if this machine below in the X79 Setup would be overkill for what I need to know, I would really appreciate it. (I don't plan to get the processor, but the same Sandy Bridge in quad-core which costs $299).
dd and ddrescue don't use much CPU and no GPU (obviously).
What you need most is good throughput to the disk, so I would recommend mounting HDDs internally, not via USB.
The only way to recover data such a file names, modification times, etc. is to be able to recover filesystem data. Testdisk can do this if it can find this data. Other utilities like foremost don't look for such data, and so do not get it.
Last edited by H_TeXMeX_H; 03-03-2013 at 03:58 AM.
I agree with mounting the HDs internally but what if I am working on hard drives after hard drives? I am getting into data recovery and forensics work now. Would I just leave the case open? I can probably only fit a couple at a time since the motherboard and mid-tower I am looking to buy only support 6 drives and 3 will be occupied (SSD as /, a 2TB SATA as /home, and a 2TB SATA or more for the actual data to be extracted). I don't store much data so I'd probably use the one I have /home mounted on to store the actual images since my root SSD will only be 120GB. This would mean I would have to shut down the machine at least between imaging 2 or 3 drives. I would imagine this is what some forensic experts do.
Another question I have is, strictly for imaging and extracting from drives, do I need to spend a lot on a new build? I do see myself purchasing 2 systems, one to be dedicated to imaging and extracting and another as my main workstation. How far do I need to go with the dedicated forensics machine? Would 32GB be better than 8 in this case? I can see hard drives being the main expense here. Any help would be appreciated. Thanks.
Last edited by rootaccess; 03-04-2013 at 02:53 AM.
You know better what your needs are, but I would try to cut costs. I would get just one large HDD and put my system on that, as well as for storing carved data. It's going to be cheaper than 1 SSD and 2 HDDs.
Yes I did look at eSATA. Seems to be the better of the SATA 1 and SATA II, but now there is a SATA III. There is an external enclosure for eSATA and USB 2.0 but only does SATA drives. I know most people are now using SATA drives but there may sometimes be those with IDE disks. I do have an enclosure for that as mentioned earlier but this does USB 2.0 OR regular SATA. For those transfers (I would assume older IDE drives are not that big anyways), I would just use the regular SATA cable. I guess my question now is, of all the different types of adapters, which one is the fastest: eSATA, SATA II, USB 3.0 or FIREWIRE? I guess any of these would be faster than mounting the drives internally? Because I really don't mind doing that if my throughput would be that much faster.
As far as the other drives, I forgot that this machine would be a dedicated forensics machine for a second as I was originally going with only 1 machine, so you are right. I will only use 1 big hard drive, around 3TB if they don't crash or have issues and just partition my system to around maybe 500GB and the rest for the data.
Do tools like ddrescue and other dd-related tools use alot of CPU or GPU?
CPU as in the VFS pushing chunked writes to the other medium? No, not really. Obviously working with malfunctioning disk HW will strain the system as the kernel tries to work around and correct errors (if possible and if that's wanted).
Quote:
Originally Posted by rootaccess
Also if anyone is in the know, do most serious forensic experts use Windows tools like EnCase for extraction? (..) But besides photorec, most of the tools destroy any evidence of the timestamp (when the file was actually created)
What you use, be it Encase, Forensic Tool Kit, X-ways or OSS like the Sleuthkit, pyFLAG or anything else, depends on the purpose and your understanding of the tools at hand. Wrt time stamps you best first ensure you have a proper understanding of file system specs (inodes, block allocation etc).
There is an external enclosure for eSATA and USB 2.0 but only does SATA drives. I know most people are now using SATA drives but there may sometimes be those with IDE disks.
For comfortable disk handling I would look for a docking station (provided it has a fast connection like eSATA) or a 5.25" hot-swap rack. These would allow an excellent transfer rate. For example something like this:
I don't know if there still are such devices for IDE disks, at least with fast SATA/eSATA connectors. As an unexpensive solution I would pull an IDE data cable outside the box through an open slot, along with a molex power connector, and set up an ESD-safe place for the drive. If your motherboard has only SATA ports inside you'll have to add a card with one or more IDE channels. This would be faster than with USB, and you could leave your box closed.
Quote:
Originally Posted by rootaccess
I guess my question now is, of all the different types of adapters, which one is the fastest: eSATA, SATA II, USB 3.0 or FIREWIRE?
Thanks for that. I did get a desktop dedicated to just recovery but I will need multiple desktops anyway to work on multiple drives. Leaving the cord out is a good idea. I do have an enclosure that takes SATA. It doesnt mention if its sata 1 or 2 or 3, and it isn't eSATA either. But I can't do more than one drive at a time on any given machine so leaving the cord out is fine for me. I am only making an image out of it, really. I haven't tried the SATA cable from the enclosure but I probably would leave that for drives under 200GB since even with the SATA cable, that will probably take longer than mounting it internally.
So that takes care of that, much appreciated. I was wondering if anyone here has worked with forensics before since I am wanting to get into this field. I am guessing that if I wanted to get some real information, the free linux tools aren't going to give me the whole lot of information about the files, just the actual files themselves, if they recover them. I was hoping I didn't have to spend thousands for software but I guess thats the price of the investment if I want to offer this service.
I've never used EnCase, but looking at the features they list on the site, most of it you can do with FLOSS tools. Now, if you're going to be working as a professional you may have to buy it anyway, but not because it can do more.
I was wondering if anyone here has worked with forensics before since I am wanting to get into this field.
There's very few people on LQ who have practical lab and field experience. I know only one who worked at an accredited forensics firm.
Quote:
Originally Posted by rootaccess
I am guessing
Guessing may be entertaining but IMHO it is one of the first things to un-learn.
Quote:
Originally Posted by rootaccess
(..) if I wanted to get some real information, the free linux tools aren't going to give me the whole lot of information about the files, just the actual files themselves, if they recover them.
Define "real information"? Please be clear.
Quote:
Originally Posted by rootaccess
I was hoping I didn't have to spend thousands for software but I guess thats the price of the investment if I want to offer this service.
Tools are expensive but they're only tools. If it's your own money you'll invest my advice would be to either not call it "forensics" but "recovery service" or else, if you really want to get into forensics, first look at what you actually need in terms of knowledge ("using a lot of forensic tools lately" doesn't equal forensics knowledge amassed over years), more importantly: accreditation (it being cheaper than litigation) and if possible try to align yourself with an established forensics firm any way you can.
Great point and you are correct. I cannot call it forensics but rather recovery service. At the very least, I may call it forensics as I believe data recovery is somewhat forensics especially if its some data besides just pictures, but I will not be performing any "forensic" work. I also thought it was interesting from reading an article last night that a couple members experimented with EnCase to find it was just as vulnerable to exploitation and manipulation as pretty much any other GUI tool. It almost makes you wonder how accurate some of these data findings really are. If a hacker can plant files on a system to obfuscate the investigation and launch code execution on the investigator's machine, how "good" can these programs really be? There are flaws in every program, by design.
I cannot call it forensics but rather recovery service. At the very least, I may call it forensics as I believe data recovery is somewhat forensics especially if its some data besides just pictures, but I will not be performing any "forensic" work.
I would define data recovery as the Art of, well, recovering data, whereas computer forensics (and I can only speak from my own experiences) encompasses everything leading up to evidence and findings admissible in court. Until you know what this actually means I would strongly suggest you market your services as just "data recovery". If you don't you may find out why I said accreditation is cheaper than litigation the hard way.
Quote:
Originally Posted by rootaccess
I also thought it was interesting from reading an article last night that a couple members experimented with EnCase to find it was just as vulnerable to exploitation and manipulation as pretty much any other GUI tool. It almost makes you wonder how accurate some of these data findings really are. If a hacker can plant files on a system to obfuscate the investigation and launch code execution on the investigator's machine, how "good" can these programs really be? There are flaws in every program, by design.
While it sure is important to understand the risks and limitations of your platform(s) and tools of choice, that any action must be reproducible and that any result must be verifiable I wouldn't give too much thought to such intricacies right now. It definitely is not one of your top priorities at this stage if you don't mind me saying.
I do understand that EnCase is more of a forensic tool but the added benefit of using it is the file names of those deleted files are kept in tact as well as the actual timestamp of the document, when it was deleted or accessed and this is great even for recovery. If I am looking for a specific document, it is easier to find it that way rather than grepping or having to resort to the Sleuth Kit using sigfind. I am actually learning the Sleuth Kit right now but have not experimented with it. So I'm not sure if it also offers the extra features I am interested in, mainly that the file name is kept in tact and the timestamp. Photorec was able to keep the timestamps, but not foremost or any of the other few tools Ive tried so far.
...and that's why I said that if you really want to get into forensics you have to first look at what you need in terms of knowledge. Knowing common file systems and their limitations should be one considered of the basics. BTW Honeypot's SOM, one Sourceforge project (forgot the name) and IIRC NIST provide a host of dd images which you can use for the purpose of skill / tool evaluation.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.