Hi folks, I think I've got a challenge for you. And I really need good help. If you can't help but know where to send me for help, please do.

I decided to take time today to put my old hard-drives to use by copying an installation of Windows 2000 (C:\ or hda1, but I don't use it anymore) and my Windows NTFS data partition (D:\ or hda6) onto the 40-GB and 80-GB drives, respectively. I have Windows 2000 and XP along with the NTFS data partition (D:\) and a couple shared partitions on one 200-GB drive, and all of my Linux installations and a couple of shared partitions on another 200-GB drive. I took out the Linux drive and added the 40-GB drive, and then worked at figuring out how to format the old drive under Knoppix and then copy the partition.

Things started well with the 40-GB drive and Windows 2000 partition. I just followed an online guide I found and used fdisk to create a partition and then dd to copy the data. I didn't test the result, because I wanted to copy my NTFS data partition first.

So I swapped out the 40-GB disk for the 80-GB disk, and followed the same procedure. However, the NTFS partition (D:\) was just slightly larger than the size of the 80-GB disk, so I decided to resize it. I got frustrated trying to use qtparted, and although ntfsresize performed its read tests well, it refused to carry out the actual resizing.

Although the drive with my Linux installations is in the slave position, the master drive MBR has GRUB, which looks to the /boot partition on the slave drive. In order to start Windows and use Partition Magic I needed the Linux drive, so I swapped the 80-GB drive for the Linux drive.

I started up Windows and looked at the hard-drive. It was nearly full, and I thought I might as well defragment it before copying. To defrag with Windows's built-in tool, I needed 15% free space; not a problem, I just looked for large data to delete. Found quite a bit but not enough, so I decided to first enlarge the partition, defrag, and then shrink it again to a size that would fit on the other disk.

This is where I went wrong.

My computer has been experiencing freezes at seemingly random times when running Windows. Linux has not had the problem, and I suspect either corrupt IDE drivers or a defect in the hard-disk with Windows. What did not register in my mind is that this might occur during the resizing operation.

And it did.

The machine sat for over a half an hour without making progress, and worried as I was I knew from experience I had no recourse but to reboot. This of course caused files that were being reindexed or moved to become corrupted.

I anxiously watched as it rebooted. I don't remember all details, but when I logged into Windows XP as administrator, the drive was still there. My most valued data on that partition is some 12,000+ digital photographs, only about 10,000 of which are backed up. The folder containing these files was visible, but selecting it in Explorer caused a message to be displayed that its contents were not available. I trembled, I very nearly cried.

I rebooted.

CHKDSK sprang into action, and displayed many thousands of messages, most of which were related to my precious digital photographs, along these lines:

I was gladened by this, but still quite worried. Clearly the system still knew about the files, but would they be in working order? When I again logged into Windows XP, the folders were now all there. The files are also displayed inside the folders, along with their size. But if I try to open the files, I get such messages as "Format of the file <filename> could not be determined." Obviously it is the digital photographs that are not available.

It's not as though my daughter has suffered, but the idea that I have lost the last six months' worth of photography ... troubles me greatly. I am a fool for not having backed up regularly. I'm an even greater fool for not simply copying the precious data to empty space on a partition on the Linux drive before resizing.

But what can I do now? I don't know enough about recovery to even get started, and I'm emotionally strained, not really yet in a position to think and make good decisions.

If this is purely a Windows problem, please someone guide me to a reliable, helpful Windows forum or other resource.

OK, first of all let me start by saying that you were foolish not to back up your data before undertaking your tasks... Honestly right now you have no one to blame but yourself.

However, i'm feeling sorry for you.

Let's see. Basically you are at a loss, possibly a complete one. And, possibly not. I've been dealing with data recovery for a long time, but usually with the case of orphaned files during chkdsk, the files in question are lost.

I'd start by running chkdsk /f on the drive in question and reboot so it runs. Most likely this will not fix the problem, however, in some cases it will.

Next, i'd start browsing through the recovered orphaned files and see if ANY at all recovered. You may notice that some of your photos are still there, however with different file names, etc... You may also notice that they are all corrupted, such as the case with the ones you tried to open by double-clicking.

"Orphaned" is a strange word because it is used in other computer related contexts, mainly web-based applications. Searching google will rise your hopes when you quickly glance at descriptions that state "find and repair orphaned files", but upon further investigation you will discover they are tools for web-servers and orphaned webpage files.

I've searched high and low this evening for a tool for you, but cannot find one. I've searched in the past too when I've run into this issue with my customers systems.

I have one last thought for you though.

Have you tried running 'foremost' on the drive from Linux? Foremost is a very handy tool for data recovery. It is mainly used for recovering files on drives that lost their file system information. It searches the disk for certain types of files that you specify in a .conf file. All of your typical photo formats are supported, such as gif, jpg, png, etc... The manual page is invaluable, so read it before using the software.

I have not recieved any customer systems with orphaned files in a long time so my theory is untested, but my thought on this is that 'foremost' possibly will detect all your JPEG's on the disk and write them out with the correct headers, footers and structures.

Try it... that's all I can say.

I'll check the status of this thread... good luck

Scott

Thanks very much for your reply. Fortunately, yesterday was my wife's 30th birthday and that gave me a good reason to cool down and not think of my technical woes.

Quite so. I think what bothers me most about that is the ease with which I could have prevented loss. There's plenty of unpartitioned space on my slave drive that I could have used to make sure the more recent photos are secure. And I've chastised others on more than one occasion for not backing up. In the back of my mind I knew there was a high risk, but I suppose I was impatient due to having limited time. Definitely foolish.

Well I thank your for your sympathy. There is some comfort in knowing that, of approximately 12,000 photos, around 10,000 are backed up. Sheer quantity doesn't say everything, because the missing ~2,000 do represent a 6-month period. (That's a real drop in activity, since the first 10,000 were made in less than a year.) Nearly as comforting is the knowledge that just two weeks ago I made an exhaustive thumbnailing project, reducing the original photos to a 640x480 resolution (originals are typically 2336x3504) and organizing them by month. I also transferred the result to my work computer, so I at least have an untouched thumbnail archive. I can see what I shot, even if it's greatly reduced.

I understand that reformatting a hard-drive is insufficient to protect its data from being recovered by a professional, and that, due to the nature of magnetic storage, even re-writing it with zeros might not be enough to get rid of data if someone has expensive equipment and sufficient expertise. So I reasoned that, theoretically, someone out there should be able to recover those files. But it's a question of effort, cost and time, so the question really is what can I do with my limited resources and knowledge.

From Partition Magic (7.0, an older version), I had it check the partition for errors. It didn't carry the operation out, returning an error (913 or something similar) and recommended I use the system tools to check the partition. After a while, but before I did any other operations, I used the same Parition Magic tool again to check the partition for errors, and this time it didn't complain, instead going ahead with the check and reporting no errors found. Then I ran chkdsk with no options, and it reported no errors. I also ran chkdsk /f and it of course reported no errors.

Of course, chkdsk had already been run shortly after the interrupted resizing operation, that is, when I rebooted as I previously wrote. This is when I got the messages about orphaned files.

From all this I conclude that chkdsk has already done the best job it could to recover the files; however, the best it could do for the files in question was to (in my imprecise interpretation of terminology) re-index the files so that some but not all information was recovered. In other words, it was able to correctly restore lost folder structures along with their contents, so that files appear in the correct location with the correct name and even other data, such as how large the file was. But somehow the actual file contents were not recovered. You write of "headers, footers and structures" --- perhaps the structures are what is wrong, while headers and footers are fine. This is not so for all files, as I will describe next. (I'm sure I'll be learning a lot more about file systems and file structures in the coming week, so my terminology should improve.)

Certainly the contents of the partition are not all lost. I am able to open some documents meaningfully, and others not. However, until I find it I have no list of which files were orphaned. At the time when the resize operation got hung, it showed only 8% finished, so it stands to reason that many files were not touched, and most of those that were might have been messed up indirectly. In any case, I'm not sure where to find a log file of the chkdsk operation. I looked under Administrative Tools > Event Viewer, and looked at a "Winlogon" entry under "Application", but I'm sure I didn't look in the right place because I found no references to orphaned files. I don't know whether this is the only place a log might have been written, so I will follow this up as soon as possible.

So to summarize, some files are corrupted while others are not, and so far I haven't discovered any missing files or files with wrong file names, just corrupted contents. But at present I have no list of orphaned files. That chkdsk log report, if it was created, would be useful to have. (Actually, I have been assuming that it was chkdsk that was trying to recover those orphaned files; I need to find out how to check logs to verify what happened.)

Your thought about foremost seems to me quite a good one. I will read up on the documentation and get a clue before proceeding.

"All's well that ends well." I hope that this will turn out to be a fun learning experience. I certainly intend to burn DVDs much more regularly, and I am considering a new strategy. Simpler is usually better --- I think I'll make copies of new photos on a partition of the slave drive, so that there are always at least two copies of each file. Then I can delete them once they are on DVD.

Your checking the correct entry type (WINLOGON). Make sure you're checking the right one however. I also believe the System log has entries from the chkdsk procedure.

Please let me know how it goes. I wish I had a drive here with orphaned files so I could test for you.

Scott

In searching for tools, I came across several softwares designed to rescue files using both filesystem info and raw scanning. I focused on rescuing the photo images, even though other valuable file types may also be botched (still not sure of the extent of the damage):
http://graphicssoft.about.com/od/digitalphotorescue/


Most of these are designed to work with external media like CompactFlash, SmartMedia, etc., but also work with HDD. I tried a few from that list, but none of the demos I tried would actually rescue files, only display them. The one I liked best (though not on that list) is Data Recovery Wizard from Easeus. They discuss why I shouldn't have let CHKDSK run: Easeus Scandisk/CHKDSK Of course, I didn't know that at the time.

This program nicely displays found, lost and deleted files including their filenames and hierarchy. But since the OS is also displaying the files I'm not confident that the program would rescue the files correctly with their filenames and locations. It doesn't display a correct thumnail of the images. So since CHKDSK tried to fix things, I bet that the file data is no longer correctly associated with the file index, as we discussed. Since I'm forced to save money right now, I suppose I'm not really willing to spend $70 on a Windows-only tool just to test whether that's right or wrong.

What I've done so far is to use PhotoRec, one of the open-source programs listed. It recovers a surprisingly wide variety of archive, multimedia, office and other formats, and works by scanning the raw data read-only. I let it run overnight, and it extracted 223,000+ files to a new partition on my slave drive. Although the original drive should only contain ~68,000 files, over 115,000 of the found files are (bogus) .txt files, .jsp files and ".h" files. There are also many double or triple copies of files, since it also finds deleted files.

The program found about 48,000 JPG files, including 17,000 which I've identified as coming from digital cameras. In order to organize them, I sorted and moved them according to file size. Let me tell you, Explorer is a poor tool for sorting and moving large numbers of files; I missed the *NIX CLI. I then used Irfanview (great freeware graphics program) to extract EXIF information from the JPG files, so that they have the following format:The first part is the original filename assigned by PhotoRec, and the second two are gleaned from the EXIF information. I created a directory structure according to year and month, and moved by hand the files that were according to the EXIF info created since the files I have backed up. The EXIF info has of course proved trustworthy. I have sorted files dated from November 2005 right up to July 2, 2006.

All I have to do now for these files is to eliminate the multiple copies, and possibly do further sorting to match the usual folder hierarchy. I can still compare to my reduced-size images. After that, I need to assess the other damage more fully, such as documents, archives, and various projects. I don't expect any big problems, considering I backed up most important projects last year.

I have touched the botched partition as little as possible. (Some Thumbs.db files may have been created while displaying viewing folder contents.) My plan is to find, create or borrow HDD space and copy (clone) the botched partition bit-for-bit. That way I can begin to mess with the current partition and still go back and do another rescue operation when I get more info or can afford software that might restore the partition more fully. It's a lot of work to sort through 68,000 files to see which ones still work and which ones don't.

So far, things are looking fairly good. It's just a lot of work.