I bought a brand-new Western Digital MyBook 3.7 TB drive, but because I didn't have a USB port faster than 2.0, and in my wuest for speed of access, I decided to mount the drive internally. I took the Hitachi HDS5C4040ALE630 4TB drive physically out of housing, and installed into my computer with SATA. Of course, using Windows 7 64-bit on an HP XW8400 workstation, it only showed up as a 2 TB drive, because it was formatted using MBR. So after a quick google I discover that should be using GPT, and I also should use 1024 instead of 512 for the sector size, to double the size of the drive for the same address capacity.

I formatted it using diskpart.exe commands:
C:\> diskpart.exe
Diskpart> list disk
Diskpart> select disk [disk#]
Diskpart> clean
Diskpart> convert GPT
Diskpart> exit
C:\> exit

and then I went back into Windows 7 disk manager, and made one big partition 3.7 terabytes in size. Reformatting the drive to GPT with 1024, it shows up as a 3.7 TB drive.

I start using it happily, no problems, for about a week. Copying lots of files onto the drive, in a one stream at a time, sequential type fashion, so as not to be making lots of fragmented files, I ultimately crossed the 2 TB line. As soon as I did this, the drive access speed seemed to go way down and the copy process became very slow. So I canceled the copy process only to find that my drive is no longer there, shows up as an unformatted drive... "Do you want to format this drive?"

After a few minutes of panicking, rebooting, trying it on XP as opposed to Windows 7, no joy, I google the problem a few times and find that I have gone past the 2 TB limit and the drive has gone back to RAW status, Which is exactly what the drive manager says. RAW, 3.7 TB, Unallocated, 0 KB used.

I google for solutions, and I am recommended to use the Linux PartedMagic ISO, burn it to a CD, and boot off it, because Linux can recognize greater than 2 TB drives. I tried this, it scanned for half a day, and ultimately told me "the hard disk (4000 GB /3726 GiB) seems too small! (<8001 GB / 7451 GiB). Check the harddisk size: HD jumper settings, BIOS detection..." it went on to say "The following partition can't be recovered:
"> MS DATA Start 7814035455 End 15627806718 Size in sectors 7813771264"

So then I googled again and I tried partition guru, running in Windows, which searched for another day and found thousands and thousands of lost files, all of which have lost their names, and are replaced with numbers (000001.MTS, 000002.MTS.., and it also found about 1% of the directory structure with files correctly filed and named, and those files appearing to be intact. It seemed that these files are some of the last ones that were copied to the drive. I didn't bother to pay the money and restore the 1% of the directory structure, because they were some of the least important files.

Then I tried Active@ Partition Recovery. Scanning was much faster, but it found the same results of the intact directory structure, not listing all the lost and garbled files.

It looks like when I continued copying over the 2 TB limit, the data started being written over the first sectors of the disk, deleting the partition table, the file structure, and just going right ahead and overwriting all the important information of my drive until I decided it was going slowly, and canceled the copy operation, this may have been a good five minutes of overwriting, and hence, these programs can't find what my drive used to look like. This may be why the only directory structure which does show up is some of the last files I was copying.

Does this make sense? Is the directory structure stored at the beginning of the disk partition, and thus has been overwritten? Or should the scanning programs be able to find more directory structure than just five minutes of copying? I know that it took most of 2 days to write all the files on to it.

I know I'm not the only person who has ever experienced this, other people have had similar issues. It seems to be a fairly common problem because the motherboard and the hard drive driver also need to be able to handle more than 2 TB to avoid this from happening (I know now), but none of them seemed to actually post if they were able to recover from this or if it's been just doomsday from that point forward.

Is it true that this file structure information is only stored in one place on the drive? or is there any other record of this information anywhere else that can be used to restructure the beginning of the drive?

Most of the scanning applications have told me that the sector size is actually 512 instead of 1024. I have been seeing if I can find a way to tell them that the sector was actually 1024 instead of 512, but I don't seem to find any options to change the scanning functions to look for that instead. Would this make a difference?

Have I lost everything? Is it possible to re-derive the partition information and directory structure from anywhere else in the drive?

I have the applications to rewrite the boot sector manually, however I don't know what !@#$%^&*() means, or what it should say.

I have one of those drives ( using usb2 ) formatted to ext4
using gparted and all default settings
the 3Tb is in fact the PR department listing it is really about 2.7 Tb


there is also a "hidden" ( to windows software) a OEM partition
with "back up MS windows software

for linux only
just reformat the whole drive and remove the MS windows only programs
I would use Gparted
It is what i use , and did use .


BUT
this is a Linux forum and NOT a MS windows forum

if this is to be used on a Windows ONLY install
use the windows7 partition tools that are part of win7
the drive is formatted using NTFS out of the factory
with a hidden "recovery" partition ( hidden to windows)

Thanks for the interesting answer.

I'm really only putting this on a Linux forum to see if anybody knows a way to get the partition table or directory structure back with Linux software.

This other partition which you mention does show up in the scanning programs, but it shows up as being completely empty and not as having anything recognizable in it. So I'm not sure what I can take out of it to replace the missing partition information or directory structure in my large partition. Additionally, if Windows cannot see it, then it wouldn't be writing any directory information and what would it? I guess that explains why it's empty.

This drive was really only for backing up photos, videos, and text files, there aren't any operating systems or programs on it. But that makes me curious as to why you mentioned to remove all the MS Windows only programs from it. And as for reformatting the whole drive, isn't that going to defeat the purpose of me trying to retrieve data from it?

this is a list of the backup and recovery software and folders
------
Extras --folder
System Volume Information --folder
User Manuals --folder
WD Quick Formatter.exe
WD SmartWare --folder
WD SmartWare.exe
WD SmartWare for Mac --folder

the drive was in NTFS format out of the factory
then you reformatted it using
"diskpart.exe"

that right there might have killed the recovery partition and the reformatting might have left a 1Mb blank space at the start of the drive

as to
yes BUT maybe no

but with using MS's GPT for there interpretation of uefi
from the msdn page
http://msdn.microsoft.com/en-us/libr.../gg463525.aspx

this needs to be fixed with Microsoft tools

unless one of the experts for the experts here knows of a way

I think you're right, thanks for your time.

I will keep trying, I really don't want to lose this drive.

Just in case anybody ever decides to install a 2.7 TB drive, 3 TB drive, 4 TB drive or anything greater into Windows 7, hopefully this thread will show up on their search. If you want to install a drive greater than 2 TB into Windows 7, even if you do use GPT instead of MBR, and even if your drive does show up as having more than 2 TB of space in drive management, you have to make damn sure that the driver for the motherboard can handle addresses higher than 2 TB, and you also have to make sure that the generic driver for the hard disk drive can handle addresses higher than 2 TB, otherwise you're running at the same thing as me, when you fill your drive above the 2 TB limit, your data will wrap around to the beginning and overwrite your partition data, your directory structure, and you will lose everything on that drive.

The best way to achieve this is to partition it into two or more sections which are less than 2 TB each, and then make it a dynamic drive and make them into one drive letter.

I know this doesn't belong on a Linux site, but I'm only saying it for reasons of public service.

It is always a good idea to split such an insanely large harddisk into multiple partitions. What if the one 4TB partition has defective sectors and you want to create an image using a tool like gddrescue - where to store a 4TB file? And if done right, you won't let any file system "repair" program doing even more damage on the original image, but instead you want a copy of that image - another 4TB file. Then you eventually want to let your "rescue" utility restore the files from the working copy of your image, and if the partition was filled to the max. with files, where to store the 4TB of rescued files? That's 12 TB temporary space required...

Think about the insanity of using such oversized partitions and choose a split ala 4x 1TB or even smaller. The largest partition i ever created was 680GB, so i can store a triple-copy of it on a 2TB harddisk i would buy for temporary use in case of such an accident. my $0.02

The vendor and the software you've used are both right.

3*1000*1000*1000*1000 = 3 TB
2.7*1024*1024*1024*1024 = 2.7 TiB ~= 2.97 TB (your software rounded slightly down to 2.7)

Fair enough.

Now as you quoted it, i meant to say 660 GB, not 680. Shall fit 3x on 2 TB (whoops).

Do you have any evidence that this would help?

I don't know exactly where the wrap around bug is that caused your problem, but it is very hard to imagine that bug is in a place where the partition size (rather than the whole disk size) is the trigger.

From the math, as they say that more than 2.2 TB is a nogo on msdos partition schemes, this quickly becomes logical if you multiply 2.2*1000*1000*1000*1000 and then divide that number by the sector size of 512 bytes, which is 4,296,875,000 which is exactly just a little bit bigger than the maximum possible 32-bit integer number (4,294,967,296).

So it's easy to see that we're dealing with a 32-bit sector number overflow (regardless of the fact that you are using GPT, this is just the math behind why around 2TB this starts to happen). In case you've used a 4TB partition, the sector number for the partition overflowed. You could use two 2TB partitions to test if the 2nd partition would behave the same way, i.e., causing mayhem, or not. If not, it was a "partition too big" problem. If this still doesn't work, then it's an entire-device-too-big problem. My guess would be that a 64-bit Windows is more likely to be able to handle a 4TB harddisk than a 4TB partition.

Boy am i glad not to use Windows anymore.. ext4 can handle partitions up to 1 exbibyte (EiB) (1152922 TB).

Moved: This thread is more suitable in <General> and has been moved accordingly to help your thread/question get the exposure it deserves.

Thanks.