[SOLVED] Partition & format hard drive mysterious used space
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Partition & format hard drive mysterious used space
Hi -
I can't account for what's using a lot of extra space on my storage drives, both internal and external.
I am familiar with reserved-blocks-percentage, the need for that space for fragmentation reasons, and how to adjust it during or after partitioning.
Since these are all data storage drives, not system drives, I set the reserved-blocks-percentage to 1 percent rather than the default 5 percent.
Here's two examples of used space I don't understand. After freshly partitioning (with one partition) and formatting:
A 1TB drive reports the size to be 931.51GB, 1.94GB is used, 929.57GB is available.
A 2TB drive reports the size to be 1.82TB, 30.31GB is used, and 1.79TB is available.
Maybe I'm doing something wrong during partitioning and formatting?
To partition the drive:
Code:
sudo parted /dev/sdx
(Where x = the drive to be worked on.)
Code:
(parted) mklabel gpt
(To create the file system.)
Code:
(parted) mkpart [label] 0% 100%
(Where 'label' is the name I want to call the disk.)
Then I quit parted.
And then to format the drive:
Code:
sudo mkfs.ext4 -L [label] -m 1 /dev/sdx1
(Where label is the same label as used earlier and 'x' is the drive letter I'm working on.)
The OS lives on a separate drive and I'm using an EFI boot. There is one storage drive in my box that is has a MSDOS rather than GPT file system. I plan to change it to GPT soon. All drives are hard disks, not SSDs.
That said, don't forget that besides reserved-blocks-percentage some space is always used by filesystem metadata: journal, inode table, copies of the superblock, and so on.
You have run across the lie that the manufacturers use to trick you into thinking you have bought a larger drive drive than you actually have. And for the inevitable apologists for scummy behaviour that will appear defending the manufacturers. The day my computer starts computing in decimal instead of binary is the day they get to use a decimal system for everything in it, not the binary it actually is using. It is about 7% that is lost I think when comparing the real binary capacity with the decimal the liars use. They have even corrupted the standards organizations to tell their lies as the post above shows with the comparison of the real vs the fake abbreviations they have had adopted by them.
They have even corrupted the standards organizations to tell their lies as the post above shows with the comparison of the real vs the fake abbreviations they have had adopted by them.
Did they have a time machine?
The International System of Units (aka SI) was established in 1960, which is when Giga- for 10^9 was adopted, whilst Mega- and Kilo- were already in use for 10^6 and 10^3 for a long time before that.
The fault for this mess lies with whichever imbeciles proceeded to mislabel 1024 bytes as a "kilobyte" in the first place (instead of deriving some other binary-based prefixes from the start). According to this Wikipedia timeline it may have been IBM in 1964.
Welp, nope, I did not know the difference between a gigabyte (GB) and gibibyte (GiB).
I see there's been lawsuits over the issue as well.
I never paid much attention to the mysterious space loss until I started using these 2"TB" drives as the "loss" numbers were getting rather large. Also, I was using MSDOS file system with extended partitions so I thought it was likely OS / partitioning overhead.
This GB / GiB hard drive manufacturer's scam reminds me that a 2"x4" isn't really two inches by four inches.
Interestingly, Wikipedia says, "Western Digital settled the challenge and added explicit disclaimers to products that the usable capacity may differ from the advertised capacity." I went to WD's website before posting this question, because the hard drives in question are WDs, to check for fine print in the specifications. I saw no disclaimers.
Anyway, thanks for the education! I'll mark this thread, "Solved."
The International System of Units (aka SI) was established in 1960, which is when Giga- for 10^9 was adopted, whilst Mega- and Kilo- were already in use for 10^6 and 10^3 for a long time before that.
The fault for this mess lies with whichever imbeciles proceeded to mislabel 1024 bytes as a "kilobyte" in the first place (instead of deriving some other binary-based prefixes from the start). According to this Wikipedia timeline it may have been IBM in 1964.
As I said there is always at least one defender of scummy behaviour.
Edit: And I would add if I had not actually lived at the time to have bought hard drives that gave you the full capacity advertised in BINARY. Until some slimy marketing ahole got wind of the idea hey we can lie and use a non-related unit of measurement that will make our drives seem larger. Time machine indeed to pull that foolishness out when a decimal measurement is clearly intended for a decimal system, binary is intended for binary.
I'll defend it too. This is something most people using computers discover pretty early. But kilo, mega, etc mean 10^3 10^6 first, as defined by SI and every other use case ever.
Just cuz 2^10 is close to 1000 bytes doesn't make it exactly one kilo-byte. Its 1024 bytes, or 1.024 kilobytes. When there's a big long number of bytes like 127,317,398,127 bytes I don't want to have to do math, I just wanna say 127.3 TB! Yes, MiB/KiB/GiB/TiB are useful of course, but those are different than megabyte, kilobyte, gigabyte, terabyte, which should use the SI prefixes normally.
This has been a noticeable issue since the dos days. Remember "1.44MB" floppy disks? They aren't 1.44MB or 1.44MiB, in that case, they are using 1000*1024! (10^3*2^10). Mixed case!
The only way to really be 100% sure is when someone actually just puts a big number with bytes at the end, or specifically says KiB/MiB/GiB etc. If you see MB or TB, you never can be sure what the person using it may mean, other than its at least 10^3/10^6/10^9 bytes and not more than 2^10/2^20/2^30 bytes.
Then, there's things that are sold with bandwith of megabits or gigabits. To confuse people who dunno bit from byte....
As I said there is always at least one defender of scummy behaviour.
Nothing in my post was defending behaviour of the manufacturers who misused the already-existing already-standardized prefixes, it was pointing out that "corrupting the standards organizations" is utter nonsense.
Whilst kibi/mebi/gibi are a tad irritating, they are logical and long overdue corrections to kilo/mega/giga.
Where we probably agree (and enigma907 appears to disagree) is that KiB/MiB/GiB abbreviations are unnecessary - the SI prefixes are lowercase, so we can safely stick with existing KB/MB/GB for the overwhelming majority of situations where the context is binary, be explicit in the rare instances that it's not, and report any deliberately misleading uses the same way as one would with other false advertising.
Actually the smaller ones too... milli, nano, pico, femto, etc, all lowercase. But yeah M for million/mega (10^6) must be capitol to distinguish it from m for milli (10^-3).
I'll defend it too. This is something most people using computers discover pretty early. But kilo, mega, etc mean 10^3 10^6 first, as defined by SI and every other use case ever.
Just cuz 2^10 is close to 1000 bytes doesn't make it exactly one kilo-byte. Its 1024 bytes, or 1.024 kilobytes. When there's a big long number of bytes like 127,317,398,127 bytes I don't want to have to do math, I just wanna say 127.3 TB! Yes, MiB/KiB/GiB/TiB are useful of course, but those are different than megabyte, kilobyte, gigabyte, terabyte, which should use the SI prefixes normally.
This has been a noticeable issue since the dos days. Remember "1.44MB" floppy disks? They aren't 1.44MB or 1.44MiB, in that case, they are using 1000*1024! (10^3*2^10). Mixed case!
The only way to really be 100% sure is when someone actually just puts a big number with bytes at the end, or specifically says KiB/MiB/GiB etc. If you see MB or TB, you never can be sure what the person using it may mean, other than its at least 10^3/10^6/10^9 bytes and not more than 2^10/2^20/2^30 bytes.
Then, there's things that are sold with bandwith of megabits or gigabits. To confuse people who dunno bit from byte....
Quote:
Then, there's things that are sold with bandwith of megabits or gigabits. To confuse people who dunno bit from byte....
That's really where I take issue. Maybe it's because I'm young and have only seen storage capacities reported by manufacturers and operating systems in decimal units that I have no problem with capacities being expressed that way, but using decimal units really muddies the waters for folks who are naively expecting transfer speeds to be expressed in decimal units. I don't care which units are used as long as they're consistent. If we're going to use decimal for one thing, we should use it for everything so things actually make sense. Otherwise, it's just dishonest.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.