LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-03-2021, 02:19 AM   #1
elaterite
LQ Newbie
 
Registered: Nov 2007
Location: Reno, NV
Distribution: Linux Mint
Posts: 21

Rep: Reputation: 0
Partition & format hard drive mysterious used space


Hi -

I can't account for what's using a lot of extra space on my storage drives, both internal and external.

I am familiar with reserved-blocks-percentage, the need for that space for fragmentation reasons, and how to adjust it during or after partitioning.

Since these are all data storage drives, not system drives, I set the reserved-blocks-percentage to 1 percent rather than the default 5 percent.

Here's two examples of used space I don't understand. After freshly partitioning (with one partition) and formatting:

A 1TB drive reports the size to be 931.51GB, 1.94GB is used, 929.57GB is available.

A 2TB drive reports the size to be 1.82TB, 30.31GB is used, and 1.79TB is available.

Maybe I'm doing something wrong during partitioning and formatting?

To partition the drive:

Code:
sudo parted /dev/sdx
(Where x = the drive to be worked on.)

Code:
(parted) mklabel gpt
(To create the file system.)

Code:
(parted) mkpart [label] 0% 100%
(Where 'label' is the name I want to call the disk.)

Then I quit parted.

And then to format the drive:

Code:
sudo mkfs.ext4 -L [label] -m 1 /dev/sdx1
(Where label is the same label as used earlier and 'x' is the drive letter I'm working on.)

The OS lives on a separate drive and I'm using an EFI boot. There is one storage drive in my box that is has a MSDOS rather than GPT file system. I plan to change it to GPT soon. All drives are hard disks, not SSDs.

Thoughts? And thanks!
 
Old 12-03-2021, 02:33 AM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,160

Rep: Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125Reputation: 4125
Quote:
Originally Posted by elaterite View Post
A 1TB drive reports the size to be 931.51GB, 1.94GB is used, 929.57GB is available.
More likely it reports:
A 1TB drive reports the size to be 931.51GiB, 1.94GiB is used, 929.57GiB is available.

Search on gigabyte versus gibibyte - thousands before you have been misled similarly.
 
1 members found this post helpful.
Old 12-03-2021, 02:36 AM   #3
shruggy
Senior Member
 
Registered: Mar 2020
Posts: 3,688

Rep: Reputation: Disabled
I presume, you know of the difference between GB and GiB?

That said, don't forget that besides reserved-blocks-percentage some space is always used by filesystem metadata: journal, inode table, copies of the superblock, and so on.

Last edited by shruggy; 12-03-2021 at 02:46 AM.
 
1 members found this post helpful.
Old 12-03-2021, 08:29 AM   #4
HappyTux
Senior Member
 
Registered: Mar 2003
Location: Nova Scotia, Canada
Distribution: Debian AMD64
Posts: 4,170

Rep: Reputation: 244Reputation: 244Reputation: 244
You have run across the lie that the manufacturers use to trick you into thinking you have bought a larger drive drive than you actually have. And for the inevitable apologists for scummy behaviour that will appear defending the manufacturers. The day my computer starts computing in decimal instead of binary is the day they get to use a decimal system for everything in it, not the binary it actually is using. It is about 7% that is lost I think when comparing the real binary capacity with the decimal the liars use. They have even corrupted the standards organizations to tell their lies as the post above shows with the comparison of the real vs the fake abbreviations they have had adopted by them.
 
1 members found this post helpful.
Old 12-03-2021, 09:14 AM   #5
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,643

Rep: Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561
Quote:
Originally Posted by HappyTux View Post
They have even corrupted the standards organizations to tell their lies as the post above shows with the comparison of the real vs the fake abbreviations they have had adopted by them.
Did they have a time machine?

The International System of Units (aka SI) was established in 1960, which is when Giga- for 10^9 was adopted, whilst Mega- and Kilo- were already in use for 10^6 and 10^3 for a long time before that.

The fault for this mess lies with whichever imbeciles proceeded to mislabel 1024 bytes as a "kilobyte" in the first place (instead of deriving some other binary-based prefixes from the start). According to this Wikipedia timeline it may have been IBM in 1964.

 
2 members found this post helpful.
Old 12-03-2021, 12:00 PM   #6
elaterite
LQ Newbie
 
Registered: Nov 2007
Location: Reno, NV
Distribution: Linux Mint
Posts: 21

Original Poster
Rep: Reputation: 0
Welp, nope, I did not know the difference between a gigabyte (GB) and gibibyte (GiB).

I see there's been lawsuits over the issue as well.

I never paid much attention to the mysterious space loss until I started using these 2"TB" drives as the "loss" numbers were getting rather large. Also, I was using MSDOS file system with extended partitions so I thought it was likely OS / partitioning overhead.

This GB / GiB hard drive manufacturer's scam reminds me that a 2"x4" isn't really two inches by four inches.

Interestingly, Wikipedia says, "Western Digital settled the challenge and added explicit disclaimers to products that the usable capacity may differ from the advertised capacity." I went to WD's website before posting this question, because the hard drives in question are WDs, to check for fine print in the specifications. I saw no disclaimers.

Anyway, thanks for the education! I'll mark this thread, "Solved."

Last edited by elaterite; 12-03-2021 at 01:20 PM.
 
Old 12-03-2021, 12:12 PM   #7
smallpond
Senior Member
 
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: Fedora
Posts: 4,162

Rep: Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268
It might help to know that the first hard drives were designed for decimal computers . See https://en.wikipedia.org/wiki/IBM_30...C#Architecture
 
1 members found this post helpful.
Old 12-03-2021, 02:11 PM   #8
HappyTux
Senior Member
 
Registered: Mar 2003
Location: Nova Scotia, Canada
Distribution: Debian AMD64
Posts: 4,170

Rep: Reputation: 244Reputation: 244Reputation: 244
Quote:
Originally Posted by boughtonp View Post
Did they have a time machine?

The International System of Units (aka SI) was established in 1960, which is when Giga- for 10^9 was adopted, whilst Mega- and Kilo- were already in use for 10^6 and 10^3 for a long time before that.

The fault for this mess lies with whichever imbeciles proceeded to mislabel 1024 bytes as a "kilobyte" in the first place (instead of deriving some other binary-based prefixes from the start). According to this Wikipedia timeline it may have been IBM in 1964.

As I said there is always at least one defender of scummy behaviour.

Edit: And I would add if I had not actually lived at the time to have bought hard drives that gave you the full capacity advertised in BINARY. Until some slimy marketing ahole got wind of the idea hey we can lie and use a non-related unit of measurement that will make our drives seem larger. Time machine indeed to pull that foolishness out when a decimal measurement is clearly intended for a decimal system, binary is intended for binary.

Last edited by HappyTux; 12-03-2021 at 02:21 PM.
 
1 members found this post helpful.
Old 12-04-2021, 12:00 AM   #9
enigma9o7
Senior Member
 
Registered: Jul 2018
Location: Silicon Valley
Distribution: Bodhi Linux
Posts: 1,413

Rep: Reputation: 564Reputation: 564Reputation: 564Reputation: 564Reputation: 564Reputation: 564
I'll defend it too. This is something most people using computers discover pretty early. But kilo, mega, etc mean 10^3 10^6 first, as defined by SI and every other use case ever.

Just cuz 2^10 is close to 1000 bytes doesn't make it exactly one kilo-byte. Its 1024 bytes, or 1.024 kilobytes. When there's a big long number of bytes like 127,317,398,127 bytes I don't want to have to do math, I just wanna say 127.3 TB! Yes, MiB/KiB/GiB/TiB are useful of course, but those are different than megabyte, kilobyte, gigabyte, terabyte, which should use the SI prefixes normally.

This has been a noticeable issue since the dos days. Remember "1.44MB" floppy disks? They aren't 1.44MB or 1.44MiB, in that case, they are using 1000*1024! (10^3*2^10). Mixed case!

The only way to really be 100% sure is when someone actually just puts a big number with bytes at the end, or specifically says KiB/MiB/GiB etc. If you see MB or TB, you never can be sure what the person using it may mean, other than its at least 10^3/10^6/10^9 bytes and not more than 2^10/2^20/2^30 bytes.

Then, there's things that are sold with bandwith of megabits or gigabits. To confuse people who dunno bit from byte....

Last edited by enigma9o7; 12-04-2021 at 10:20 AM.
 
Old 12-04-2021, 10:01 AM   #10
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,643

Rep: Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561
Quote:
Originally Posted by HappyTux View Post
As I said there is always at least one defender of scummy behaviour.
Nothing in my post was defending behaviour of the manufacturers who misused the already-existing already-standardized prefixes, it was pointing out that "corrupting the standards organizations" is utter nonsense.

Whilst kibi/mebi/gibi are a tad irritating, they are logical and long overdue corrections to kilo/mega/giga.

Where we probably agree (and enigma907 appears to disagree) is that KiB/MiB/GiB abbreviations are unnecessary - the SI prefixes are lowercase, so we can safely stick with existing KB/MB/GB for the overwhelming majority of situations where the context is binary, be explicit in the rare instances that it's not, and report any deliberately misleading uses the same way as one would with other false advertising.

 
Old 12-04-2021, 10:03 AM   #11
shruggy
Senior Member
 
Registered: Mar 2020
Posts: 3,688

Rep: Reputation: Disabled
Quote:
Originally Posted by boughtonp View Post
the SI prefixes are lowercase
Nope. Only k for kilo is lowercase in SI.
 
Old 12-04-2021, 10:17 AM   #12
enigma9o7
Senior Member
 
Registered: Jul 2018
Location: Silicon Valley
Distribution: Bodhi Linux
Posts: 1,413

Rep: Reputation: 564Reputation: 564Reputation: 564Reputation: 564Reputation: 564Reputation: 564
Quote:
Originally Posted by shruggy View Post
Nope. Only k for kilo is lowercase in SI.
Actually the smaller ones too... milli, nano, pico, femto, etc, all lowercase. But yeah M for million/mega (10^6) must be capitol to distinguish it from m for milli (10^-3).
 
Old 12-04-2021, 10:24 AM   #13
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,643

Rep: Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561Reputation: 2561
Quote:
Originally Posted by shruggy View Post
Nope. Only k for kilo is lowercase in SI.
Oh yeah. That's irritating, but I still stand by the rest of what I said.

 
Old 09-21-2022, 11:07 AM   #14
Mr. Macintosh
Member
 
Registered: Sep 2015
Distribution: Debian
Posts: 297

Rep: Reputation: 60
Quote:
Originally Posted by enigma9o7 View Post
I'll defend it too. This is something most people using computers discover pretty early. But kilo, mega, etc mean 10^3 10^6 first, as defined by SI and every other use case ever.

Just cuz 2^10 is close to 1000 bytes doesn't make it exactly one kilo-byte. Its 1024 bytes, or 1.024 kilobytes. When there's a big long number of bytes like 127,317,398,127 bytes I don't want to have to do math, I just wanna say 127.3 TB! Yes, MiB/KiB/GiB/TiB are useful of course, but those are different than megabyte, kilobyte, gigabyte, terabyte, which should use the SI prefixes normally.

This has been a noticeable issue since the dos days. Remember "1.44MB" floppy disks? They aren't 1.44MB or 1.44MiB, in that case, they are using 1000*1024! (10^3*2^10). Mixed case!

The only way to really be 100% sure is when someone actually just puts a big number with bytes at the end, or specifically says KiB/MiB/GiB etc. If you see MB or TB, you never can be sure what the person using it may mean, other than its at least 10^3/10^6/10^9 bytes and not more than 2^10/2^20/2^30 bytes.

Then, there's things that are sold with bandwith of megabits or gigabits. To confuse people who dunno bit from byte....

Quote:
Then, there's things that are sold with bandwith of megabits or gigabits. To confuse people who dunno bit from byte....

That's really where I take issue. Maybe it's because I'm young and have only seen storage capacities reported by manufacturers and operating systems in decimal units that I have no problem with capacities being expressed that way, but using decimal units really muddies the waters for folks who are naively expecting transfer speeds to be expressed in decimal units. I don't care which units are used as long as they're consistent. If we're going to use decimal for one thing, we should use it for everything so things actually make sense. Otherwise, it's just dishonest.
 
  


Reply

Tags
efi, formatting, gpt, partition, partitioning



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Available space & used space of partition not tallying? ravisingh1 Linux - Hardware 4 04-27-2014 03:45 AM
Copying a partition from one hard drive to the same partition on another hard drive pylagan Linux - General 1 03-23-2010 12:02 PM
mysterious, hard drive intensive process after start up... lumix Linux - General 4 03-30-2007 02:32 PM
Phục hồi dữ liệu bị mất???, cứ pollsite General 1 06-27-2005 12:39 PM
Gotta love those ٱٱٱٱٱٱٱ&# iLLuSionZ Linux - General 5 11-18-2003 07:14 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 07:09 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration