LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-11-2015, 01:07 PM   #1
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
Why Am I Seeing Less Space on A Hard Disk After Formatting It?


I'm not sure where to start to look here or research this.

I added four new disks in VCenter for a VM, which is a RHEL Server

Disk 1: 500 GB
Disk 2: 1 TB
Disk 3: 1 TB
Disk 4: 1 TB

After addding them to the VM, power it on and check fdisk

Fdisk shows the following:

Code:
532 GB 
1045 GB 
1045 GB 
1045 GB
This shows more space then what was allocated in VCenter.

I then proceed to create a partitions on each, and then from create LVMs on them (pvcreate, vgcreate and lvcreate) and use up all of the space.

I then format them with ext3.

Once I'm done, I run df -ha, which shows way less space. I'm wondering why? Is is that the space is eaten up by the Hypervisor and the OS? Or basically how does this work or how can I explain this back to someone else, such as a manager?
 
Old 11-11-2015, 01:48 PM   #2
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
How much less space?

fdisk and drive manufacturers uses GB (base 10), df uses GiB (base 2)
532 GB = 495 GiB
1045 GB = 973 GiB

Also, ext defaults to 5% reserved sectors for root. Add up your used space and available space on df and it should be ~5% less than the total space. If you want to change the amount of reserved space use tune2fs, eg on a system with 4kiB blocks:
Code:
tune2fs -r 20000
would change the reserved space from 5% of the total capacity to ~80 MB (that's usually what I use on non-system drives).

Between the two of those you're looking at around a 15% drop from the drive's advertised size versus the available space reported in df when it's empty. If you're seeing a difference bigger than that it could be something else.

Last edited by suicidaleggroll; 11-11-2015 at 01:52 PM.
 
Old 11-11-2015, 05:20 PM   #3
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,978

Rep: Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624Reputation: 3624
I'd wonder why you are using ext3 too. Just curious.
 
Old 11-12-2015, 08:15 AM   #4
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
This is what df -ha displays.

Code:
  

/dev/mapper/VolGroup03-LogVol00
                      495G  198M  469G   1% /oraexp1
/dev/mapper/VolGroup04-LogVol00
                      987G  200M  936G   1% /oraexp2
/dev/mapper/VolGroup05-LogVol00
                      987G  200M  936G   1% /oraexp3
/dev/mapper/VolGroup06-LogVol00
                      987G  200M  936G   1% /oraexp4
However from vSphere, the disks shows

Code:
500 GB 
1000 GB 
1000 GB 
1000 GB
So I'm not familiar with reserve space or sectors. Also didn't know that df used GB while fdisk used GiB. I'm not entirely familiar with those so I need to read up on that.

As for why we are using ext3, we are holding onto RHEL5 till the end.
 
Old 11-12-2015, 08:40 AM   #5
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
Looks pretty standard

469 / 495 = .947, so there's your 5 reserved sectors, and you can see the mapping between GB and GiB in my post above which accounts for the drop from 532 to 495.

If you want to convert yourself, just take the advertised size from the drive manufacturer or fdisk in GB, and multiply by (1000/1024)^3 to convert to GiB (what df reports).
 
Old 11-12-2015, 11:16 AM   #6
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Well - there are other overheads.

You show that you are using LVM - which imposes some overhead for tracking volumes (not certain of the actual amount, but I think it around 1MB - depending on how large things are... it can grow as more disks are added to a logical volume).

Using Ext3/4 also has overheads - boot blocks, volume headers, backup headers, inode list (a block per inode), free block lists, block cluster management... When a file gets created some blocks (when the file is larger than a minimum) will hold metadata to identify other blocks where the data is - thus more overhead.

If this is layered on top of a raid, there is the raid configuration, error management/recovery overhead (more blocks used to identify the configuration, backups of the configuration,... When a logical volume includes multiple devices, the logical volume adds some overhead to be able to reconstruct the volume at boot time. The same applies to raid devices..

So there is overhead for using a raid
plus overhead for using logical volumes
plus overhead for the filesystem itself.

In addition, there is the easy confusion between a vendor selling a disk (anything that makes it look bigger is good - so using base 10 GB makes the disk look bigger even though everything with the disk uses base 2... hence the Gib reference for 1024 -> 1Kib. 1024 Kib -> 1Mib, and 1024 Mib -> 1 Gib all make disks look smaller).
 
Old 11-14-2015, 09:18 PM   #7
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
Unfortunately I'm not to terribly familiar with tune2fs, GB (Base 10) and GiB (Base 2), so I have more reading to do.

thanks again
 
Old 11-14-2015, 10:07 PM   #8
Emerson
LQ Sage
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~amd64
Posts: 7,661

Rep: Reputation: Disabled
The correct units are KiB (not Kib) and kB (not KB or Kb), see more here: http://physics.nist.gov/cuu/Units/binary.html
 
Old 11-15-2015, 08:11 AM   #9
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
Quote:
Originally Posted by Emerson View Post
The correct units are KiB (not Kib)
KiB = kibibyte
Kib = kibibit

Both are valid units.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] disk space used by user formatting asistant Programming 8 05-09-2012 12:40 PM
[SOLVED] Some Allocated Disk Space When Formatting A Device ethereal1m Linux - Newbie 7 09-16-2010 07:18 AM
lost my disk space after formatting the fedora installed usb pheonix7597 Linux - Newbie 4 11-17-2008 09:06 AM
hard disk partitioning/I am out of space /how to increase linux space? RMLinux Red Hat 8 09-05-2008 12:33 PM
formatting hard disk rohans Linux - Software 2 08-24-2007 07:10 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:59 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration