Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
|
02-24-2015, 05:53 PM
|
#1
|
Senior Member
Registered: Nov 2008
Posts: 1,052
Rep: 
|
Partition full (100%) even if sum of all files is less than 50%
Hi,
So I have an encrypted /home partition (created with LVM and encryption during Centos 6.5 install) that ran out of space all of a sudden (or it was close to 100% for weeks and I never noticed...) but the problem is that the sum of all files and folders is less than 50% of the partition's capacity.
df -h
Code:
[root@workstation workstation-user]# df -H
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_workstation-lv_root
53G 9.8G 41G 20% /
tmpfs 4.2G 2.6M 4.2G 1% /dev/shm
/dev/sda1 500M 62M 412M 14% /boot
/dev/mapper/vg_workstation-lv_home
923G 876G 98M 100% /home
If I do a du -h --max-depth=1 on /home I get:
Code:
[root@workstation home]# du -h --max-depth=1 .
16K ./lost+found
816G ./workstation-user
816G .
Then finally if I go in "workstation-user" and issue the same command:
Code:
[root@workstation workstation-user]# du -h --max-depth=1 .
4.0K ./.vnc
96K ./.gnome2
4.0K ./.themes
12K ./.dvdcss
168K ./.pulse
4.0K ./.gvfs
18M ./.eclipse
371G ./file-storage
16K ./.gnupg
36K ./.macromedia
8.0K ./.redhat
668K ./.gstreamer-0.10
68K ./.java
23M ./.mozilla
6.0G ./.local
980K ./.kde
16K ./.mplayer
16K ./.gnote
8.0K ./.cups
20K ./Desktop
4.0K ./.qt
12K ./.dbus
36K ./.sane
36K ./.pki
580K ./.adobe
70M ./.cache
68K ./.fontconfig
28K ./.designer
4.0K ./.nautilus
12K ./.abrt
5.8M ./.config
4.0K ./.gnome2_private
136K ./.nv
4.0K ./.icons
20G ./to-sort
8.0K ./.ssh
8.0K ./.emacs.d
852K ./.gconf
80K ./.gconfd
524K ./.gimp-2.6
152K ./.blender
816G .
As you can see, under /home the "biggest" folder is "file-storage" and it is 371GB. There's still 552GB to go until the capacity of 923GB is reached... All other folders and files do not account to the missing 552GB (not even close), so whats going on?
The only thing I can think of, is the encryption... Would it be possible that somehow every encrypted GB takes another GB for the encryption? Even that, roughly adding everything "du" lists and doing X2 will yield to 796GB which is still far from 923GB..
Any insight appreciated!..
Thanks
Last edited by lpallard; 02-24-2015 at 05:54 PM.
|
|
|
02-24-2015, 07:25 PM
|
#2
|
LQ Guru
Registered: Feb 2003
Location: Virginia, USA
Distribution: Debian 12
Posts: 8,385
|
I think that whatever partition contains /home contains a lot more than just /home.
--------------------
Steve Stites
|
|
|
02-24-2015, 07:48 PM
|
#3
|
Member
Registered: May 2006
Location: UK
Distribution: Debian
Posts: 447
Rep:
|
It could be open unlinked files. You can find them with
Code:
lsof | grep deleted
They normally only appear in tmp directories though.
|
|
|
02-24-2015, 11:41 PM
|
#4
|
Senior Member
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,206
|
Quote:
Originally Posted by lpallard
Hi,
So I have an encrypted /home partition (created with LVM and encryption during Centos 6.5 install) that ran out of space all of a sudden (or it was close to 100% for weeks and I never noticed...) but the problem is that the sum of all files and folders is less than 50% of the partition's capacity.
df -h
Code:
[root@workstation workstation-user]# df -H
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_workstation-lv_root
53G 9.8G 41G 20% /
tmpfs 4.2G 2.6M 4.2G 1% /dev/shm
/dev/sda1 500M 62M 412M 14% /boot
/dev/mapper/vg_workstation-lv_home
923G 876G 98M 100% /home
If I do a du -h --max-depth=1 on /home I get:
Code:
[root@workstation home]# du -h --max-depth=1 .
16K ./lost+found
816G ./workstation-user
816G .
Then finally if I go in "workstation-user" and issue the same command:
Code:
[root@workstation workstation-user]# du -h --max-depth=1 .
4.0K ./.vnc
96K ./.gnome2
4.0K ./.themes
12K ./.dvdcss
168K ./.pulse
4.0K ./.gvfs
18M ./.eclipse
371G ./file-storage
16K ./.gnupg
36K ./.macromedia
8.0K ./.redhat
668K ./.gstreamer-0.10
68K ./.java
23M ./.mozilla
6.0G ./.local
980K ./.kde
16K ./.mplayer
16K ./.gnote
8.0K ./.cups
20K ./Desktop
4.0K ./.qt
12K ./.dbus
36K ./.sane
36K ./.pki
580K ./.adobe
70M ./.cache
68K ./.fontconfig
28K ./.designer
4.0K ./.nautilus
12K ./.abrt
5.8M ./.config
4.0K ./.gnome2_private
136K ./.nv
4.0K ./.icons
20G ./to-sort
8.0K ./.ssh
8.0K ./.emacs.d
852K ./.gconf
80K ./.gconfd
524K ./.gimp-2.6
152K ./.blender
816G .
As you can see, under /home the "biggest" folder is "file-storage" and it is 371GB. There's still 552GB to go until the capacity of 923GB is reached... All other folders and files do not account to the missing 552GB (not even close), so whats going on?
The only thing I can think of, is the encryption... Would it be possible that somehow every encrypted GB takes another GB for the encryption? Even that, roughly adding everything "du" lists and doing X2 will yield to 796GB which is still far from 923GB..
Any insight appreciated!..
Thanks
|
I think files only grow about 8k-12k when encrypted.
|
|
|
02-25-2015, 06:12 AM
|
#5
|
Senior Member
Registered: Nov 2008
Posts: 1,052
Original Poster
Rep: 
|
Quote:
Originally Posted by jailbait
I think that whatever partition contains /home contains a lot more than just /home.
--------------------
Steve Stites
|
What do you mean by "a lot more than just /home"? Symlinks to other partitions? /home is being used as /home (that is, personal files no system files).. A few weeks ago I was not using encryption and I had less than 400GB of files... Impossible that it grew to almost a terabyte in less than 3 weeks..
lsof | grep deleted yielded 3 entries under /tmp... Nothing under /home
As for the encrypted space.... Thats what I expected.. a marginal increase in disk usage but not 200%+..
Im baffled...
|
|
|
02-25-2015, 06:26 AM
|
#6
|
Senior Member
Registered: Nov 2008
Posts: 1,052
Original Poster
Rep: 
|
Got it...
I ran a fsck -fv on the faulty filesystem, it didnt report any errors (not that I could see) but nevertheless upon rebooting now I get
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_workstation-lv_root
53G 9.7G 41G 20% /
tmpfs 4.2G 148k 4.2G 1% /dev/shm
/dev/sda1 500M 62M 412M 14% /boot
/dev/mapper/vg_workstation-lv_home
923G 427G 450G 49% /home
Which is more normal to me....
I am using ext4 with Centos 6.6... First time in a long time using linux that I see this kind of glitch...
|
|
|
02-25-2015, 08:41 AM
|
#7
|
LQ Guru
Registered: Sep 2011
Location: Upper Hale, Surrey/Hants Border, UK
Distribution: One main distro, & some smaller ones casually.
Posts: 5,872
Rep: 
|
I neither use LVM nor encription, but usually when you see 100% usage, (allowing for root's reserved space), & know there should be more space, it is a lack of inodes.
You probably have a lot of small files on your disk using up all the inodes, only way to overcome this situation is to have more inodes for this systems usage.
|
|
|
02-25-2015, 04:37 PM
|
#8
|
Senior Member
Registered: Nov 2008
Posts: 1,052
Original Poster
Rep: 
|
Quote:
...& know there should be more space, it is a lack of inodes
|
Would fsck fix this?
Quote:
You probably have a lot of small files on your disk using up all the inodes, only way to overcome this situation is to have more inodes for this systems usage.
|
Exactly 588,534 files on that FS... Not much if you ask me..
Thats really strange
|
|
|
02-25-2015, 06:05 PM
|
#9
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,815
|
Quote:
Originally Posted by lpallard
Got it...
I ran a fsck -fv on the faulty filesystem, it didnt report any errors (not that I could see) but nevertheless upon rebooting now I get
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_workstation-lv_root
53G 9.7G 41G 20% /
tmpfs 4.2G 148k 4.2G 1% /dev/shm
/dev/sda1 500M 62M 412M 14% /boot
/dev/mapper/vg_workstation-lv_home
923G 427G 450G 49% /home
Which is more normal to me....
I am using ext4 with Centos 6.6... First time in a long time using linux that I see this kind of glitch...
|
That strongly suggests that there were indeed open, deleted files, and the the fsck was superfluous. "lsof | grep deleted" won't find all of them. "lsof | grep -i del" will find a lot more that show "DEL" in the "FD" column and do not show "(deleted)" after the name. "DEL" indicates deleted files that were mmap()-ed and the file descriptor closed, but are still "in use" because the mmap() is still active.
|
|
|
02-25-2015, 06:26 PM
|
#10
|
Senior Member
Registered: Nov 2008
Posts: 1,052
Original Poster
Rep: 
|
Generally speaking, is it something to worry about? Why would this happen?
|
|
|
02-25-2015, 06:42 PM
|
#11
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,815
|
Quote:
Originally Posted by lpallard
Generally speaking, is it something to worry about? Why would this happen?
|
It's only something to worry about if you run out of space. It happens because a program intentionally unlinked some mmap()-ed file, perhaps a temporary database, to ensure that it would be deleted automatically when the program terminated. Another place it happens a lot, though not generally under /home, is with libraries that get updated while some program is still using the old version. The old library has to stay around until it is no longer in use, and those libraries are almost always mmap()-ed and not held on an open file descriptor.
You would have to run that "lsof | grep -i del" while the problem was present to see exactly what was causing it.
|
|
|
02-25-2015, 07:06 PM
|
#12
|
Senior Member
Registered: Nov 2008
Posts: 1,052
Original Poster
Rep: 
|
Quote:
You would have to run that "lsof | grep -i del" while the problem was present to see exactly what was causing it.
|
Allright! I guess Ill keep an eye on that and next time it happens, Ill run the command you suggested!
Thanks!!!
|
|
|
02-25-2015, 07:28 PM
|
#13
|
LQ Veteran
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,385
|
Hmmm - how often do you reboot ?. A reboot should both run fsck if needed, and get rid of unlinked filed descriptors.
If you reboot regularly (FSVO "regularly") and the problem persisted, but went away after a fsck that reported no errors (and reboot) , that is odd.
|
|
|
02-25-2015, 10:06 PM
|
#14
|
Senior Member
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora
Posts: 3,935
Rep: 
|
What fatmac said. Check:
And, no, a fsck(8) does not fix that. You can either grow the filesystem, or you can dump(8), recreate the filesystem with a more suitable bytes-to-inode ratio, and restore(8).
|
|
|
02-26-2015, 05:46 AM
|
#15
|
Senior Member
Registered: Nov 2008
Posts: 1,052
Original Poster
Rep: 
|
Quote:
Hmmm - how often do you reboot ?
|
About every week... I usually put the machine in suspend to RAM instead of shutting it down. Been doing this for years with the same hardware and between real reboots, I cannot pretend that the FS changed so much (added or removed thousands of files, etc).. Actually, a few weeks ago I reinstalled centos and restored everything from a backup.
df -i currently reports:
[workstation-user@workstation ~]$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/vg_workstation-lv_root
3276800 297867 2978933 10% /
tmpfs 1002657 11 1002646 1% /dev/shm
/dev/sda1 128016 87 127929 1% /boot
/dev/mapper/vg_workstation-lv_home
57229312 592891 56636421 2% /home
|
|
|
All times are GMT -5. The time now is 01:34 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|