LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (https://www.linuxquestions.org/questions/slackware-14/)
-   -   huge difference between du and df (https://www.linuxquestions.org/questions/slackware-14/huge-difference-between-du-and-df-4175589814/)

BCarey 09-20-2016 12:55 PM
huge difference between du and df
 
Hi all,

I had a system (slack64 14.1) crash this morning and I couldn't start X on reboot. Turns out my / filesystem was full, as reported by df. So, trying to clear things up, I used du to find out where the bloat was. Unfortunately, du seemed to find a lot less used space than du reported.

To be more specific, "du -xsh " showed 22G used while "df -h" showed 28G used. Since the filesystem was on lvm I was able to increase the volume size to solve my immediate problem, but the discrepancy between du and df still exists. I have run fsck on the root filesystem, which reports all okay. I have rebooted to ensure that no running processes could be still claiming deleted files. Never-the-less I still have this non-trivial discrepancy. Any ideas about how to research further?

Thanks,
Brian

Richard Cranium 09-20-2016 01:05 PM
I assume that du does not account for the metadata associated with a file whereas df does.

BCarey 09-20-2016 01:09 PM
Quote:
Originally Posted by Richard Cranium (Post 5607861)
I assume that du does not account for the metadata associated with a file whereas df does.
Could that really account for a 6G (21-27%) difference?

smallpond 09-20-2016 01:22 PM
du reports allocated disk usage. df reports filesystem space available to a non-root user. df does not include reserved space as available.

phenixia2003 09-20-2016 01:31 PM
Hello,

By default, 5% of disk space is reserved. 28-22=6 => 5%(X)=6G => X=120G



--
SeB

BCarey 09-20-2016 01:33 PM
Quote:
Originally Posted by phenixia2003 (Post 5607868)
Hello,

By default, 5% of disk space is reserved. 28-22=6 => 5%(X)=6G => X=120G



--
SeB
But this partition was only 30G, not 120G, so 5% would only be 1.5G. Since I expanded the file system to 50G, I'm getting identical results from df and du. Reserved space would still only account for 2.5G, which still leaves a huge amount "unaccounted" for.

rknichols 09-20-2016 01:49 PM
That difference is almost always due to a huge deleted file still held open by some process, often a big log file. You can run (as root)
Code:
lsof | grep '(deleted)' | less
and scroll down looking for something huge.

If you have rebooted and still see the problem, then "fsck -f" on the filesystem is warranted.

And, now that I see you've already done that, about the only remaining possibility is files hidden under an active mount point. Try this
Code:
mkdir /tmp/tmpmnt
mount --bind / /tmp/tmpmnt
du /tmp/tmpmnt

phenixia2003 09-20-2016 02:03 PM
Quote:
Originally Posted by BCarey (Post 5607870)
But this partition was only 30G, not 120G, so 5% would only be 1.5G. Since I expanded the file system to 50G, I'm getting identical results from df and du. Reserved space would still only account for 2.5G, which still leaves a huge amount "unaccounted" for.
I've had an issue like that, but without crash, and without lvm. I've found there was huge files in hidden folders and large files (~20M) in folder like /var/tmp/kdecache-seb with commands below :

Code:
$ du -h ~ | grep -E "^[[:digit:]]+[G]"
$ du -h /var/tmp | grep -E "^[[:digit:]]+[G]"
$ du -h /tmp | grep -E "^[[:digit:]]+[G]"

$ du -h ~ | grep -E "^[[:digit:]]{2}[M]"
$ du -h /var/tmp | grep -E "^[[:digit:]]{2}[M]"
$ du -h /tmp | grep -E "^[[:digit:]]{2}[M]"
--
SeB

BCarey 09-20-2016 02:05 PM
Quote:
Originally Posted by rknichols (Post 5607877)
And, now that I see you've already done that, about the only remaining possibility is files hidden under an active mount point. Try this
Code:
mkdir /tmp/tmpmnt
mount --bind / /tmp/tmpmnt
du /tmp/tmpmnt
This looks promising. When doing this du reported the 28G. So I will investigate that. Thank you very much.

Brian

BCarey 09-21-2016 10:37 AM
So indeed there were 6 Gigs of files hiding under one of the mounts. Thank you rknichols for pointing to the solution. Thank you also phenixia for your efforts on my behalf. Now both df and du report 22G used, not even a discrepancy d/t reserved space.

Brian

saurabhve 08-23-2019 12:10 AM
Quote:
Originally Posted by rknichols (Post 5607877)
That difference is almost always due to a huge deleted file still held open by some process, often a big log file. You can run (as root)
Code:
lsof | grep '(deleted)' | less
and scroll down looking for something huge.

If you have rebooted and still see the problem, then "fsck -f" on the filesystem is warranted.

And, now that I see you've already done that, about the only remaining possibility is files hidden under an active mount point. Try this
Code:
mkdir /tmp/tmpmnt
mount --bind / /tmp/tmpmnt
du /tmp/tmpmnt


binding the mount showed up that there was a local directory which had some used space and at some point of time this directory was mounted to some different disk partition, thus leaving behind the old data in the previous partition.

Richard Cranium 08-23-2019 01:41 AM
"Better late than never" isn't always true.


All times are GMT -5. The time now is 02:26 PM.