Varrying info about disk usage on /home

JosephS · 08-25-2007, 07:41 PM

I tried getting info about my home directory:

Code:

du -h /home/joe/

995 M

Code:

du -hc /home/joe/*

902 M

Code:

du -hc home/joe/.*

2.1G

Doesn't * mean all files and .* hidden files?

When I run:

Code:

bash-3.1$ df -h /home/joe

Quote:

Filesystem Size Used Avail Use% Mounted on
/dev/hda3 51G 1.2G 47G 3% /home

I get 1.2G used
When I use kinfo I get 3797M used on my home partition

I have /home/joe partitioned at 55 GB
Why do I have some many different outcomes?

Thanks.

jschiwal · 08-25-2007, 09:15 PM

The "*" wildcard doesn't expand to include hidden files and directories. You could use "du -hc /home/joe/{*,.*}".

There are other things that can give you different results. If a file has holes, it might have a larger value for the file size then it takes up on the file system. On example is a sparse file. Another thing, is if you have a short config file, for example. There are a lot of them in the hidden directories. They may only be 100 bytes long, but they will take up the whole block. Plus HD space for the directory entry and the inode. If you have 1024 blocksized then 924 is unused. Another thing that can take up space is the reserved area for the root user. For the ext3 filesystem you can use tune2fs to change the percentage of space reserved for the root user. By this I don't mean space for the root users /root home directory. I'm referring to an amount of disk space that only root can use. It is a parachute of sorts in case the system runs out of disk space.

choogendyk · 08-25-2007, 09:15 PM

There are a number of reasons that can happen.

Here is the section from the Sunmanagers FAQ which is probably pretty relevant here too.

ftp://ftp.cs.toronto.edu/pub/jdd/sunmanagers/faq

Quote:

5.10) Where is my disk space? The "du" and "df" commands disagree.

If a process is holding open a file, and that file is removed, the
space belonging to the file is not freed until the process either
exits or closes the file. This space is counted by "df" but not by
"du". This often happens in /var/log or /var/adm when a long-running
process (e.g. syslog) is holding open a file. In the case of syslog,
send it a HUP (e.g. kill -HUP <syslog's process ID>).

You can use LSOF
(ftp://ftp.cerias.purdue.edu/pub/tool.../sysutils/lsof) to find
which processes are holding open a particular file. Thanks to Stefan
Voss <s dot voss at terradata.de> and Michael R. Zika <zika at oconto
dot tamu dot edu>

Under Solaris 2.6 and later, files which have been unlinked can still
be accessed through the /proc interface. If a process is holding open
such a file for writing, but it's inconvenient or impractical to kill
the process or get it to close the file, you can free up the disk
space by truncating (not removing) the file from under /proc; e.g.,

# cd /proc/1234/fd
# ls -l
c--------- 1 root 24, 12 Jan 1 11:33 0
c--------- 1 root 24, 12 Jan 1 11:33 1
c--------- 1 root 24, 12 Jan 1 11:33 2
--w------- 1 root 314159265 Jan 1 11:37 3
# : > 3
# ls -l
c--------- 1 root 24, 12 Jan 1 11:33 0
c--------- 1 root 24, 12 Jan 1 11:33 1
c--------- 1 root 24, 12 Jan 1 11:33 2
--w------- 1 root 0 Jan 1 11:38 3

Thanks to Dan Astoorian <djast at cs dot toronto dot edu>

Brian Poole <raj at cerias dot purdue dot edu> writes:
Another possible cause of df & du disagreeing is if the files are
being 'hidden' under a mount. I ran into this recently where I
had a large number of files in /tmp (from adding patches in single
user mode) that were on the root partition. Thus when I was
looking for them in multiuser mode, I couldn't find them because
of the tmpfs overlay. I exported the root partition via NFS and
upon mounting it found the hidden files and deleted them.

JosephS · 08-25-2007, 10:23 PM

When using du what do the wildcards * and .*, and *.* represent?

Why is there such a big difference between what kinfo (3,797M) and du -h /home/joe (995M)

choogendyk · 08-26-2007, 07:21 PM

So, you want specific answers to your system? You could hand out your login and we could all go look. ;-)

Sorry. It just seems you've been given a fair bit of guidance, and now the specific answers depend on the particulars within your own system.

Use ls as well as du and dk to explore what's up. `ls -la`, `ls -ld *`, `ls -ld .*`, etc. will show you what is coming up. The wildcard expansion will be the same as it would be for du. The d option prevents it from expanding all the directories just to keep the output manageable.

But, remember all those other possible things mentioned in the general answers posted earlier.

On Solaris I can even create a snapshot of a filesystem that is disconnected from the directory tree. That's going to eat space without even showing up on any kind of `ls -la`. I would have to use the fssnap command to see what's up with it. I believe it won't show up on `du -k`, but it would show up on `df -k`. I don't recall seeing your distribution, so I don't know what other options you might have.

JosephS · 08-30-2007, 07:24 AM

Thanks for your help.