Missing 200megs of space in /var

epoh · 03-29-2008, 10:58 AM

Okay, I am loosing my mind.

This is the result of a df -k:

/var # df -k
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/cciss/c0d0p7 379303 336749 22971 94% /
/dev/cciss/c0d0p1 51358 7703 41003 16% /boot
/dev/cciss/c0d0p6 16610024 12481180 3285104 80% /home
none 1158676 0 1158676 0% /dev/shm
/dev/cciss/c0d0p2 33135496 29040252 2412040 93% /usr
/dev/cciss/c0d0p8 252863 234618 5190 98% /var

Then, if I cd to /var and do a du -sk:

/var # du -sk
41773 .

How is that possible? The system says I've got almost 250M allocated to /var, but if you actually check disk usage of var it says I've got 40megs. Where is the 200megs? What am I missing here? This server is gonna die in a fiery crash pretty soon if I can't figure this out. This is, obviously, an old system, with no other storage available to it. Even if there was, I cannot expand the filesystem while it's online, since it's running Redhat 7.2.

Any help would be greatly appreciated.

colucix · 03-29-2008, 11:22 AM

Maybe you have sparse files. A common one is /var/log/lastlog which can be very large sometimes: I have seen a lastlog of about 1 Gb in the past. Try

Code:

du -bs /var/log/lastlog
du -bs --apparent-size /var/log/lastlog

and see if the difference is your missing 200 Mb.

epoh · 03-29-2008, 11:29 AM

Sadly that's not it. lastlog is only 32k. We cleaned that up yesterday.

Well, maybe I spoke too soon. ls -hal shows it's 18megs. That's not the 200 I'm looking for, but it's something, lol.

epoh · 03-29-2008, 11:40 AM

Quote:

Originally Posted by colucix

Maybe you have sparse files. A common one is /var/log/lastlog which can be very large sometimes: I have seen a lastlog of about 1 Gb in the past. Try

Code:

du -bs /var/log/lastlog
du -bs --apparent-size /var/log/lastlog

and see if the difference is your missing 200 Mb.

Is there a way to search the filesystem for sparse files?

colucix · 03-29-2008, 02:59 PM

Quote:

Originally Posted by epoh

Is there a way to search the filesystem for sparse files?

You can do that by scripting. Using the stat command you can retrieve the total size of files in bytes and the actually allocated blocks. Comparing these two values (the latter multiplied by the block size) when the size of allocated blocks is smaller than the total size of the file, you can guess it is a sparse file. I've written a little script in bash to do this:

Code:

#!/bin/bash
gap=0
block_size=$(stat -c %B $0)
while read line
do
   size_in_bytes=$(echo $line | cut -f 1 -d \ )
   block_allocated=$(echo $line | cut -f 2 -d \ )
   file=$(echo $line | cut -f 3- -d \ )
   if [ $(expr $block_allocated \* $block_size) -lt $size_in_bytes ]
   then
      echo $file
      gap=$(( gap + ( $size_in_bytes - $block_allocated * $block_size ) / 1000 ))
   fi
done < <( find /var -type f -exec stat -c "%s %b %n" '{}' \; )
echo Rough estimate of total unallocated space in sparse files is: $gap k

It also gives you the total amount (in kbytes) of unallocated block in sparse files, that is a rough estimate of the gap between the output of df and du. This should bring to the missing 200 Mb. Maybe...

epoh · 03-29-2008, 03:44 PM

Tried the script but i'm getting errors that -c isn't a valid flag. i'll have to look into it some more.

Quote:

Originally Posted by colucix

You can do that by scripting. Using the stat command you can retrieve the total size of files in bytes and the actually allocated blocks. Comparing these two values (the latter multiplied by the block size) when the size of allocated blocks is smaller than the total size of the file, you can guess it is a sparse file. I've written a little script in bash to do this:

Code:

#!/bin/bash
gap=0
block_size=$(stat -c %B $0)
while read line
do
   size_in_bytes=$(echo $line | cut -f 1 -d \ )
   block_allocated=$(echo $line | cut -f 2 -d \ )
   file=$(echo $line | cut -f 3- -d \ )
   if [ $(expr $block_allocated \* $block_size) -lt $size_in_bytes ]
   then
      echo $file
      gap=$(( gap + ( $size_in_bytes - $block_allocated * $block_size ) / 1000 ))
   fi
done < <( find /var -type f -exec stat -c "%s %b %n" '{}' \; )
echo Rough estimate of total unallocated space in sparse files is: $gap k

It also gives you the total amount (in kbytes) of unallocated block in sparse files, that is a rough estimate of the gap between the output of df and du. This should bring to the missing 200 Mb. Maybe...