How come file size differs between ls and df?

felixrabe · 01-06-2008, 01:13 PM

Hi there,

I have a large file (a VirtualBox hard disk) which shows a size of about 8 GB in ls and about 4 GB of total disk usage in df. How come?

Code:

$ ls -l disk.vdi 
-rw------- 1 fr fr 8589967872 2007-08-29 15:53 disk.vdi
$ ls -lh disk.vdi 
-rw------- 1 fr fr 8.1G 2007-08-29 15:53 disk.vdi
$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda7              19G  4.8G   13G  28% /media/bak7-fr-2
$ mount | grep sda7
/dev/sda7 on /media/bak7-fr-2 type ext3 (rw,nosuid,nodev,uhelper=hal)

pixellany · 01-06-2008, 01:28 PM

Do you know that disk.vdi is on /dev/sda7?

What does "du disk.vdi" tell you?

since the file is for a VM, is it possible that the file size gets reported differently?

colucix · 01-06-2008, 01:47 PM

Maybe it is a "sparse file". You can find a definition on the Wikipedia. A common example on many systems is the file /var/log/lastlog

Code:

$ ls -l /var/log/lastlog
-rw-r--r-- 1 root tty 292584 2008-01-06 12:11 /var/log/lastlog

$ du /var/log/lastlog
28      /var/log/lastlog

You can also try the following test using the command find: cd to the directory where the file is stored, then do

Code:

# the following will print the allocated (total) space
find . -name disk.vdi -printf "%s\n"

# the following will print the actual size if sparse
find . -name disk.vdi -printf "%k\n"

felixrabe · 01-06-2008, 02:53 PM

Quote:

Originally Posted by colucix

Maybe it is a "sparse file". You can find a definition on the Wikipedia. ...

That's totally it, thanks. I'm currently copying the file (over USB 1) around with 'cp -ax' and I'm glad that cp supports sparse files (saved quite some time

).

Code:

$ ls -lh disk.vdi 
-rw------- 1 fr fr 8.1G 2007-08-29 15:53 disk.vdi
$ du -h disk.vdi 
4.2G	disk.vdi

pixellany · 01-06-2008, 05:20 PM

How would you create a "sparse file"? (Other than the dd method in the Wikipedia article)

colucix · 01-06-2008, 05:57 PM

I don't know any other *NIX utility to do that. On the other hand you can always use a programming language like C or Python. Here is a simple example in Python

Code:

#!/usr/bin/python
file = open("/tmp/sparsefile", 'w')
file.truncate(1024 * 1048000L)
file.close()

this will create a sparse file of 1G, whose actual size is 0 bytes (since nothing has been written into).

xptools · 01-06-2008, 06:26 PM

FYI, there can be differences between df and du -s as well.