rsync size discrepency

kav · 11-13-2006, 12:09 AM

I ran this command on a 110G directory to two local hard drives

rsync -vrlHptog --progress --delete /mnt/myhome/user /mnt/backup

"df -h" says both partitions now have 110G used, but "du -s" reports 114808759 in the source directory and 115015088 in the destination directory. Something else I noticed was that despite the fact that they both have 110G used, the source directory's size is 114G with 3.9G free, which makes sense, but the destination directory has a size of 119G with only 2.4G available. Why is that?

Would using different filesystems cause these discrepencies? The source directory filesystem is reiser3 and the destination filesystem is ext3. Is the reason as simple as that, or are the two rsync'd directories not identical?

baikonur · 11-13-2006, 02:15 AM

The filesystem does have an effect on the size of files on disk.
I believe the difference between the free space and (total - used space) is normal.
I've been wondering off and on for months, since I saw it on all 3 linux servers
we are running here, too.
I don't know for sure but I imagine it has to do with the drive's fragmentation.

Quigi · 11-13-2006, 09:52 AM

I think baikonur answered kav's question (size differences between reiser4 and ext3) but brought up a new one:

Quote:

Originally Posted by baikonur

I believe the difference between the free space and (total - used space) is normal.
I've been wondering off and on for months, since I saw it on all 3 linux servers
we are running here, too.
I don't know for sure but I imagine it has to do with the drive's fragmentation.

Yes, this difference is normal because some blocks (e.g., 5%) are reserved for root.

Quote:

Originally Posted by man mkfs.ext3

-m reserved-blocks-percentage
Specify the percentage of the filesystem blocks reserved for the super-user. This
avoids fragmentation, and allows root-owned daemons, such as syslogd(8), to con‐
tinue to function correctly after non-privileged processes are prevented from
writing to the filesystem. The default percentage is 5%.

E.g., on one of my disks,

Code:

Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/sda6                20158      5187     13948  28% /

Here 95% of 20158M = 19150M (available for normal users), and 5187M + 13948M = 19135M. Not exactly the same, but close -- 19G out of 20G are available for normal users, 1G is set aside for root.

kav · 11-13-2006, 10:33 AM

Thanks, that answers one question. But does it mean that the exact same data takes up a different amount of space on 2 different file systems?

I think this might in fact be the case. $ du -s * reveals several empty directories that take up 0 on the ext3 partition. Those same directories take up 4 on the reiser partition. But would that really add up to the size difference that I have between the two of 114808759 and 115015088?

baikonur · 11-13-2006, 12:14 PM

Quote:

I think this might in fact be the case. $ du -s * reveals several empty directories that take up 0 on the ext3 partition. Those same directories take up 4 on the reiser partition

Interesting... now, to see if that has anything to do with rsync, you could create an empty directory on the ext3 partition. I think you'd get something with a size bigger than 0.
On my laptop, I get

Code:

mpf@cvlaptop:~$ mkdir empty
mpf@cvlaptop:~$ ls -al empty
insgesamt 12
drwxr-xr-x   2 mpf mpf    6 2006-11-13 19:07 .
drwxr-xr-x  60 mpf mpf 8192 2006-11-13 19:07 ..
mpf@cvlaptop:~$

('insgesamt' being german for 'total')
So that's two links on something which is 6 bytes big, right? That something being completely empty. Maybe you could post the ls -al results for both source and destination empty directory.

curious...

Quigi · 11-13-2006, 12:48 PM

Hi Baikonur, I'm afraid we just added another "It depends". It looks like you benefit from some small file optimization, because it costs me not 6 bytes but 4K:

Code:

$ df .
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda7             19346964   6919992  11444200  38% /home
$ mkdir empty
$ ls -l empty
total 0
$ ls -la empty
total 8
drwxr-xr-x 2 brech dba   4096 Nov 13 13:39 ./
drwxr-xr-x 6 brech brech 4096 Nov 13 13:39 ../
$ df .      
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda7             19346964   6919996  11444196  38% /home
$ rmdir empty
$ df .
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda7             19346964   6919992  11444200  38% /home
$ mount | grep /home
/dev/sda7 on /home type ext3 (rw)

This is on ubunutu. Grepping mount confirms it's ext3. BTW, 6 looks suspiciously like the string length of "empty\0". Do you care to mkdir some-much-longer-directory-name?

baikonur · 11-14-2006, 02:43 AM

Hello Quigi,

here's mine:

Code:

cvlaptop:/home/mpf# mount | grep home
/dev/hda5 on /home type xfs (rw)
cvlaptop:/home/mpf# df .
Dateisystem          1K-Blöcke   Benutzt Verfügbar Ben% Eingehängt auf
/dev/hda5             35149040   4101740  31047300  12% /home
cvlaptop:/home/mpf# mkdir ridiculous-long-directory-name-to-see-if-it-has-anything-to-do-with-the-length
cvlaptop:/home/mpf# ls -al ridiculous-long-directory-name-to-see-if-it-has-anything-to-do-with-the-length/
insgesamt 12
drwxr-xr-x   2 root root    6 2006-11-14 09:18 .
drwxr-xr-x  61 mpf  mpf  8192 2006-11-14 09:18 ..
cvlaptop:/home/mpf# df .
Dateisystem          1K-Blöcke   Benutzt Verfügbar Ben% Eingehängt auf
/dev/hda5             35149040   4101740  31047300  12% /home

I notice that on my pc the used blocks are not reduced by craeting an empty directory...
and now, to maximize confusion:

Code:

cvlaptop:/home/mpf# touch ridiculous-long-directory-name-to-see-if-it-has-anything-to-do-with-the-length/empty_file
cvlaptop:/home/mpf# ls -al ridiculous-long-directory-name-to-see-if-it-has-anything-to-do-with-the-length/
insgesamt 12
drwxr-xr-x   2 root root   23 2006-11-14 09:39 .
drwxr-xr-x  61 mpf  mpf  8192 2006-11-14 09:18 ..
-rw-r--r--   1 root root    0 2006-11-14 09:39 empty_file
cvlaptop:/home/mpf# df .
Dateisystem          1K-Blöcke   Benutzt Verfügbar Ben% Eingehängt auf
/dev/hda5             35149040   4101924  31047116  12% /home

so that empty file is 0 bytes big... but creating it takes away 184 k-blocks, but creating a directory doesn't use up any blocks, even though it is 6 bytes.

Quigi · 11-14-2006, 08:52 AM

Höchst merkwürdig!

Still six bytes (and no impact on df). I hope it frees those 184k again when you delete the empty file! (Or was there some other process creating another file). I think xfs is weirder than ext3. Then again, I'm used to the latter.

JZL240I-U · 11-15-2006, 03:06 AM

Umm, just my

1. The size of the Blocks should matter, since they (can) vary from file system to file system and from installation to installation.

2. The systems you compare are journalling systems which do their journalling differently (e.g. ext3 is using lists (of inodes) and XFS as well as Reiser B+trees (plus --partly-- lists). I certainly can't say what that does to the size...

3. Extra cent at no extra cost

: Neither of you did sync their disk(s). That might matter as well...

kav · 11-15-2006, 12:47 PM

Very informative. I'll stop worrying about rsync not actually copying all my files.

Thanks for all your responses

baikonur · 11-15-2006, 01:02 PM

just for the record:
the 184 k must indeed have been due to something else:

Code:

cvlaptop:/home/mpf# df .
Dateisystem          1K-Blöcke   Benutzt Verfügbar Ben% Eingehängt auf
/dev/hda5             35149040   4124492  31024548  12% /home
cvlaptop:/home/mpf# rm ridiculous-long-directory-name-to-see-if-it-has-anything-to-do-with-the-length/empty_file
cvlaptop:/home/mpf# df . Dateisystem          1K-Blöcke   Benutzt Verfügbar Ben% Eingehängt auf
/dev/hda5             35149040   4124492  31024548  12% /home
cvlaptop:/home/mpf# touch ridiculous-long-directory-name-to-see-if-it-has-anything-to-do-with-the-length/empty_file
cvlaptop:/home/mpf# df . Dateisystem          1K-Blöcke   Benutzt Verfügbar Ben% Eingehängt auf
/dev/hda5             35149040   4124492  31024548  12% /home

sorry... I have to work on this rig sometimes between my posts on LQ.

so: neither an empty file nor an empty dir uses any blocks. The directory just seems to consume two bytes in the journal.
aha.

Quigi · 11-15-2006, 03:27 PM

Quote:

Originally Posted by kav

I'll stop worrying about rsync not actually copying all my files.

To see if "all" files are there (same names), you could run ls -R in both /mnt/myhome/user and /mnt/backup, redirecting the output to temporary files, and diff the files. Use ls -lR to also compare mode, ownership, size and mtime.
If you want to check if the contents are exactly the same, [b]find . -type f | xargs md5sum > /tmp/f0[b] (and f1 from the other dir) and then diff, but that will take a LONG time because md5sum will have to read through the whole 110G. Maybe you have to give xargs one of -s, -n, or -L to avoid too long argument lists. On some distros it does a reasonable thing by itself.