LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices

Reply
 
LinkBack Search this Thread
Old 11-05-2012, 12:44 PM   #1
ptricky
LQ Newbie
 
Registered: Nov 2012
Distribution: centos
Posts: 4

Rep: Reputation: Disabled
df showing different values after performing a rsync


So I have a NAS with two different file systems, the primary filesystem has 33 TB of data. I rsync'd the data over to the second file system. df shows about 299 GB less data on the backup filesystem then the primary filesystem. I wrote a script to go through the file system and compare sizes via ls, and they match but df shows differnet size, why?

The backup filesystem is a cluster of 4.5 TB disk's, DEFAULT_BLOCKSIZE is 4096 and Readahead 512KB.

The primary file system is a cluster of 2.1 and 2.7 TB disks, DEFAULT_BLOCKSIZE is 4096 and
Readahead 2048KB.

Thanks.
 
Old 11-06-2012, 06:02 AM   #2
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 3,999

Rep: Reputation: 1003Reputation: 1003Reputation: 1003Reputation: 1003Reputation: 1003Reputation: 1003Reputation: 1003Reputation: 1003
you can use du to check the occupied disk space. Would be nice to see how rsync was configured/invoked. Do you have sparse files? Rsync is able to create hard links in some cases also.
 
Old 11-06-2012, 08:55 AM   #3
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 1,158

Rep: Reputation: 493Reputation: 493Reputation: 493Reputation: 493Reputation: 493
Best guess would be sparse files, but only if you used rsync's "-S" (--sparse) option. If you did use that option, any 4096-byte blocks that were all zeros will not use space on the destination regardless of whether they used space on the source. (Without that option, rsync will never make a sparse file, and the destination could use significantly more space than the source.)

Some other cases where less space is used on the destination:
  • Files that are unlinked from the directory tree on the source but are still held open by some process,
  • Files that are in the process of being written on the source and thus have some extra space pre-allocated,
  • Directories that once contained a large number of entries but now contain only a few. Space allocated to a directory file is never reclaimed until the directory is removed entirely. That unneeded space will not be allocated on the destination.
Of those, only the first is likely to involve any significant amount of space.

Afterthought: By any chance was the source a filesystem that supported transparent compression?

Last edited by rknichols; 11-06-2012 at 08:58 AM. Reason: Add afterthought.
 
1 members found this post helpful.
Old 11-07-2012, 04:14 PM   #4
ptricky
LQ Newbie
 
Registered: Nov 2012
Distribution: centos
Posts: 4

Original Poster
Rep: Reputation: Disabled
hmmm... on the source filesystem there were a couple of directories that could not be deleted due to dangling inodes and it requires a fsck to fix it. So I think " Space allocated to a directory file is never reclaimed until the directory is removed entirely" is probably the likely cause.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync solaris + ld.so.1: rsync: fatal: libiconv.so.2: open failed: xxx_anuj_xxx Solaris / OpenSolaris 25 02-23-2012 03:23 AM
[SOLVED] Screen not showing on laptop but showing on projector when connected someshpr Linux - General 3 04-18-2011 04:58 PM
rsync showing wrong output system11 Linux - Server 3 02-17-2011 06:38 AM
[SOLVED] rsync fails in cron - ssh key prob for rsync? jonathansfl Linux - Server 6 12-09-2010 09:48 AM
rsync showing unocode errors mirror51 Linux - Newbie 3 04-30-2010 11:43 AM


All times are GMT -5. The time now is 06:29 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration