backup using rsync

mkrems · 04-04-2009, 10:29 AM

Hello there,

I used a rsync in Ubuntu to backup my media and data (~60 GB) hard drive to an external hard drive. I used the following options "-avizP --delete --stats". After copying one hard drive to the other, I ran it again, and this time it should have copied exactly 0 files, correct? However, it proceeded to recopy the entire hard drive.

I have used these exact same options for simpler file systems, and it works perfectly. Can anyone offer any advice? Note that the original media and data file system was constructed in Windows, although I can't imagine this would matter as I have used a simpler Windows file system and it has worked perfectly. Does rsync just have problems with huge file systems (the one I am trying to backup is over 30000 files) rendering it totally useless for a practical backup system?!?

Thanks!

amani · 04-04-2009, 11:52 AM

Maybe there are no permissions.

AlucardZero · 04-04-2009, 12:20 PM

What filesystem are the partitions? You sure it worked the first time? No, it doesn't have problems with large numbers of files.

mkrems · 04-04-2009, 01:04 PM

Quote:

Originally Posted by AlucardZero

What filesystem are the partitions? You sure it worked the first time? No, it doesn't have problems with large numbers of files.

Both are FAT file systems. It seems to have problems with large nubmers of files because I can do the exact same thing with around 1000 files and it works fine. And yes, it did work the first time. It does succesfully copy the drive...it just has not idea how to update it, only how to start over again.

AlucardZero · 04-04-2009, 01:15 PM

Quote:

Both are FAT

Found the problem. FAT32 is such an old and featureless filesystem, it doesn't play well with Unix files (it has no concept of permissions) or rsync. Try the following, from the man page:

Code:

       BUGS

       When transferring to FAT filesystems rsync may re-sync unmodified files.  See the comments on the --modify-window option.

Code:

       --modify-window
              When comparing two timestamps, rsync treats the timestamps as being equal  if  they differ  by  no more than the modify-window value.  This is normally 0 (for an exact match), but you may find it useful to set this to a larger  value  in  some  situations.   In  particular,  when transferring to or from an MS Windows FAT filesystem (which represents times with a 2-second resolution),  --modify-window=1  is  useful (allowing times to differ by up to 1 second).

mkrems · 04-04-2009, 01:30 PM

Quote:

Originally Posted by AlucardZero

Found the problem. FAT32 is such an old and featureless filesystem, it doesn't play well with Unix files (it has no concept of permissions) or rsync. Try the following, from the man page:

Code:

       BUGS

       When transferring to FAT filesystems rsync may re-sync unmodified files.  See the comments on the --modify-window option.

Code:

       --modify-window
              When comparing two timestamps, rsync treats the timestamps as being equal  if  they differ  by  no more than the modify-window value.  This is normally 0 (for an exact match), but you may find it useful to set this to a larger  value  in  some  situations.   In  particular,  when transferring to or from an MS Windows FAT filesystem (which represents times with a 2-second resolution),  --modify-window=1  is  useful (allowing times to differ by up to 1 second).

Ok, actually I was copying from an NTFS system to a FAT32 but I guessing your analysis still applies. I missed that info in the man page. Thanks for your help!

mkrems · 04-04-2009, 02:07 PM

Quote:

Originally Posted by mkrems

Ok, actually I was copying from an NTFS system to a FAT32 but I guessing your analysis still applies. I missed that info in the man page. Thanks for your help!

this worked like a charm. thanks again!