LinuxQuestions.org - [SOLVED] rsync keeps overwriting some unmodified files

- Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)

- - rsync keeps overwriting some unmodified files (https://www.linuxquestions.org/questions/linux-software-2/rsync-keeps-overwriting-some-unmodified-files-780608/)

rsync keeps overwriting some unmodified files

Hello people.

I'm having this strange problem with rsync. I'm trying to synchronize my music collection with the music collection on my mp3 player (Cowon iAUDIO). The mp3 player has a harddisk with the FAT32 file system. The problem is that it keeps overwriting unmodified files on the mp3 player.

I use this command to synchronize:

Code:

rsync -avhpu --delete --progress --exclude-from "/home/serrano/backup-filters-iaudio.txt" "/windows/D/music/" "/media/IAUDIO/music/"

The above command should work, because I use the very same command (with different paths) to backup my linux home folder to an NTFS external harddisk. And that works just fine.

I've done some test to get a view of what's going on:

*Connect the IAUDIO*
*run the rsync command*
See attachment: rsync_result1.txt
*run the rsync command again*
See attachment: rsync_result2.txt (so far so good)
*Disconnect the IAUDIO*
*Connect the IAUDIO*
*run the rsync command again*
See attachment: rsync_result3.txt (overwrites unmodified files)

First, everything seems to work just fine. As seen in rsync_result2.txt, rsync skips the files that have already been synchronized.
You would expect the same result once I reconnect the mp3 player. But as seen in in rsync_result3.txt, some unmodified files are overwritten again. What also is very strange, is that it keeps overwriting the same files while it skips others without an apparent reason.

Could someone please explain to me what's happening?

System information:
openSUSE 11.2
rsync version 3.0.6 protocol version 30

Have you tried md5suming the files before and after rsync3?

Quote:

Originally Posted by rweaver (Post 3817860)

Have you tried md5suming the files before and after rsync3?

80d5ee2fa4823f7877aa56a991724687 01. Arctic Monkeys - My Propeller.mp3
*reconnect mp3*
80d5ee2fa4823f7877aa56a991724687 01. Arctic Monkeys - My Propeller.mp3
*run rsync command* ("01. Arctic Monkeys - My Propeller.mp3" is being overwritten)
80d5ee2fa4823f7877aa56a991724687 01. Arctic Monkeys - My Propeller.mp3
*run rsync command again* ("01. Arctic Monkeys - My Propeller.mp3" is ignored)
80d5ee2fa4823f7877aa56a991724687 01. Arctic Monkeys - My Propeller.mp3

So it indeed overwrites unmodified files. Could this have anything to do with the file dates?

Code:

core:~/test/test20$ touch fred-fred-burger

core:~/test/test20$ ls -al fred-fred-burger

-rw-r--r-- 1 core core 0 2010-01-08 12:00 fred-fred-burger

core:~/test/test20$ md5sum fred-fred-burger

d41d8cd98f00b204e9800998ecf8427e  fred-fred-burger



core:~/test/test20$ touch fred-fred-burger

core:~/test/test20$ ls -al fred-fred-burger

-rw-r--r-- 1 core core 0 2010-01-08 12:01 fred-fred-burger

core:~/test/test20$ md5sum fred-fred-burger

d41d8cd98f00b204e9800998ecf8427e  fred-fred-burger



core:~/test/test20$ touch fred-fred-burger

core:~/test/test20$ ls -al

total 12

drwxr-xr-x  2 core core 4096 2010-01-08 12:00 .

drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..

-rw-r--r--  1 core core  46 2010-01-08 11:45 datafile

-rw-r--r--  1 core core    0 2010-01-08 12:07 fred-fred-burger



core:~/test/test20$ mkdir ../test21

core:~/test/test20$ rsync -avhe ./* ../test21

core:~/test/test20$ ls -al ../test21

total 12

drwxr-xr-x  2 core core 4096 2010-01-08 12:07 .

drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..

-rw-r--r--  1 core core  46 2010-01-08 12:07 datafile

-rw-r--r--  1 core core    0 2010-01-08 12:07 fred-fred-burger



core:~/test/test20$ ls -al

total 12

drwxr-xr-x  2 core core 4096 2010-01-08 12:00 .

drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..

-rw-r--r--  1 core core  46 2010-01-08 11:45 datafile

-rw-r--r--  1 core core    0 2010-01-08 12:07 fred-fred-burger





core:~/test/test20$ touch fred-fred-burger

core:~/test/test20$ ls -al

total 12

drwxr-xr-x  2 core core 4096 2010-01-08 12:00 .

drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..

-rw-r--r--  1 core core  46 2010-01-08 11:45 datafile

-rw-r--r--  1 core core    0 2010-01-08 12:11 fred-fred-burger



core:~/test/test20$ rsync -avhe ./* ../test21

sending incremental file list

fred-fred-burger



sent 80 bytes  received 31 bytes  222.00 bytes/sec

total size is 0  speedup is 0.00



core:~/test/test20$ ls -al

total 12

drwxr-xr-x  2 core core 4096 2010-01-08 12:00 .

drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..

-rw-r--r--  1 core core  46 2010-01-08 11:45 datafile

-rw-r--r--  1 core core    0 2010-01-08 12:11 fred-fred-burger



core:~/test/test20$ ls -al ../test21

total 12

drwxr-xr-x  2 core core 4096 2010-01-08 12:11 .

drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..

-rw-r--r--  1 core core  46 2010-01-08 12:07 datafile

-rw-r--r--  1 core core    0 2010-01-08 12:11 fred-fred-burger



core:~/test/test20$ md5sum ./fred-fred-burger

d41d8cd98f00b204e9800998ecf8427e  fred-fred-burger

core:~/test/test20$ md5sum ../test21/fred-fred-burger

d41d8cd98f00b204e9800998ecf8427e  ../test21/fred-fred-burger



core:~/test/test20$ md5sum ./datafile

65d20a3372dc1d4d9deac22e0d6a987d  ./datafile

core:~/test/test20$ md5sum ../test21/datafile

65d20a3372dc1d4d9deac22e0d6a987d  ../test21/datafile

So yes, it will resend each time the time stamp changes. You can use --modify-window to avoid some of that behavior. Check the manpage.

Quote:

Originally Posted by rweaver (Post 3819024)

So yes, it will resend each time the time stamp changes. You can use --modify-window to avoid some of that behavior. Check the manpage.

rsync manpage:

Code:

      --modify-window

              When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than  the  modify-window  value.

              This  is  normally  0  (for  an  exact  match),  but  you  may  find it useful to set this to a larger value in some situations.  In

              particular, when transferring to or from an MS Windows FAT filesystem (which represents times with a  2-second  resolution),  --mod-

              ify-window=1 is useful (allowing times to differ by up to 1 second).

That did the trick! So it's the FAT filesystem that's causing this. Thank you very much rweaver! :D

So for everyone who's having the same problem, here's my modified rsync command:

Code:

rsync -avhpu --delete --progress --modify-window 1 --exclude-from "/home/serrano/backup-filters-iaudio.txt" "/windows/D/music/" "/media/IAUDIO/music/"