LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 01-07-2010, 01:29 PM   #1
figure002
Member
 
Registered: Nov 2008
Distribution: Ubuntu Maverick
Posts: 31

Rep: Reputation: 16
rsync keeps overwriting some unmodified files


Hello people.

I'm having this strange problem with rsync. I'm trying to synchronize my music collection with the music collection on my mp3 player (Cowon iAUDIO). The mp3 player has a harddisk with the FAT32 file system. The problem is that it keeps overwriting unmodified files on the mp3 player.

I use this command to synchronize:
Code:
rsync -avhpu --delete --progress --exclude-from "/home/serrano/backup-filters-iaudio.txt" "/windows/D/music/" "/media/IAUDIO/music/"
The above command should work, because I use the very same command (with different paths) to backup my linux home folder to an NTFS external harddisk. And that works just fine.

I've done some test to get a view of what's going on:

*Connect the IAUDIO*
*run the rsync command*
See attachment: rsync_result1.txt
*run the rsync command again*
See attachment: rsync_result2.txt (so far so good)
*Disconnect the IAUDIO*
*Connect the IAUDIO*
*run the rsync command again*
See attachment: rsync_result3.txt (overwrites unmodified files)

First, everything seems to work just fine. As seen in rsync_result2.txt, rsync skips the files that have already been synchronized.
You would expect the same result once I reconnect the mp3 player. But as seen in in rsync_result3.txt, some unmodified files are overwritten again. What also is very strange, is that it keeps overwriting the same files while it skips others without an apparent reason.

Could someone please explain to me what's happening?

System information:
openSUSE 11.2
rsync version 3.0.6 protocol version 30
Attached Files
File Type: txt rsync_result1.txt (7.0 KB, 3 views)
File Type: txt rsync_result2.txt (10.7 KB, 2 views)
File Type: txt rsync_result3.txt (8.1 KB, 2 views)
 
Old 01-07-2010, 01:41 PM   #2
rweaver
Senior Member
 
Registered: Dec 2008
Location: Louisville, OH
Distribution: Debian, CentOS, Slackware, RHEL, Gentoo
Posts: 1,833

Rep: Reputation: 163Reputation: 163
Have you tried md5suming the files before and after rsync3?
 
Old 01-07-2010, 01:57 PM   #3
figure002
Member
 
Registered: Nov 2008
Distribution: Ubuntu Maverick
Posts: 31

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by rweaver View Post
Have you tried md5suming the files before and after rsync3?
80d5ee2fa4823f7877aa56a991724687 01. Arctic Monkeys - My Propeller.mp3
*reconnect mp3*
80d5ee2fa4823f7877aa56a991724687 01. Arctic Monkeys - My Propeller.mp3
*run rsync command* ("01. Arctic Monkeys - My Propeller.mp3" is being overwritten)
80d5ee2fa4823f7877aa56a991724687 01. Arctic Monkeys - My Propeller.mp3
*run rsync command again* ("01. Arctic Monkeys - My Propeller.mp3" is ignored)
80d5ee2fa4823f7877aa56a991724687 01. Arctic Monkeys - My Propeller.mp3

So it indeed overwrites unmodified files. Could this have anything to do with the file dates?
 
Old 01-08-2010, 11:04 AM   #4
rweaver
Senior Member
 
Registered: Dec 2008
Location: Louisville, OH
Distribution: Debian, CentOS, Slackware, RHEL, Gentoo
Posts: 1,833

Rep: Reputation: 163Reputation: 163
Code:
core:~/test/test20$ touch fred-fred-burger
core:~/test/test20$ ls -al fred-fred-burger
-rw-r--r-- 1 core core 0 2010-01-08 12:00 fred-fred-burger
core:~/test/test20$ md5sum fred-fred-burger
d41d8cd98f00b204e9800998ecf8427e  fred-fred-burger

core:~/test/test20$ touch fred-fred-burger
core:~/test/test20$ ls -al fred-fred-burger
-rw-r--r-- 1 core core 0 2010-01-08 12:01 fred-fred-burger
core:~/test/test20$ md5sum fred-fred-burger
d41d8cd98f00b204e9800998ecf8427e  fred-fred-burger

core:~/test/test20$ touch fred-fred-burger
core:~/test/test20$ ls -al
total 12
drwxr-xr-x  2 core core 4096 2010-01-08 12:00 .
drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..
-rw-r--r--  1 core core   46 2010-01-08 11:45 datafile
-rw-r--r--  1 core core    0 2010-01-08 12:07 fred-fred-burger

core:~/test/test20$ mkdir ../test21
core:~/test/test20$ rsync -avhe ./* ../test21
core:~/test/test20$ ls -al ../test21
total 12
drwxr-xr-x  2 core core 4096 2010-01-08 12:07 .
drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..
-rw-r--r--  1 core core   46 2010-01-08 12:07 datafile
-rw-r--r--  1 core core    0 2010-01-08 12:07 fred-fred-burger

core:~/test/test20$ ls -al
total 12
drwxr-xr-x  2 core core 4096 2010-01-08 12:00 .
drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..
-rw-r--r--  1 core core   46 2010-01-08 11:45 datafile
-rw-r--r--  1 core core    0 2010-01-08 12:07 fred-fred-burger


core:~/test/test20$ touch fred-fred-burger
core:~/test/test20$ ls -al
total 12
drwxr-xr-x  2 core core 4096 2010-01-08 12:00 .
drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..
-rw-r--r--  1 core core   46 2010-01-08 11:45 datafile
-rw-r--r--  1 core core    0 2010-01-08 12:11 fred-fred-burger

core:~/test/test20$ rsync -avhe ./* ../test21
sending incremental file list
fred-fred-burger

sent 80 bytes  received 31 bytes  222.00 bytes/sec
total size is 0  speedup is 0.00

core:~/test/test20$ ls -al
total 12
drwxr-xr-x  2 core core 4096 2010-01-08 12:00 .
drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..
-rw-r--r--  1 core core   46 2010-01-08 11:45 datafile
-rw-r--r--  1 core core    0 2010-01-08 12:11 fred-fred-burger

core:~/test/test20$ ls -al ../test21
total 12
drwxr-xr-x  2 core core 4096 2010-01-08 12:11 .
drwxr-xr-x 27 core core 4096 2010-01-08 12:06 ..
-rw-r--r--  1 core core   46 2010-01-08 12:07 datafile
-rw-r--r--  1 core core    0 2010-01-08 12:11 fred-fred-burger

core:~/test/test20$ md5sum ./fred-fred-burger
d41d8cd98f00b204e9800998ecf8427e  fred-fred-burger
core:~/test/test20$ md5sum ../test21/fred-fred-burger
d41d8cd98f00b204e9800998ecf8427e  ../test21/fred-fred-burger

core:~/test/test20$ md5sum ./datafile
65d20a3372dc1d4d9deac22e0d6a987d  ./datafile
core:~/test/test20$ md5sum ../test21/datafile
65d20a3372dc1d4d9deac22e0d6a987d  ../test21/datafile
So yes, it will resend each time the time stamp changes. You can use --modify-window to avoid some of that behavior. Check the manpage.

Last edited by rweaver; 01-08-2010 at 11:28 AM.
 
Old 01-08-2010, 01:16 PM   #5
figure002
Member
 
Registered: Nov 2008
Distribution: Ubuntu Maverick
Posts: 31

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by rweaver View Post
So yes, it will resend each time the time stamp changes. You can use --modify-window to avoid some of that behavior. Check the manpage.
rsync manpage:
Code:
       --modify-window
              When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than  the  modify-window  value.
              This  is  normally  0  (for  an  exact  match),  but  you  may  find it useful to set this to a larger value in some situations.  In
              particular, when transferring to or from an MS Windows FAT filesystem (which represents times with a  2-second  resolution),  --mod-
              ify-window=1 is useful (allowing times to differ by up to 1 second).
That did the trick! So it's the FAT filesystem that's causing this. Thank you very much rweaver!

So for everyone who's having the same problem, here's my modified rsync command:
Code:
rsync -avhpu --delete --progress --modify-window 1 --exclude-from "/home/serrano/backup-filters-iaudio.txt" "/windows/D/music/" "/media/IAUDIO/music/"
 
  


Reply

Tags
files, overwrite, rsync


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Could I run rsync to download files from a server without rsync daemon? Richard.Yang Linux - Software 1 09-18-2009 04:08 AM
How to copy files without overwriting? Stephan_Craft Linux - Newbie 7 02-17-2009 05:28 AM
move files without overwriting files nathan2225 Linux - General 1 04-11-2008 04:01 PM
Please Help! Ubuntu keeps overwriting system files! ckr Ubuntu 2 07-13-2006 05:51 PM
php/ftp overwriting files.. prob. collen Programming 1 03-14-2005 04:56 AM


All times are GMT -5. The time now is 02:28 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration