LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 01-10-2012, 06:51 PM   #1
imagiro1
LQ Newbie
 
Registered: Jul 2011
Posts: 9

Rep: Reputation: Disabled
Rsync dry run


I am wanting to use rsync to backup a folder with pictures to a linux box. I copied all the pictures over to the drive via USB and now I am doing a dry run to see what will be transferred over. It looks like it's going to copy everything over again. And not just the test file I added.

Code:
rsync -avzr --dry-run --delete --exclude '.DS_Store' "/Users/diegoaguilera/Pictures" root@192.168.2.4:/media/F/Photos
When I run the code it returns the following.
Code:
Pictures/random/Tree Project/2011-01-03/IMG_1823.JPG
Pictures/random/Tree Project/2011-01-03/IMG_1824.JPG
Pictures/random/Tree Project/2011-01-03/IMG_1825.JPG
Pictures/random/Tree Project/2011-01-03/IMG_1826.JPG
Pictures/random/Tree Project/2011-01-03/IMG_1827.JPG
Pictures/random/Tree Project/2011-01-03/IMG_1828.JPG
Pictures/random/Tree Project/2011-01-03/IMG_1829.JPG
Pictures/random/Tree Project/2011-01-03/IMG_1830.JPG
Pictures/random/Tree Project/2011-01-03/IMG_1831.JPG

sent 2116907 bytes  received 271814 bytes  62044.70 bytes/sec
total size is 79881373728  speedup is 33441.06
I've omitted each file cause it's every picture I have. A few thousands. Am I using the command wrong? Do I need to need to sync the folder from zero? I am trying not to transfer 50gigs of pictures. Thanks.
 
Old 01-10-2012, 07:03 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,987
Blog Entries: 54

Rep: Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742
Rsync has different modes to compare files with. Some are more "expensive" (less efficient) compared to others but may provide the right solution for you. I think you want "--size-only" (which AFAIK implies "--ignore-times") but you will want to explicitly add "--times" to keep subsequent runs efficient. If that works OK for you then you should have initially copied files over with 'cp --preserve=timestamps'.
 
Old 01-10-2012, 10:20 PM   #3
rng
Member
 
Registered: Aug 2011
Posts: 746

Rep: Reputation: 23
The formats of the drive also affect. What formats are there on disk and on usb flash drive?
 
Old 01-10-2012, 10:34 PM   #4
imagiro1
LQ Newbie
 
Registered: Jul 2011
Posts: 9

Original Poster
Rep: Reputation: Disabled
Sorry unSpawn, maybe my question wasn't clear or I'm not following your answer. I copied the folder to a USB drive and connected it to the linux box. When I run the rsync command I mentioned above it should not copy anything because the files and folders are the same. Instead it scrolls through every file and folder.

I've tried making a test directory on both machines. I created 3 test files on each machine. When I run the command, again it wants copy all 3 files over. I'm guessing I have to run rsync and copy the files via the network and not USB?

The main machine is a Macbook so HFS+. The USB drive I'm trying to write to is NTFS.
 
Old 01-11-2012, 11:07 AM   #5
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,987
Blog Entries: 54

Rep: Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742
Quote:
Originally Posted by imagiro1 View Post
When I run the rsync command I mentioned above it should not copy anything because the files and folders are the same.
"Same" has different meanings: time stamps, hash, file size.


Quote:
Originally Posted by imagiro1 View Post
I'm guessing I have to run rsync and copy the files via the network and not USB?
Depending on what switches you use that should not matter.


BTW, noticed something in the command line. See 'man rsync' about end slash usage and try this --dry-run ( == -n):
Code:
rsync -navr --exclude ".DS_Store"' /Users/diegoaguilera/Pictures/ /media/F/Photos/

Last edited by unSpawn; 01-11-2012 at 11:12 AM.
 
Old 01-11-2012, 12:44 PM   #6
imagiro1
LQ Newbie
 
Registered: Jul 2011
Posts: 9

Original Poster
Rep: Reputation: Disabled
By "same" I want the file structure to be the same. I would like to keep the date of when the file was created.

I just want to have my pictures and eventually my documents folder sync'ed to the linux box. So if my computer dies, I can just copy the directories from usb to my computer. So I want to do incremental backups.

I thought this was going to be a simple project.
 
Old 01-11-2012, 06:57 PM   #7
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,987
Blog Entries: 54

Rep: Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742
Quote:
Originally Posted by imagiro1 View Post
I thought this was going to be a simple project.
Sometimes it helps to just focus on your goal. Do post if the line I suggested works.
* BTW just running Rsync doesn't equal making incremental backups. Best read up on backup schemes so you know the terminology and pitfalls.
 
Old 01-11-2012, 07:17 PM   #8
imagiro1
LQ Newbie
 
Registered: Jul 2011
Posts: 9

Original Poster
Rep: Reputation: Disabled
Sorry. I think you had an extra ' in the command. I removed it and it works.
 
Old 01-11-2012, 09:43 PM   #9
imagiro1
LQ Newbie
 
Registered: Jul 2011
Posts: 9

Original Poster
Rep: Reputation: Disabled
Thanks for all the help unSpawn. I decided that all I needed was a mirror of the folder. I ended up using the following command.

Code:
rsync -avzrh --progress --delete --exclude '.DS_Store' --exclude '.localized' --exclude 'Lightroom' --exclude 'iChat Icons' "/Users/diegoaguilera/Pictures/" root@192.168.2.4:/media/F/Pictures
I'll remove the v, h, and --progress later.

I did have to start from zero. It's currently copying files and will be for a while. No matter what I tried, I could not copy the files over via USB and then run the rsync command. It always wanted to delete and copy everything over. I guess I can't win them all.
 
Old 01-12-2012, 10:00 AM   #10
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,987
Blog Entries: 54

Rep: Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742Reputation: 2742
I'm not sure about HFS+ (rather loath to fire up the old machine) but there was something with NTFS and time stamps. And if you force --delete then yes, it will delete everything and copy over files. Other than that you seem happy with what you've achieved so thanks for posting back the command you use and please mark the thread solved.

Last edited by unSpawn; 01-12-2012 at 10:48 AM. Reason: //typo
 
Old 01-12-2012, 12:20 PM   #11
Cedrik
Senior Member
 
Registered: Jul 2004
Distribution: Slackware
Posts: 2,140

Rep: Reputation: 242Reputation: 242Reputation: 242
rsync accepts --exclude-from <FILE> option that will exclude all files listed in file FILE...

It could be a solution to use rsync with usb drive directly, with the exclude file updated after each rsync operation
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Patch cluster dry run SteveK1979 Solaris / OpenSolaris 2 10-20-2010 05:47 AM
Could I run rsync to download files from a server without rsync daemon? Richard.Yang Linux - Software 1 09-18-2009 04:08 AM
Dry Run of Ubuntu Installation daniell Linux - Newbie 2 11-04-2008 06:52 PM
do u dry eye? cigarstub General 8 07-25-2006 05:30 AM


All times are GMT -5. The time now is 11:00 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration