Find command and rsync combine

b-RAM · 05-27-2009, 11:31 PM

Hi all, sorry for disturbing your time i've got some question to ask,
laterly i try to backup just new modified file to my usb harddrive using find command and rsync like this :

Code:

find /home/* -type f -mtime 0 -exec rsync -avz {} /mnt/usb_harddrive/home/ \;

the result is all new files can be backup but the directory of that file is not backup, can anyone advise me about that?, cause i know rsync just backup the file based the path on find command but if that command only backup the new files without the directory it will be messy' and in the future i will confuse when loaded that files.
I also had try to delete -type f but the result is it's backup not only
backup the directory and new modified file but also old files had been backup.

Need help and advise please.
Thank's a lot.

billymayday · 05-27-2009, 11:37 PM

Why bother with find at all? rsync should take care of changed/new files - that's what it does.

b-RAM · 05-28-2009, 12:38 AM

Thanks for your reply billymayday. ^^
I use find at all cause i use a new HDD backup if i just use rsync to backup home data to that new HDD Backup, i will just backup all files to HDD backup while the last files had been backup to my old HDD Backup. So, on my new HDD backup i just need to backup change/modified files and in the end i have 2 Backup one old and one new with modified/change file/new file.
Need advise or any method can you tell me to backup new modified file/change file with include the directory.

Thanks a lot for your reply.

billymayday · 05-28-2009, 12:55 AM

If you want to preserve the path, it may be simplest to add the files to a tar archive,

b-RAM · 05-28-2009, 01:08 AM

Sorry, do you mean to make that files on new hdd backup became tar files?, but when i extract that tar file someday it's include
with directory something like this:
when i backup new modified files it's include the directory where that files i backup ex: stock.xls on folder /home/warehouse/stock_data/.
And sorry billymayday for my noob, how do i make the backup become tar on new HDD Backup.

Thank's a lot for your advise and help.

billymayday · 05-28-2009, 01:23 AM

Something like

find /home/bill/temp/ -type f -exec tar -rf /home/bill/test/b.tar {} \;

AlucardZero · 05-28-2009, 09:35 AM

Code:

       -R, --relative
              Use  relative  paths.  This means that the full path names specified on the command
              line are sent to the server rather than just the last parts of the filenames.  This
              is  particularly  useful when you want to send several different directories at the
              same time. For example, if you used this command:

                 rsync -av /foo/bar/baz.c remote:/tmp/

              ... this would create a file named baz.c in /tmp/ on the remote machine. If instead
              you used

                 rsync -avR /foo/bar/baz.c remote:/tmp/

              then  a  file named /tmp/foo/bar/baz.c would be created on the remote machine, pre‐
              serving its full path.  These extra path elements are called “implied  directories”
              (i.e. the “foo” and the “foo/bar” directories in the above example).

?

billymayday · 05-28-2009, 03:13 PM

Good call. I'd thought there was something along those lines but couldn't find it for some reason.

Quakeboy02 · 05-28-2009, 04:09 PM

AlucardZero, I think you forgot something in your post:

Quote:

Beginning with rsync 3.0.0, rsync always sends these implied
directories as real directories in the file list, even if a path
element is really a symlink on the sending side. This prevents
some really unexpected behaviors when copying the full path of a
file that you didn't realize had a symlink in its path.

IOW, -R is now the default behavior. I did a doublecheck on my backups and the paths are preserved within the directory structure of the backup directory.

Example command:

Code:

rsync -av /home/qb2  /data/qb2/backup

The example file /home/qb2/Chess/Bird.pgn would be saved as /data/qb2/backup/qb2/Chess/Bird.pgn

Or am I misunderstanding the whole issue?

BTW, if the object of this is to make a faithful backup of a whole drive, then the proper way would probably be to setup a RAID1.

billymayday · 05-28-2009, 04:32 PM

I ran a quick test yesterday and it didn't keep the paths.

billymayday · 05-28-2009, 04:47 PM

Actually it works with the -a option that I didn't test with of course

Edit - actually is doesn't, and I assume this is because find passes the full filename, not just a higher path (ie, rsync -av /home/qb2/Chess/bird.pgn /data/qb2/backup, would produce /data/qb2/backup/Bird.pgn, not /data/qb2/backup/qb2/Chess/Bird.pgn)

Quote:

find ~/temp -type f -exec rsync -avR {} ~/test \;

seems to work OK though (although it puts the whole path in place):

Quote:

$ ll /home/bill/temp
total 8.0K
-rw-r--r-- 1 bill users 289 May 28 13:29 t1
-rw-r--r-- 1 bill users 289 May 28 13:30 t2

$ ll /home/bill/test/home/bill/temp
total 8.0K
-rw-r--r-- 1 bill users 289 May 28 13:29 t1
-rw-r--r-- 1 bill users 289 May 28 13:30 t2

Quakeboy02 · 05-28-2009, 06:27 PM

Quote:

Originally Posted by billymayday

Edit - actually is doesn't, and I assume this is because find passes the full filename, not just a higher path (ie, rsync -av /home/qb2/Chess/bird.pgn /data/qb2/backup, would produce /data/qb2/backup/Bird.pgn, not /data/qb2/backup/qb2/Chess/Bird.pgn)

You're misquoting me. I said that for the command "rsync -av /home/qb2 /data/qb2/backup" an example file "/home/qb2/Chess/Bird.pgn" would be saved as "/data/qb2/backup/qb2/Chess/Bird.pgn".

billymayday · 05-28-2009, 06:30 PM

@b-RAM, you may want to try something like:

Code:

touch /home/increment

then write a backup script along the lines of:

Code:

find /home/* -type f -newer /home/increment -exec rsync -avzR {} /mnt/usb_harddrive/home/ \;
touch /home/increment

What this does it timestamp /home/increment, then find everything newer than touch /home/increment, rsync those files, then update the timestamp for the next run. This way, if you backup doesn't run for any given period, it will still backup anything changed in the meantime.

I haven't tested the exact code, but the jist is correct.

HTH.

BM.

billymayday · 05-28-2009, 06:33 PM

Sorry - that's not what I meant. What you have works as you've stated. What I meant is that it doesn't work in conjunction with the find command. I was trying to put it in context for what the OP is trying to do.

Does that make sense? Apologies for any confusion.

Quakeboy02 · 05-28-2009, 07:16 PM

Quote:

Originally Posted by billymayday

Does that make sense? Apologies for any confusion.

Oh. Oops. My mistake.