Rsync for incremental backups?
I'm looking to find a way to get rsync to only do incremental backups of files. We have a fileserver, which in turns is running a script that continually runs rsync to back up files. However, rsync is copying every file every time. Some of these files are huge, and during our testing none of them were changed. I would like to know if there is a method to get rsync to compare files before copying somehow, rather than just copy each and every file. I've looked through the arguments for the rsync app and didn't see anything that would work. Right now, I believe the script is running it as:
It's copying everything to the right place, it's just doing it over and over again, overwriting the same files each time. Anyone have any ideas?
Re: Rsync for incremental backups?
Firstly, using -a makes -rpg unnecessary.
Secondly, -u actually should only copy
Thirdly, to check whether there are problems
you should temporarily remove -q.
That said, I've seen a similar problem in
a rsync backup where the target filesystem
didn't support linux permissions and attributes.
Where are you copying to?
rsync -ave ssh <LOCAL_DIR> <REMOTE_HOST>:<REMOTE_DIR> works fine for me.
Thanks for the replies.
Thanks for the tip about the arguments, I'll give it a shot without the -rpg to see if it helps any. As for the -q, I'm only using that to cut down on some of the text that shows up. Running it from terminal via python produces the same text over and over again ('file not found' when the folder is empty for instance), so I wanted to keep that to a minimum. Even before I did that though, I didn't see any errors. It just keeps copying all the files over each time.
Iainr, does that setup you're using do incremental backups, or is that doing everything each time it runs? I thought the -u argument was needed for incremental.
Edit: Couple things to add. First, it seems it's not copying every file over each time. It is however copying the same group of files over each time. I don't know how it's choosing the files it's moving, but there are still quite a few that are copied each time. It doesn't matter if I use the -u or not either.. it always seems to only do certain files, even though none of them have been changed.
Second, it appears I'm not the only one having the problem. A quick check at http://samba.anu.edu.au/rsync/ shows that this is in the FAQ, and that people are still experiencing this issue, and have been for a long time. Unfortunately, it seems that nobody have found a way to get it to copy only modified files yet.
|All times are GMT -5. The time now is 02:51 AM.|