LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 12-12-2007, 11:56 AM   #1
deadlock
Member
 
Registered: Apr 2003
Distribution: Red Hat
Posts: 65

Rep: Reputation: 15
Question backup changed files only.


Trying to work out whether the following is possible.

Day 1: Backup all files
Day 2: Backup only those files changed from Day 1
Day 3: Backup only files changed from Day 2
Day 4: etc...
Day 8: Full backup...

Therefore to get back to Day 3 you restore your Day 1, 2 and 3 backups in sequence.

rsync --backup seems to do the reverse of what I want - i.e. I get a folder with all the Day 1 files, a folder with Day 1 + Day 2 files, a folder with Day 1, Day2 + Day 3 files...

Any pointers?
 
Old 12-12-2007, 11:58 AM   #2
pljvaldez
Guru
 
Registered: Dec 2005
Location: Somewhere on the String
Distribution: Debian Squeeze (x86)
Posts: 6,092

Rep: Reputation: 269Reputation: 269Reputation: 269
I use Backuppc to do just this. It utilizes rsync as well.
 
Old 12-12-2007, 12:02 PM   #3
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 62
There are similar cases to this on the rsync examples page: http://rsync.samba.org/examples.html

Or an alternative approach described here: http://www.mikerubel.org/computers/r...s/#Incremental
 
Old 12-13-2007, 04:23 AM   #4
deadlock
Member
 
Registered: Apr 2003
Distribution: Red Hat
Posts: 65

Original Poster
Rep: Reputation: 15
Unhappy

I can only get the the rsync examples to do the reverse of what I want.

For example if files have not changed over the week, then I get 7 copies of same file (1 in each incremental folder). I only want those files in the Day 1 backup.

Will check the other suggestions.
 
Old 12-13-2007, 05:36 AM   #5
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 62
Which method do you use?
 
Old 12-13-2007, 05:44 AM   #6
deadlock
Member
 
Registered: Apr 2003
Distribution: Red Hat
Posts: 65

Original Poster
Rep: Reputation: 15
Code:
BACKUPDIR=~/test/backups/`date +%R:%S`
rsync -b --backup-dir=$BACKUPDIR -rv testsrc/ ~/test/backups/current
Backupdir is H:M:S for testing - will by %F when live

If you run the above without changing anything in testsrc you get a new "dated" folder identical to "current"
 
Old 12-13-2007, 07:41 PM   #7
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,189

Rep: Reputation: 105Reputation: 105
Are you sure you aren't looking at hard links? You may have a single copy of an unchanged file and the folder with the image of all the directory structure on the next occasion has a hard link to it because it hasn't changed. For files that changed, it may remove the link and then copy the new file in. I'm speculating from your description, but this is the way some procedures work.
 
Old 12-14-2007, 04:50 AM   #8
deadlock
Member
 
Registered: Apr 2003
Distribution: Red Hat
Posts: 65

Original Poster
Rep: Reputation: 15
Definatly not links.

Steps as follows:

1) Create folder in ~ called test
2) Create folder ~/test/testsrc and ~/test/backups
3) Create script containing only the code given above
4) touch testsrc/test1.txt
5) Run the script
6) touch testsrc/test2.txt

At this point you should have the following tree in backups:

~/test/backups/current/test1.txt
~/test/backups/current/test2.txt
~/test/backups/10:45:00/test1.txt

Edit test1.txt in current and it has no effect on test1.txt and vice versa
 
Old 12-14-2007, 08:03 AM   #9
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,189

Rep: Reputation: 105Reputation: 105
What I meant was links between your multiple backups. The explicit example you just gave is only one backup.

However, looking into it a bit more, it's pretty clear this isn't the issue. You are simply creating new directories and doing new full backups.

Check out the link mathewg42 gave to Mike Rubel's web page. Or get a copy of the O'Reilly book "Backup & Recovery" by W. Curtis Preston. This is all spelled out in Chapter 7 which covers rsync with snapshots, rsnapshot, rdiff, etc. rsnapshot implements the whole procedure and may be available with your distribution.

If you want to script it yourself with rsync, the basic idea is to first create a full backup with rsync. Then replicate the full backup using only hardlinks. Then rsync to the replicate letting it delete the links for those files that have changed and copy the new files in. Repeat the process on the most recent replicate of the backup. I would also change the -rv that you are using to -av. That catches more options that you want for a backup.

so, initially

Code:
BACKUPDIR=~/test/backups/`date +%R:%S`
rsync -av testsrc/. $BACKUPDIR
then

Code:
NEWDIR=~/test/backups/`date +%R:%S`     {this will be different now}
cp -al $BACKUPDIR/. $NEWDIR/            {copy is all just links}
rsync --delete -av testsrc/. $NEWDIR    {replace links only for changed content}
There are variations on this, and there are options in rsync that will create the links. And there is also rsnapshot. And, of course, jumping off in other directions, BackupPC, Amanda, . . . which can get more complicated.
 
Old 12-14-2007, 11:36 AM   #10
deadlock
Member
 
Registered: Apr 2003
Distribution: Red Hat
Posts: 65

Original Poster
Rep: Reputation: 15
In the end I found a script example using find and tar taht does what I want.

So now if it's a Sunday I do a full backup of the source tree, and every other day do the same for files changed in the last 24 hours.

Thanks for the help

Dan
 
  


Reply

Tags
backup, script


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
all my text files have changed to realmedia files erikus Linux - General 2 12-28-2006 04:43 PM
logwatch on RHEL keeps saying files have changed laggerific Linux - Software 2 02-23-2006 01:46 PM
10.1 -> 10.2 which conf files changed? Vgui Slackware 3 09-21-2005 04:55 PM
IP addy changed, which files do I edit? WorldBuilder Linux - Networking 1 08-24-2005 10:01 PM
Howto list last changed files MicroSun Linux - Newbie 3 02-18-2005 05:52 PM


All times are GMT -5. The time now is 11:07 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration