LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-14-2017, 01:58 AM   #1
babag
Member
 
Registered: Aug 2003
Posts: 370

Rep: Reputation: 30
rsync question


i want to back up certain files but want to keep ALL versions of the sources in the backup directory and haven't been able to find a good method. the gui based offerings don't seem to have the depth or, if they do, they'd require so many switches that it seems like just going to a script would be better.

i keep a series of files with ideas for projects. i'm constantly amending these files. being that this is something of a creative process, there are often a lot of changes, additions, deletions to these files. it would be nice to be able to have my backup directory keep all of the various iterations of these files so that i can go back in time if it seems like i might have gotten side-tracked somewhere and want to go back and re-orient myself.

does rsync have anything that will allow for keeping many copies of the same file, maybe using the timestamp to differentiate?

thanks,
BabaG
 
Old 02-14-2017, 03:34 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 13,576

Rep: Reputation: 4341Reputation: 4341Reputation: 4341Reputation: 4341Reputation: 4341Reputation: 4341Reputation: 4341Reputation: 4341Reputation: 4341Reputation: 4341Reputation: 4341
rsync can use different directories day by day for example. probably you can use this as a starting point: http://www.mikerubel.org/computers/rsync_snapshots/
by the way if you want to handle source files you ought to go with git.
 
Old 02-14-2017, 03:49 AM   #3
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 4,428
Blog Entries: 3

Rep: Reputation: 2205Reputation: 2205Reputation: 2205Reputation: 2205Reputation: 2205Reputation: 2205Reputation: 2205Reputation: 2205Reputation: 2205Reputation: 2205Reputation: 2205
Several options can be used together in doing incremental backups to avoid wasting space on unchanged files:

https://rsync.samba.org/examples.html
https://blog.interlinked.org/tutoria...e_machine.html
 
Old 02-14-2017, 10:26 AM   #4
agillator
Member
 
Registered: Aug 2016
Distribution: Mint 19.1
Posts: 273

Rep: Reputation: Disabled
Take a look at rsnapshot. It sounds as if it does exactly what you describe and you wouldn't have to reinvent the wheel. It uses rsync and hard links.
 
Old 02-14-2017, 10:44 AM   #5
TenTenths
Senior Member
 
Registered: Aug 2011
Location: Dublin
Distribution: Centos 5 / 6 / 7
Posts: 3,019

Rep: Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275
Take a look at dirvish this works quite well for giving you a folder structure where you can easily go back to specific points in time.
 
Old 02-14-2017, 12:14 PM   #6
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2137Reputation: 2137Reputation: 2137Reputation: 2137Reputation: 2137Reputation: 2137Reputation: 2137Reputation: 2137Reputation: 2137Reputation: 2137Reputation: 2137
Yes rsync can do this, you want the "--link-dest" flag. Every backup you do should go to a new, clean directory, and use --link-dest to point to the previous backup. Rsync will compare the source file to the same file in the previous backup, if they're the same, it will hard link the file from the previous backup to the new backup. If they are different, rsync will copy the file fresh from the source to the new backup. The end result is you have two completely independent and navigable backup directories for the two days, but they only take up the space of one full backup plus the files that changed between the two backups.
 
Old 02-14-2017, 01:41 PM   #7
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,360

Rep: Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001
There is also rdiff-backup, which combines the best features of a mirror and an incremental backup, working by maintaining history as a series of reverse diffs working back from a current mirror. It is not without its faults, being essentially unmaintained since 2009, but is considered good enough and unique enough to be used as the back end for a new project, safekeep.
 
Old 02-14-2017, 04:08 PM   #8
TenTenths
Senior Member
 
Registered: Aug 2011
Location: Dublin
Distribution: Centos 5 / 6 / 7
Posts: 3,019

Rep: Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275Reputation: 1275
Quote:
Originally Posted by suicidaleggroll View Post
Yes rsync can do this, you want the "--link-dest" flag. Every backup you do should go to a new, clean directory, and use --link-dest to point to the previous backup. Rsync will compare the source file to the same file in the previous backup, if they're the same, it will hard link the file from the previous backup to the new backup. If they are different, rsync will copy the file fresh from the source to the new backup. The end result is you have two completely independent and navigable backup directories for the two days, but they only take up the space of one full backup plus the files that changed between the two backups.
This is the underlying mechanism that dirvish uses, it's effectively a wrapper script around rsync.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync with --link-dest: Question about interrupted rsync session bvz Linux - Server 9 09-10-2016 12:31 PM
rsync question hepburnenthorpe Linux - General 1 11-19-2006 06:19 PM
rsync question nazs Linux - Software 1 10-23-2006 08:31 PM
Rsync Question stefaandk Linux - General 2 05-13-2006 12:12 AM
Another Rsync question phatboyz Linux - Networking 2 07-02-2004 09:46 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:54 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration