LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)
-   -   How can I achieve this with rsync (or some other way)? (https://www.linuxquestions.org/questions/linux-general-1/how-can-i-achieve-this-with-rsync-or-some-other-way-4175658967/)

lucmove 08-11-2019 11:10 AM

How can I achieve this with rsync (or some other way)?
 
If I run rsync with the --delete-before option, it will delete from the destination everything that is not present in the source.

If I run sync with the -n option, it will tell me what it would have done and won't really do it.

What happens if I combine both? I get a report (mixed with other data) of everything that is present in the source and not in the destination.

Now, I want to use that report to actually copy those files present in the source and absent in the destination into a THIRD location.

Does rsync do that? I can't find it in the manual.

wpeckham 08-11-2019 07:00 PM

rsync does not do that itself, but you can certainly log that ooutput, filter it for a source list, and then copy those specific files to that third location.

If I may ask, why is it that you want to do this?

Beryllos 08-13-2019 12:03 AM

I don't see how the logic of --delete-before (or more simply --delete) applies to the task as you eventually described it.

Quote:

copy those files present in the source and absent in the destination into a THIRD location.
However, rsync can do what you want in one easy step. Use the --compare-dest option:

Code:

rsync -a FIRST --compare-dest=SECOND THIRD

Beryllos 08-13-2019 03:35 AM

Sorry, the rsync command I wrote in the previous post may result in the creation of some empty directories in THIRD. The --compare-dest=SECOND option compares and excludes files only.

Why is that? To compare and exclude directories would require comparison of the entire directory hierarchy, that is, the contents of the directory with all its subdirectories. This is possible in principle, but apparently rsync does not attempt it.

wpeckham 08-13-2019 06:34 AM

Quote:

Originally Posted by Beryllos (Post 6024600)
Sorry, the rsync command I wrote in the previous post may result in the creation of some empty directories in THIRD. The --compare-dest=SECOND option compares and excludes files only.

Why is that? To compare and exclude directories would require comparison of the entire directory hierarchy, that is, the contents of the directory with all its subdirectories. This is possible in principle, but apparently rsync does not attempt it.

Wait. Are you saying that you think that rsync only compares one level of the source and destination path?
What makes you think that? Have you examined the rsync man page?

Beryllos 08-13-2019 11:05 AM

Quote:

Originally Posted by wpeckham (Post 6024631)
Are you saying that you think that rsync only compares one level of the source and destination path?

No, I am not saying that, but my wording was perhaps ambiguous enough to suggest it.

What I meant to say is that rsync with compare-dest does all of its comparing, matching, and excluding at the level of individual files. The compare-dest option does not try to compare, match, or exclude deep structures such as whole directory trees with their contents.

The typical case where such a deep comparison would be desirable is a sparse backup of just the files and directories that have changed since a previous backup. Suppose that the user requires that the sparse backup contains only those directories which have changed at any subdirectory level, and within each subdirectory, only the files that have changed. Accordingly, if a whole directory and all of its contents are unchanged since the previous backup, the whole directory should be excluded from the sparse backup.

This is what I am saying rsync can't do, or at least I haven't figured out how to do it in a single rsync command. When a whole directory and all of its contents are unchanged since the previous backup, rsync with compare-dest will create the full directory tree containing no files at any level.

If one wishes to eliminate unnecessary empty directories, it can surely be done by other means, but before we go there, the Original Poster should clarify the requirements.

lucmove 08-14-2019 12:22 AM

The problem is that I am working with a relatively small file system that cannot hold everything, and I want to create the sparse backup somewhere else where it is isolated so I have a better view of the content and can delete files, reorganize it, etc.


All times are GMT -5. The time now is 10:30 PM.