LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-01-2015, 11:16 PM   #1
andrew.comly
Member
 
Registered: Dec 2012
Distribution: Lubuntu 14.04 LTS
Posts: 233
Blog Entries: 2

Rep: Reputation: Disabled
Question rsync: Can't turn off recursion


I would like to sync the folder ~/.maxima and all it's *.ini files to another folder, without copying/syncing its' folders/subdirectories. Unfortunatedly when I use rsync it not only updates the *.ini files, but also copies over ~/.maxima's subfolder: binary.

My preamble:
Code:
#!/bin/bash

#VARIABLES
#Initialize
unset BKROOT HMA

#Define
BKROOT=/media/username/Seagate/comp1
HMA=~

#wxMaxima
	if [ -d ${BKROOT}${HMA}/.maxima ]; then
		ATTEMPTS 1-4
	else
		cp -apr --parents ~/.maxima/*.ini ${BKROOT}/
	fi
ATTEMPTS 1-4
So far I've tried:
Code:
1) cp -apr --parents ~/.maxima/*.ini ${BKROOT}
2) rsync -Savi -Eu --no-r -stat --delete ~/.maxima ${BKROOT}${HMA}
3) rsync -Savi -Eu -stat --delete ~/.maxima/*.ini ${BKROOT}${HMA}/.maxima
4) rsync -Savi -Eu --exclude 'binary' -stat --delete ~/.maxima ${BKROOT}${HMA}
RESULTS
1) The cp command works great for the first backup when the directory ${BKROOT} doesn't exist yet, but in case you delete a file you don't need any more in the source folder, using this the next backup won't delete that file from the target directory.

2) Gets rid of old files you don't want, but it ALWAYS copies over the undesirable subfolder "binary".

3) This is nice because it won't copy over anything other than a *.ini file (even folders/directories it doesn't copy), but when you delete an old *.ini file on the source, it won't delete that file on the target.

4) A strength here is that it doesn't copy over that specific directory, but it does still copy over other folders/directories and other files (e.g. text.log, a.txt etc.)

Can rsync just sync "*.ini" files while excluding subdirectories/subfolders? Strange how in the rsync man pages there is no "maxdepth=" option.

Last edited by andrew.comly; 11-11-2015 at 08:07 AM. Reason: more examples
 
Old 11-02-2015, 12:24 AM   #2
berndbausch
Senior Member
 
Registered: Nov 2013
Location: Tokyo
Distribution: Redhat/Centos, Ubuntu, Raspbian, Fedora
Posts: 1,687

Rep: Reputation: 353Reputation: 353Reputation: 353Reputation: 353
Attempt 2 probably wants --stats instead of -stat. If you type -stat with a single dash (and without the final s), rsync interprets this as -s -t -a, and the -a reverses the earlier --no-r.

Now if you try
Code:
rsync -Savi -Eu --no-r --stats --delete ~/.maxima ${BKROOT}${HMA}
you will get the error message that --delete requires --dirs or --recursive. Add --dirs and all should be fine. Perhaps

Last edited by berndbausch; 11-02-2015 at 12:30 AM. Reason: lots of typos and additional info
 
1 members found this post helpful.
Old 11-02-2015, 12:36 AM   #3
berndbausch
Senior Member
 
Registered: Nov 2013
Location: Tokyo
Distribution: Redhat/Centos, Ubuntu, Raspbian, Fedora
Posts: 1,687

Rep: Reputation: 353Reputation: 353Reputation: 353Reputation: 353
Quote:
Originally Posted by berndbausch View Post
you will get the error message that --delete requires --dirs or --recursive. Add --dirs and all should be fine
Not quite, actually. This will not create .maxima on the target, and it will also copy the binary directory, though without content.

So I guess it requires a little more work, but perhaps it's a good start.
 
1 members found this post helpful.
Old 12-12-2015, 08:37 AM   #4
andrew.comly
Member
 
Registered: Dec 2012
Distribution: Lubuntu 14.04 LTS
Posts: 233
Blog Entries: 2

Original Poster
Rep: Reputation: Disabled
Talking results

Quote:
Originally Posted by berndbausch View Post
Attempt 2 probably wants --stats instead of -stat. If you type -stat with a single dash (and without the final s), rsync interprets this as -s -t -a, and the -a reverses the earlier --no-r.
Actually this just uncovers the fact that I never actually realized that in order to get a verbose set of statistics on the file transfer, one shouldn't use "-stat" nor even "-stats"; The option one should use is "--stats"! I really had been screwing this up for quite some time. Blame it on LibreOffice autocorrect, which always autochanges "--" to a long "-". ;(...Better late than never!)
Quote:
Originally Posted by berndbausch View Post
Now if you try
Code:
rsync -Savi -Eu --no-r --stats --delete ~/.maxima ${BKROOT}${HMA}
you will get the error message that --delete requires --dirs or --recursive. Add --dirs and all should be fine. Perhaps
I attempted:
Code:
REMOTE=media/a/LG/local;  MYHOME=home/a
$ rsync -Sai -Eu --stats --no-r --delete --dirs /home/a/.maxima /${REMOTE}/${MYHOME}
cd+++++++++ .maxima/

Number of files: 1 (dir: 1)
Number of created files: 1 (dir: 1)
Number of deleted files: 0
Number of regular files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 63
Total bytes received: 26

sent 63 bytes  received 26 bytes  178.00 bytes/sec
total size is 0  speedup is 0.00

$ find /home/a/.maxima    #displays everything in source folder
/home/a/.maxima
/home/a/.maxima/pm2.ini
/home/a/.maxima/am.ini
/home/a/.maxima/pm.ini
/home/a/.maxima/binary
/home/a/.maxima/binary/binary-gcl
/home/a/.maxima/binary/binary-gcl/share
/home/a/.maxima/binary/binary-gcl/share/draw
/home/a/.maxima/binary/binary-gcl/share/draw/gnuplot.o
/home/a/.maxima/binary/binary-gcl/share/draw/picture.data
/home/a/.maxima/binary/binary-gcl/share/draw/grcommon.o
/home/a/.maxima/binary/binary-gcl/share/draw/vtk.o
/home/a/.maxima/binary/binary-gcl/share/draw/grcommon.data
/home/a/.maxima/binary/binary-gcl/share/draw/vtk.data
/home/a/.maxima/binary/binary-gcl/share/draw/picture.o
/home/a/.maxima/binary/binary-gcl/share/draw/gnuplot.data
/home/a/.maxima/New.txt
Destination:
$ find local/home/a/.maxima/             #displays everything in destination folder
local/home/a/.maxima/
Thus it looks like "--no-r" takes "no recursive" one more step than I participated: it JUST copies over the directory, ITSELF. This really shows how computers take things quite literally.
 
Old 12-12-2015, 09:14 AM   #5
andrew.comly
Member
 
Registered: Dec 2012
Distribution: Lubuntu 14.04 LTS
Posts: 233
Blog Entries: 2

Original Poster
Rep: Reputation: Disabled
ORIGINAL PROBLEM: I would like to sync the folder ~/.maxima and all it's *.ini files to another folder, without copying/syncing its' folders/subdirectories.

Recently I have learned two new rsync options: 1) --exclude/include; 2) --prune-empty-dirs.


One can complete the above job with --exclude/include:

Method01
Code:
rsync -Sai -Eu --stats --exclude binary --exclude *.txt /${MYHOME}/.maxima /${REMOTE}/${MYHOME}
cd+++++++++ .maxima/
>f+++++++++ .maxima/am.ini
>f+++++++++ .maxima/pm.ini
>f+++++++++ .maxima/pm2.ini

Number of files: 4 (reg: 3, dir: 1)

For this small situation method01 works. But what about a more versatile solution for a more complex directory structure containing more types of files? For this we need --prune-empty-dirs:

Method02:

Code:
$ rsync -Sai -Eu --stats --include *.ini --prune-empty-dirs -f "+ */" -f "- *" /${MYHOME}/.maxima /${REMOTE}/${MYHOME}
cd+++++++++ .maxima/
>f+++++++++ .maxima/am.ini
>f+++++++++ .maxima/pm.ini
>f+++++++++ .maxima/pm2.ini

Number of files: 9 (reg: 3, dir: 6)
Number of created files: 4 (reg: 3, dir: 1)
Number of deleted files: 0
Number of regular files transferred: 3
#RSYNC manual file:
3155-3157) The combination of "+ */", "+ *.c", and "- *" would include all
directories and C source files but nothing else (see also the
--prune-empty-dirs option)
1801-1803) -F option is a shorthand for adding two --filter rules to your command.

This worked quite well. The '+' in '-f "+ */"' means include and the '*/' means directories(recursive). In -f "- *", the "-" means exclude and the "*" means any files.


But can't --prune itself include *.ini files also?

Method03:

Code:
$ rsync -Sai -Eu --stats --prune-empty-dirs -f "+ */" -f "+ *.ini" -f "- *" /${MYHOME}/.maxima /${REMOTE}/${MYHOME}
This is as effective as method02, but more concise.


But what if I had made some unwanted files on the destination(e.g. text.txt)? rsync's --prune-empty-dirs is also compatible with "--delete"! And remember, we don't need the "--no-r" option, thus no need to slavishly annex "--dirs" after "--delete":

METHOD04:
Code:
$ rsync -Sai -Eu --stats --delete --prune-empty-dirs -f "+ */" -f "+ *.ini" -f "- *" /${MYHOME}/.maxima /${REMOTE}/${MYHOME}
*deleting   .maxima/text.txt/
.d..t...... .maxima/
My personal favorite is METHOD04. Hope some of you can better understand rsync's --prune-empty-dirs versatility!

Last edited by andrew.comly; 12-12-2015 at 09:59 AM. Reason: spacing
 
  


Reply

Tags
automation, back-up, bash scripting, directories, rsync


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Quick Rsync Question, Recursion. BashTin Slackware 6 05-22-2015 07:30 AM
My CnMBook Manual page 5 shows how to turn off on desk top, but how do I turn on? salmagundi Linux - Newbie 2 04-04-2009 03:46 PM
Recursion in C hubabuba Programming 12 10-03-2005 08:46 AM
How to turn off Xserver in the gui and turn it off in the command line geminiviper Linux - Newbie 8 08-20-2004 09:05 AM
tar: '--no-recursion' option doesn't prevent recursion Earl Parker II Slackware 12 08-17-2004 03:49 AM


All times are GMT -5. The time now is 03:25 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration