I am posting this to the Newbie forum as it seems to be rather fundamental and an answer might be helpful here. In a nutshell, I have a bunch of files representing my monthly data backup in a stucture as shown below. My objective is to copy or move the files to two new directory trees - one containing the .tar.gz files and the other containing the .tar.gz.gpg files. Here is a sample of the source directory tree and files:

/data/stage/

/data/stage/data/
/data/stage/data/LibreOffice.tar.gz
/data/stage/data/LibreOffice.tar.gz.gpg
/data/stage/data/thunderbird.tar.gz
/data/stage/data/thunderbird.tar.gz.gpg

/data/stage/static/
/data/stage/static/PersonalDocuments.tar.gz
/data/stage/static/PersonalDocuments.tar.gz.gpg
/data/stage/static/Receipts&Statements.tar.gz
/data/stage/static/Receipts&Statements.tar.gz.gpg

The objective is to copy or move the documents to a structure like:

/data/proton/data/LibreOffice.tar.gz
/data/proton/data/thunderbird.tar.gz
/data/proton/static/PersonalDocuments.tar.gz
/data/proton/static/Receipts&Statements.tar.gz

/data/mega/data/LibreOffice.tar.gz.gpg
/data/mega/data/thunderbird.tar.gz.gpg
/data/mega/static/PersonalDocuments.tar.gz.gpg
/data/mega/static/Receipts&Statements.tar.gz.gpg

I can cp the entire /data/stage/ tree to /data/proton/ However, that copies both the .gz and .gpg files. I have not found any option to cp which will allow me to filter on only one file extension. No luck with mv either.

If I create the entire target tree manually to start with I can use a "find ... -exec" approach to copy of move the files one by one. And I could hard code the commands to create the target tree in my script. I am looking for a more elegant and flexible approach. Any advice greatly appreciated.

TIA,

Ken

for example tar or rsync can do that

https://askubuntu.com/questions/3336...m-being-copied
As Pan64 said, rsync is the best option.
In the linked example they refer to a directory to exclude. but you should be able to use *.gpg

Thanks folks,

I do not see how tar would help. I am already using tar to combine and compress the actual source data files on a directory by directory basis. For example LibreOffice.tar.gz contains all of my Libre Office documents, spreadsheets etc. gpg with the --batch option encrypts the tar.gz file and places the results in the corresponding tar.gz.pgp file. gpg does not seem to allow me to specify a target directory for the output file. It places the encrypted file in the same directory as the source file. That is the root of the problem.

rsync with the --remove-source-files could do the trick I guess. It would of course create second copies of each file during the process as opposed to using mv. With the relatively small amount of data I am dealing with (and it is located on a high speed PCIe SSD) the downside would only be academic.

Of course I could cp the whole tree to one target directory, delete all of the .gpg files from that tree, delete all the .gz files from the original tree and mv the original to the second target. That is sort of my typical cave man with a large stone hammer approach

Let me play with rsync a little.

Thanks again,

Ken

With tar it would be something like the following:

That is the basic method and then adjustments can be made to select only certain directories or files or deal with the cd a little differently.

Thanks Turbocapitalist,

I do not have a problem creating the tar files where I want them in the tree. My script runs a loop for each major source data directory (/data/data/, /data/static/, /data/reference/) and places the tar files for all data subdirectories in the appropriate destination.

/data/data/LibreOffice is tarred -> /data/stage/data/LibreOffice.tar.gz
/data/data/thindebird is tarred -> /data/stage/data/thunderbird.tar.gz

I could the upload the stage/ tree to ProtonDrive. The problem is I wish to encrypt each of these files before uploading to Mega. gpg --batch places the encrypted file in the same directory as the unencrypted file.

As to rsync, I do not think that will work. If I runrsync cannot find the files as they are in subdirectories under /data/stage/ If I run rsync finds the files in question but places them all in the /data/mega/ director. The recursive option is ignored. Am I missing an option with rsync?

Ken

The -a option is what you are probably looking for.

And mind the closing slash on the directory otherwise the result will be quite different from what you intend.

1 members found this post helpful.

Thanks again Turbocapitalist,

I will give that a try. I use rsync for some purposes and always to like to learn more about it. As to the original issue raised in this thread... I got a reply to an earlier thread

https://www.linuxquestions.org/quest...on-4175698453/

and now I can plunk my .gpg files exactly where I want them. No need to separate them as I was trying to do in this thread. Still it is good to have these options available.

Ken

Excellent. You can move files with rsync too. See the --remove-source-files option, not to be confused with the --delete option.

I don't see how -a can help here compared to just -r. I guess you have to issue two rsync commands: