[SOLVED] Can't preserve directory structure when copying files

andrew.comly · 06-29-2015, 01:25 AM

Copying files of certain type while preserving directory structure

I would like to copy all *.man files from /home/a/Manfiles to /home/a/Documents/Manfiles. The source directory /home/a/Manfiles has some *.man files in it, and it also has subdirectories, each having at least one *.man file in them.

Attempt 01:
I decide to use the 'cp' command with the '-r' option, as to not only copy over '/home/a/Manfiles' but also all of it's subdirectories and the '*.man' files they contain.

Code:

$ cp -pr --parents /home/a/Manfiles/*.man /home/a/Documents/Manfiles

but I get the results:

Code:

$ cp -pr --parents /home/a/Manfiles/*.man /home/a/Documents/Manfiles
cp: target /home/a/Documents/Manfiles is not a directory

Even if I first run 'mkdir /home/a/Documents/Manfiles', still there is no 'recursive' effect at all; No subdirectories(e.g. Manfiles/1, Manfiles/1/2) nor the '*.man' files they contain were copied.

Attempt 02:

Code:

find /home/a/Manfiles/ -type f -name '*.man' | cpio -p -dumv /home/a/Documents/Manfiles

This is the closest to being correct, especially since I didn't have to first create the directory structure before running this command (like in attempt 01). Unfortunately, it copies the entire directory structure, rather than just the recursive directory structure. What do I mean by this? I mean that my goal is to copy the directory structure AFTER(/to the right) of "Manfiles". I don't want directory structure BEFORE(/to the left, e.g. /home/a/). For example, I would like to achieve the target directory structure of /home/a/Documents/Manfiles/1/2, but what I really get is /home/a/Documents/Manfiles/home/a/Manfiles/1/2. Notice how the "/home/a" before "Documents" is a duplicate of the "home/a" between the "Manfiles". The two "Manfiles" are also duplicates.

Attempt 03:

Code:

find /home/a/Manfiles -name '*.man' -exec cp -rat /home/a/Documents/Manfiles {} +

This fails with the following output:

Code:

a@a-NC210-NC110:/home/a$ find /home/a/Manfiles/ -name '*.man' -exec cp -rat /home/a/Documents/Manfiles {} +
cp: failed to access /home/a/Documents/Manfiles: No such file or directory

Moreover, even as with attempt 01 and I first 'mkdir /home/a/Documents/Manfiles', this merely just copies all the *.man files from all directories within /home/a/Manfiles/ into the directory "/home/a/Documents/Manfiles", without preserving the directory structure(no subdirectories, e.g. Manfiles/1/2). This clearly isn't my goal.

veerain · 06-29-2015, 02:15 AM

Check whether this directory '/home/a/Documents/Manfiles' still exists and then

Try this:

Code:

cp -dpr /home/a/Manfiles/*.man /home/a/Documents/Manfiles

fatmac · 06-29-2015, 07:16 AM

Possibly,

Code:

cp -a /home/a/Manfiles /home/a/Documents

andrew.comly · 06-29-2015, 07:36 AM

Quote:

Originally Posted by veerain

Check whether this directory '/home/a/Documents/Manfiles' still exists and then

Try this:

Code:

cp -dpr /home/a/Manfiles/*.man /home/a/Documents/Manfiles

Veera,
I apologize for not explaining my situation better. Thanks for the help, but this produces nearly the same result as Attempt 01. I need a command that can recursively copy both a target directory and it's subdirectories that initially don't exist.

---------- Post added 06-29-15 at 08:37 PM ----------

Everyone,
Does anyone know of a way of producing the same result as Attempt 02: 'find /home/a/Manfiles/ -type f -name '*.man' | cpio -p -dumv /home/a/Documents/Manfiles', but without the duplicate directory problem(described in Attempt 02)?

andrew.comly · 06-29-2015, 07:58 AM

Quote:

Originally Posted by fatmac

Code:

cp -a /home/a/Manfiles /home/a/Documents

fatmac,
this copies everything, not just the *.man files. I just want to copy the directory structure (except any directories that don't have *.man files) and the *.man files.

andrew.comly · 06-29-2015, 08:34 AM

A possible use for this is the new android 4.4 standard video output '*.3gp'. Many new smartphones have 13 Mega-pixel cameras. Many of you think that 13 Mega-pixel is quite alot. Actually this is the lowest possible resolution necessary to shoot certain crucial code that is shortly displayed on a video screen during many extra help sessions where the TA explains how to make certain programs. Does the TA want to go slowly? No, he/she wants to go quickly so he/she can get to the people waiting in line for help BEHIND YOU.

Nine and a half minutes of video notes (recording) takes up an unbelievable 1.1 GB!! Thus there is a need to move all *.3gp files from a built in hard-drive (especially with a netbook that only has a SSD!) and onto a portable backup hard drive.

Of course, if you do this you MUST know which extra help session this came from, which most people would use directories named for the topic covered. It wouldn't make sense to name the video notes 3gp file and put them all into one directory since along with that 3gp file there are quite a few programs you wrote trying to emulate the TA's success, not to mention each topic being located in a certain chapter (subdirectories).

Hence preserving directory/subdirectory structure IS ESSENTIAL. In case anyone wants to know why I want to know something so exacting, then I hope this application can help cure your boredom.

veerain · 06-29-2015, 10:47 AM

Try this:

Code:

export dest_dir="/home/a/Documents/Manfiles"
mkdir -p "${dest_dir}"

export base_var="/home/a/Manfiles/"

find "${base_var}" -mindepth 1 -name "*\.man"
 | while read var1; do
     base_dir="`dirname "${var1#${base_var}}"`"
     mkdir -p "${dest_dir}/${base_dir}"
     cp -dp "${var1}" "${dest_dir}/${base_dir}"
    done

andrew.comly · 07-10-2015, 09:32 PM

veerain,
Thanks for your reply. But I am a bit behind on learning bash, and with your response I have started to dedicate two hours a day learning this. This way I will be able to understand your method more clearly, but I won't be able to try it out until at least 2 weeks later.

Thanks again!

unSpawn · 07-11-2015, 04:39 AM

Quote:

Originally Posted by andrew.comly

Attempt 02:

Code:

find /home/a/Manfiles/ -type f -name '*.man' | cpio -p -dumv /home/a/Documents/Manfiles

This is the closest to being correct, especially since I didn't have to first create the directory structure before running this command (like in attempt 01). Unfortunately, it copies the entire directory structure, rather than just the recursive directory structure. What do I mean by this? I mean that my goal is to copy the directory structure AFTER(/to the right) of "Manfiles". I don't want directory structure BEFORE(/to the left, e.g. /home/a/).

...then you 'cd' into that directory first as that "anchors" your 'find'? With 'cpio':

Code:

cd /home/a/Manfiles && find . -type d -name "*.man" | cpio -pdm /home/a/Documents/Manfiles

or 'install':

Code:

cd /home/a/Manfiles && find . -type d -name "*.man" -print0 | xargs -0 -iX install -d /home/a/Documents/Manfiles/'X'

andrew.comly · 07-22-2015, 09:55 PM

Veerain,
Again, I am still working on learning bash, have just grasped the two concepts of variables and exporting variables. From a previous C prog course I am familiar with "do .. while" conditions, but I still have the following question:

Quote:

Originally Posted by veerain

...

Code:

 | while read var1; do
     base_dir="`dirname "${var1#${base_var}}"`"
     mkdir -p "${dest_dir}/${base_dir}"
     cp -dp "${var1}" "${dest_dir}/${base_dir}"
    done

May I ask what "var1" is? Is it a third variable?

andrew.comly · 12-14-2015, 08:01 PM

I have accomplished what I was trying to do with rsync --prune-empty-dirs

Code:

$ LOCAL=media/a/LG/3gp
$ REMOTE=media/a/Sea_ext4/recent
rsync -Sai -Eu --stats --prune-empty-dirs -f "+ */" -f "+ *.3gp" -f "- *" /${REMOTE}/AC/Learn/Programming/C* /${LOCAL}

This really does exactly what I want. It first includes directories(or else it couldn't find target file types in subdirectories) with:

Code:

 -f "+ */"

-f stands for "filter rule", "+" at the beginning means "include", and "*/" means directory.

The second section of code

Code:

-f "+ *.3gp"

means include any file with name ending in 3gp.

The 3rd section of code

Code:

-f "- *"

means exclude any/every file. "-" means exclude. Naturally the wildcard "*" means any/every file.