Moving certain files recursively preservice folder structure

DigiTalk · 09-07-2005, 09:41 AM

Hi to all!

I already searched here, but found only a thread about moving files recursively into one directory.
What I need seems quite simple:

All files of a specific type in a folder and its sub-folders should be moved into another directory, but into the very same sub-folders.
But the original sub-folders may not be deleted (only if they are empty afterwards)

For example:
root
dir1
file.mov
file2.mov
file3.txt
dir11
file.ren
file3.mov
dir2
file.hak
dir22
file.mov

now all mov files should be moved into

root
movfiles
dir1
file.mov
file2.mov
dir11
file3.mov
dir2
dir22
file.mov

The dest-subfolders don't exist.
Is there any possibility to do this with one command?
Because I have hundreds of files in thousands of directories which should be moved (specifically - all drive-image files should be moved onto their own partition, but, for easier finding - in the same subdirectories as they have been before.)

Thanks in advance!

Greetings,
Digi

P.S. mv -r would be a great option if it would exist :-)

addy86 · 09-07-2005, 09:49 AM

I doubt that there exists a command which is so specific (it would be against the philosophy of Unix

). Instead you can write a short script which does exactly what you want.

Bruce Hill · 09-07-2005, 09:58 AM

Welcome to LQ!

I believe "cp -aR" will do what you want, although nothing is moved.
Which might be better, since you could check afterwards if you have
achieved your desired results.

theYinYeti · 09-07-2005, 10:32 AM

Code:

$ cd /source/dir/
$ find . -filters... -print | xargs tar cf - | (cd /dest/dir/ && tar xpvf - >/tmp/list)
$ while read file; do rm -f "$file"; done </tmp/list
$ rm -f /tmp/list

I can't test now, but it should work.

Yves.

DigiTalk · 09-08-2005, 03:22 AM

Thanx for your fast help, sorry for my late answer.

@Chinaman:
Thanx for the welcome :-)
I just tried cp - but it doesn't find the files (mov in my example) which may be due to the fact, that the real dirs contain spaces.
Also when using a test-directory structure without spaces, only the first file is copied.

@theYinYeti:
I'm very sorry, but I could get your script ro run...
Most likely because I'm doing something wrong...
I copied your code to a textfile, and executed as bash.
I replaced "$file" with "*.mov", but filters... gives me an error.

P.S. Using SuSE 9.3, logged in as root on tty2.

Thanx again for your help...

theYinYeti · 09-08-2005, 04:04 AM

I should have given more details. One never knows what knowledge others have

So for your specific need, it should be:

Code:

$ cd /source/dir/
$ find . -type f -iname "*.mov" -print | xargs tar cf - | (cd /dest/dir/ && tar xpvf - >/tmp/list)
$ while read file; do rm -f "$file"; done </tmp/list
$ rm -f /tmp/list

where /source/dir/ is the directory in which files have to be moved, and /dest/dir/ is the directory where the moved directory structure has to be put.

Advice:
- Please report here, before executing anything, if some of your files' names contain spaces in them.
- Make sure that /source/dir/ is not a place somewhere inside /dest/dir/, or the other way around, as this could lead to unpredictable results.
- Don't proceed with lines 3 and 4 until you have checked that the files you want to move are OK in the destination directory.
- Don't change directory ("cp" command) between lines 2 and 3.

Yves.

DigiTalk · 09-08-2005, 04:32 AM

Thanx for the detailed advice :-)

First: Yes, file names and folder names often contain spaces.
Second: I tried executing on a test folder structure / file structure and got an error which corresponds to the first point...
Tar could not find folders (which are given as parts of foldernames with spaces)

I won't start on the real files before my testsructure works correctly, because they are image files (between 100MB and 10GB big) on the fileserver of a mid-sized company and contain very important customer data...
So I test the commands on a test-machine with its own, unimportant linux-installation using a test-structure with spaces etc.

So, thanx for your help and especially your patence!

Bruce Hill · 09-08-2005, 05:07 AM

If you'd like to remove the spaces from the filenames and folders,
this works in one directory and changes the space to _ (underscore):

Code:

To remove the spaces in files ->
for i in *; do mv "$i" `echo $i | tr ' ' '_'`; done

And this bash script is a little more elegant:

Code:

#!/bin/bash
IFS='
'
j=`find $1 -printf "%d\n" | sort -u | tail -n 1`
j=$((j-1))
echo "Max dir depth:" $j
for (( i=0; i<=j ; i++ ))
do
  for name in `find -mindepth $i -maxdepth $i -iname "* *" -printf "%p\n"`
  do
    newname=`echo "$name" | tr " " "_"`
    echo "$name" "$newname"
    mv "$name" "$newname"
  done
done

Since spaces are escape codes in *nix, I had to rid all those Windoze
files and folders of them when migrating...especially music files.

DigiTalk · 09-08-2005, 05:26 AM

Unfortunately the files are on a samba share for Windows-clients...
And there are hundrets of files in thousands of folders.
And most files can't be renamed because the development software won't find them afterwards anymore...
So I need to preserve the spaces.

blindcoder · 09-08-2005, 06:11 AM

This should help:

Code:

blindcoder@ceres:~/tmp$ while read file ; do dirname=$(dirname "$file"); mkdir -p target/${dirname#source/}; mv "$file" "target/${file#source/}" ; done < <( find source/ -type f -a -name '*' )

Here's an example run:

Code:

blindcoder@ceres:~/tmp$ ls -lR
.:
total 8
drwxr-xr-x  6 blindcoder users 4096 Sep  8 13:05 source
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:05 target

./source:
total 16
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:05 a
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:06 b
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:06 c
drwxr-xr-x  3 blindcoder users 4096 Sep  8 13:06 d

./source/a:
total 0
-rw-r--r--  1 blindcoder users 0 Sep  8 13:05 foo

./source/b:
total 0
-rw-r--r--  1 blindcoder users 0 Sep  8 13:06 bar

./source/c:
total 0
-rw-r--r--  1 blindcoder users 0 Sep  8 13:06 Name with spaces

./source/d:
total 4
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:06 e

./source/d/e:
total 0
-rw-r--r--  1 blindcoder users 0 Sep  8 13:06 do not touch me

./target:
total 0
blindcoder@ceres:~/tmp$ while read file ; do dirname=$(dirname "$file"); mkdir -p target/${dirname#source/}; mv "$file" "target/${file#source/}" ; done < <( find source/ -type f -a -name '*' )
blindcoder@ceres:~/tmp$ ls -lR
.:
total 8
drwxr-xr-x  6 blindcoder users 4096 Sep  8 13:05 source
drwxr-xr-x  6 blindcoder users 4096 Sep  8 13:10 target

./source:
total 16
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:10 a
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:10 b
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:10 c
drwxr-xr-x  3 blindcoder users 4096 Sep  8 13:06 d

./source/a:
total 0

./source/b:
total 0

./source/c:
total 0

./source/d:
total 4
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:10 e

./source/d/e:
total 0

./target:
total 16
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:10 a
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:10 b
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:10 c
drwxr-xr-x  3 blindcoder users 4096 Sep  8 13:10 d

./target/a:
total 0
-rw-r--r--  1 blindcoder users 0 Sep  8 13:05 foo

./target/b:
total 0
-rw-r--r--  1 blindcoder users 0 Sep  8 13:06 bar

./target/c:
total 0
-rw-r--r--  1 blindcoder users 0 Sep  8 13:06 Name with spaces

./target/d:
total 4
drwxr-xr-x  2 blindcoder users 4096 Sep  8 13:10 e

./target/d/e:
total 0
-rw-r--r--  1 blindcoder users 0 Sep  8 13:06 do not touch me
blindcoder@ceres:~/tmp$

It won't remove empty directories, but this can be easily done in a second run:

Code:

blindcoder@ceres:~/tmp$ find source/ -type d -a -empty -exec rmdir {} \;

Run this command until all empty directories are removed.

blindcoder · 09-08-2005, 06:43 AM

I have put this into a shellscript for easier use:
http://shellscripts.org/project/mvpartial
Please read the instructions located at:
http://shellscripts.org/wiki/mvpartial

DigiTalk · 09-08-2005, 08:04 AM

This works perfectly well!
That is really gread and I didn't have to move around 20GB of image files around by hand (with really really long paths...)