[SOLVED] Bash help needed: How to recursively browse subfolders in a for loop?

frisil · 02-19-2010, 12:44 PM

Hi,
I wrote a script for copying mp3 files to my girlfriend's mp3-player and it looks like this:

Code:

mount /dev/sdc /test -o sync
for i in *.mp3; do 
  cp $i /test/Musik/newalbum; 
  sleep 5; 
done;

This is the only way to make the player see the files in the correct order and it finally works! (after toying with rsync and timestamps in vain) Thus, the 5 seconds pause after each copy is necessary to make sure the player's internal controller did really write the file, so I cannot use xcopy or rsync.

So, I have to go to the album folder on my hd and call the script, and the album is transfered to the player. So far so good.

But I want the script to browse subfolders also, something like this (pseudo code

Code:

for dir=all_sub_folders; do
  mkdir /test/Musik/$dir
  for i in *.mp3; do 
    cp $i /test/Musik/$dir; 
    sleep 5; 
  done;
done;

How do I realize that (dir=all_sub_folders) in bash?

GrapefruiTgirl · 02-19-2010, 12:48 PM

Rather than using the for--do--done loop to find all the files, why not either use:

1) `find` -- which will recursively enter subfolders automatically, or
2) just use the -R switch on your `cp` command, to copy recursively?

Note that both these methods do not create your new subdirs, so you'd need to add some code to create those subdirs first; maybe do a preliminary `find` operation first, to identify all subdir names, and create them in the target area, and THEN do the `find` recursively for the files themselves, and copy them over to their respective subdirs?

acid_kewpie · 02-19-2010, 12:49 PM

Code:

find -d /where/ever | while read DIR
do
  ... something using $DIR...
done

frisil · 02-19-2010, 01:09 PM

Quote:

Originally Posted by acid_kewpie

Code:

find -d /where/ever | while read DIR
do
  ... something using $DIR...
done

Seems interesting, but does not work. I tried the following for a test:

Code:

#!/bin/sh
find -d /home/bert/00fuertigerplayer | while read DIR
do
echo $DIR found
done

and get the following error:

Code:

find: warning: the -d option is deprecated; please use -depth instead, because the latter is a POSIX-compliant feature.
find: Der Pfad muß vor dem Suchkriterium stehen: /home/bert/00fuertigerplayer
Aufruf: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [Pfad...] [Suchkriterium]

translation: path must be before search criteria

when I use

Code:

#!/bin/sh
find -d | while read DIR
do
echo $DIR found
done

in the correct folder I get every file listed, not just the folders.

frisil · 02-19-2010, 02:27 PM

I'm very close now with "for directory in `find . -type d `", but it does not work when there are space in the folder names, and some folders have spaces. I tried:

Code:

for directory in `find . -type d `
do
echo $directory found
done

for a test run and it lists the folders, but goes haywire when foldes have spaces in their names. Like for example:

Code:

./Musik/Dead found
can found
Dance found
./Musik/Dead found
can found
Dance/Serpent's found
Egg found
./Musik/Dead found
can found
Dance/Within found
the found
Realm found
of found
a found
Dying found
Sun found

is what I get from the folder "Dead can Dance" with the subfolders "Serpent's Egg" and "Within the Realm of a Dying Sun".

How do I fix this? Or is there a way to autorename the folders and convert spaces to _ (underscore)? I know krename can do that with filenames, but not with subfolders, or am I wrong?

acid_kewpie · 02-19-2010, 03:08 PM

ahh pants, yeah, autopilot... meant -type d, sorry.

you should follow the rest of the example though, the while read DIR puts the WHOLE line into the variable, doing it in a for loop there makes each word a seperate value to be iterated over.

frisil · 02-19-2010, 03:29 PM

Okay, that works, I get things like:

./Musik/Dead can Dance found
./Musik/Dead can Dance/Serpent's Egg found

from my test script

now I need to strip the first "./" from my $directory variable so I can use

Code:

cp "$i" "/test/Musik/$directory"

How do I do this? It should be very easy I guess, but I'm not too experienced with bash.

whizje · 02-19-2010, 04:15 PM

much simpler

Code:

cp -R --copy-contents source destination

whizje

frisil · 02-19-2010, 04:24 PM

Quote:

Originally Posted by whizje

much simpler

Code:

cp -R --copy-contents source destination

whizje

If you'd read my initial post carefully, you'd see why that doesn't work: I need the sleep 5 pause after each copy to make sure the filesystem really syncs. Thus, I cannot use any recursive copy option, I need to script my own one with a 5 seconds delay after each copy. Basically, I'm just trying to rebuild cp -R or rsync -r by hand using bash, so I can get the delay in.

whizje · 02-19-2010, 04:33 PM

But to answer your question

Code:

${i#"./"}

This removes ./ from the front of $i and delivers the required $

whizje · 02-19-2010, 05:28 PM

And here a version wich automaticaly creates subdirs and sleeps 5 seconds after every new creation

Code:

find .  \( \( -name '*.mp3' \) -o ( -type d \) \)-exec sh -c 'cp -R --copy-contents "$@" /home/whiz/test2/;sleep 5' sh {} \;

Copies from the current directory to the directory /home/whiz/test2 in this example and copies the directory tree

whizje

frisil · 02-19-2010, 07:16 PM

Wow, that looks advanced! I must honestly say I don't understand it, but I will try it out. Does it pause 5 seconds after each file, too?

ghostdog74 · 02-19-2010, 08:12 PM

Quote:

Originally Posted by frisil

But I want the script to browse subfolders also, something like this (pseudo code

Code:

for dir=all_sub_folders; do
  mkdir /test/Musik/$dir
  for i in *.mp3; do 
    cp $i /test/Musik/$dir; 
    sleep 5; 
  done;
done;

How do I realize that (dir=all_sub_folders) in bash?

if you have Bash 4.0,something like this

Code:

#!/bin/bash
shopt -s globstar
for dir in **/
do
  echo "directory: $dir"
  #mkdir -p <use $dir>
  for file in $dir/*mp3
  do
     echo "file: $file"
     # do your copy here.
  done
done

frisil · 02-20-2010, 12:08 AM

Thank you all for your ideas! Thisi is what my script looks like now (and it works):

Code:

#!/bin/sh
cd $home/4player
find -type d | while read directory
do
  echo "processing: ${directory#"./"}"
  cd "$home/4player/${directory#"./"}"
  mkdir "/test/Musik/${directory#"./"}"
  for i in *.mp3; do
   cp "$i" "/test/Musik/${directory#"./"}"
   sleep 5
  done
done

crude, but working ;-)