How to run a command in many subdirectories?

mcgao07 · 01-27-2009, 10:07 PM

I have a quick question that seems easy to answer, but I have not figured it out...

I have a directory that contains many subdirectories: such as
p+0.1_8.6103_0
p+0.1_8.6103_10
p+0.1_8.6103_12
p-0.1_8.6103_15
p-0.1_8.6103_2
p-0.1_8.6103_4
p-0.1_8.6103_6
...

Now I want to do "qsub submission" ["submission" is a job submission script] in each subdirectory in bash. How should I do it? It must be related to "foreach" command, but I am in a big hurry...

Thanks a lot...

Michael

Tinkster · 01-27-2009, 10:19 PM

I don't know that the "queue job submission command" is, but
try something like
find /path/to/subdirs -execdir command ;

man find
/execdir

for details

crabboy · 01-27-2009, 10:27 PM

Code:

#!/bin/sh

CWD=`pwd`
for i in `find . -type d -maxdepth 1`; do
   cd $i
   # do something here
   cd $CWD
done

mcgao07 · 01-27-2009, 11:19 PM

Quote:

Originally Posted by crabboy

Code:

#!/bin/sh

CWD=`pwd`
for i in `find . -type d -maxdepth 1`; do
   cd $i
   # do something here
   cd $CWD
done

Crabboy,

Thank you, but there are problems.

1) there is error message:
find: warning: you have specified the -maxdepth option after a non-option argument -type, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it). Please specify options before other arguments.

2) the script always find the parent directory. I want to exclude it from the list. How to do it?

Thank you.

Michael

crabboy · 01-28-2009, 08:19 AM

1. Try switching the options. I don't see the message with my find, but the options work either way.

2. There is always the option of changing the . in find to be the full path to the parent directory. find will then produce a full path to the subdirectories, but you will still need supply the parent, as it is the reference to find the children.

Code:

#!/bin/sh

for i in `find /var/www -type d -maxdepth 1`; do
   cd $i
   # do something here
done

mcgao07 · 01-28-2009, 11:50 AM

Quote:

Originally Posted by crabboy

1. Try switching the options. I don't see the message with my find, but the options work either way.

2. There is always the option of changing the . in find to be the full path to the parent directory. find will then produce a full path to the subdirectories, but you will still need supply the parent, as it is the reference to find the children.

Code:

#!/bin/sh

for i in `find /var/www -type d -maxdepth 1`; do
   cd $i
   # do something here
done

Hi,

Even I specified the path, it always find the parent directory first. Consequently, it also submits a job at the parent directory, which is not what I want.

I found the following works in my case since all my subdirectories start with "p", but not the most desirable:

#!/bin/bash

for i in `find . -name "p*"`; do
cd $i
qsub /nfs/home/mgao/script/atat_runstruct_vasp8
cd ../
done

crabboy · 01-28-2009, 12:02 PM

I see now.

You can put this right after the for.

Code:

   if [ `pwd` = $i ]; then
      continue
   fi

Tinkster · 01-28-2009, 10:40 PM

Hmmm ... not that crabboys solution is bad, but what's wrong
with

Code:

find -type d -mindepth 1 -maxdepth 1 -execdir qsub /nfs/home/mgao/script/atat_runstruct_vasp8 {} +

mcgao07 · 01-28-2009, 11:12 PM

Quote:

Originally Posted by crabboy

I see now.

You can put this right after the for.

Code:

   if [ `pwd` = $i ]; then
      continue
   fi

Well, it did not work for some reason.

Michael

mcgao07 · 01-28-2009, 11:15 PM

Quote:

Originally Posted by Tinkster

Hmmm ... not that crabboys solution is bad, but what's wrong
with

Code:

find -type d -mindepth 1 -maxdepth 1 -execdir qsub /nfs/home/mgao/script/atat_runstruct_vasp8 {} +

Well, it did not find the subdirectory, instead, it submitted the job n times (n=total # of subdirectories) in the parent directory.

crabboy · 01-29-2009, 08:03 AM

The find on my slack machine has no execdir

try adding the mindepth as tink suggested:

Code:

#!/bin/sh

for i in `find /var/www -type d -maxdepth 1 -mindepth 1`; do
   cd $i
   # do something here
done

schneidz · 01-29-2009, 10:23 AM

how about this:

Code:

for dir in *
do
 cd $dir
 qsub submission # no idea what this is ?
 cd ..
done

should this be in the programming forum ?

crabboy · 01-29-2009, 07:54 PM

schneidz, you script may not work all that well. If there are files in the directory, the 'for dir in *' will pick up the files as well as directories. The cd $dir will fail, on the files, and the script will cd .. regardless. This would put you up one directory and then the you'd attempt another cd $dir which would likely fail.

schneidz · 01-29-2009, 11:21 PM

^good feedback; i should've put a disclaimer that the above code was quick-and-dirty. the original poster said they needed an answer in a hurry so i didnt do any testing on it.

the original poster should tweak it to their hearts content anyways...

one could handle each exception in an if or test but i am too lazy rite now.

mcgao07 · 01-30-2009, 06:58 AM

Quote:

Originally Posted by crabboy

The find on my slack machine has no execdir

try adding the mindepth as tink suggested:

Code:

#!/bin/sh

for i in `find /var/www -type d -maxdepth 1 -mindepth 1`; do
   cd $i
   # do something here
done

Yes, this works now. Thank you all for quick rescue.

Michael