LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-24-2014, 02:53 PM   #1
TalkingRice
LQ Newbie
 
Registered: Jun 2014
Posts: 10

Rep: Reputation: Disabled
How to run csh script on files in different directories


Hi everyone, I'm a beginner of Linux. I have 20 cases (each in one folder). And I hope to run a script on a specific file in each folder. How can I do it only one time instead of doing it one by one in each folder? What code should I add into the script to make it? Thanks!
 
Old 06-24-2014, 04:20 PM   #2
scottstensland
LQ Newbie
 
Registered: Feb 2010
Posts: 8

Rep: Reputation: 2
One way would be to do a recursive directory listing to parent dir then grep for some string hopefully in common to the filenames of your 20 case files then feed this listing into your shell script:

Say you have this directory structure :

some/path/parent_dir_blah/child_dirname1/case1
some/path/parent_dir_blah/child_dirname2/case2
some/path/parent_dir_blah/child_dirname3/case3

find /some/path/to/parent_dir/of/case/dirs | grep some_string_in_common_to_all_case_filenames

... as in

find some/path/parent_dir_blah | grep case

The find command is very powerful, yet in simplest case just give it the path to parent dir of dirs where your case files live.

Output of above find command piped into a grep on common string will return a listing of case files.
Here is a concrete example - do something with every file containing string : py in python dir :


for curr_casefile in `find ../python/ | grep py `; do

echo curr_casefile $curr_casefile

echo "____ now do something with this file _____"

done
 
Old 06-24-2014, 06:54 PM   #3
TalkingRice
LQ Newbie
 
Registered: Jun 2014
Posts: 10

Original Poster
Rep: Reputation: Disabled
Hi scottstensland, thanks for the help. I tried to put the adjusted lines in your example in my script but it doesn't work well. I put the original commands in the script in the quote after the second echo. But when I run the script in terminal, it returns with
for: Command not found.
curr_casefile: Command not found.
do: Command not found.
curr_casefile: Undefined variable.
I probably did something wrong about it. But I can't tell what is wrong.
 
Old 06-25-2014, 06:04 AM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
Start by posting your entire script in code tags https://www.linuxquestions.org/quest...do=bbcode#code

Also post a few examples of the filepaths and filenames you are dealing with.
 
Old 06-25-2014, 11:17 AM   #5
TalkingRice
LQ Newbie
 
Registered: Jun 2014
Posts: 10

Original Poster
Rep: Reputation: Disabled
Hi~ The code I hope to run for each cases is

Code:
3dTagalign -master ~/Documents/Script_test/master_anat+orig. -affine -nokeeptags -matvec matvec-affine -prefix anat_align -affine T2+orig
3dWarp -matvec_in2out matvec-affine -prefix rest_align-affine EPISE+orig
There are two files (T2+orig and EPISE+orig) in each case folder involved in this code.

I use
Code:
find | grep T2 -o EPISE
But it said "grep: EPISE: No such file or directory"

And also, I tried to only run the first code on T2+orig following the example scottstensland gave. But it said:
for: Command not found.
curr_casefile: Command not found.
do: Command not found.
curr_casefile: Undefined variable.
Is it because the for loop format is incorrect?

Thanks!
 
Old 06-25-2014, 01:18 PM   #6
scottstensland
LQ Newbie
 
Registered: Feb 2010
Posts: 8

Rep: Reputation: 2
Sounds like your script is not getting recognized as a shell script when you say :

"for: Command not found"

In your terminal window type this :

echo $SHELL

it should return something like this :

/bin/bash

which gives full path to reach your bash executable.
Whatever it does return put it at top line of your shell script like this :

#!/usr/bin/env /bin/bash

Now when you execute your shell script the system knows its a shell script.
Its good to give your shell script a name like :

something_meaningful.sh

the important part is it ends with a .sh
To make the script executable do this :

chmod +x something_meaningful.sh

but if you are already able to execute the script you probably already did that.
Now you should NOT get that error from before :

"for: Command not found"

Important - your code says this :

find | grep T2 -o EPISE

I guess you want to grep for files which contain either T2 or EPISE ???
If so do this

find /some/parent/dir | egrep '(T2|EPISE)'

above is doing two steps separated by the pipe which is the symbol : |
The find command can be execute manually :

find /some/parent/dir

type that in to confirm it returns the files you are interested in.
The pipe command just take the output of that command and makes it the
input into the following command.

egrep '(T2|EPISE)'

this syntax of egrep is an either or on filtering for T2 or EPISE, notice the | which is an OR in this case
and NOT a pipe which may be disconcerting at first.

Let us know how you get on with these tips

Last edited by scottstensland; 06-25-2014 at 01:19 PM. Reason: typo
 
Old 06-25-2014, 03:58 PM   #7
TalkingRice
LQ Newbie
 
Registered: Jun 2014
Posts: 10

Original Poster
Rep: Reputation: Disabled
Thanks, scottstensland, the find command works well. But when I type chmod +x script_name.sh, it returns nothing (no response). The editted code is
Code:
#!/bin/bash

for curr_casefile in '(find | egrep '(anat1|rest1)')'; do
echo curr_casefile $curr_casefile
echo 
"rm matvec-affine
3dTagalign -master ~/Documents/Script_test/master_anat+orig. -affine -nokeeptags -matvec matvec-affine -prefix anat2_align -affine anat1+orig
3dWarp -matvec_in2out matvec-affine -prefix rest2_align-affine rest1+orig"

done
 
Old 06-26-2014, 11:50 AM   #8
scottstensland
LQ Newbie
 
Registered: Feb 2010
Posts: 8

Rep: Reputation: 2
OK cool progress

As far as possible unix is silent when things work OK so its normal for the chmod command to just return.
When you do a dir listing of your shell program file before and after issuing the chmod cmd you may have
noticed its displayed with a different color to indicate its executible bit has been turned on.

An important concept is the notion of processes and their children subprocesses ...
unlike most modern languages, shell makes explicit this distinction.
do a google on : difference between sourcing versus executing
In your script the for command is being fed the output of its child process in the
way it executes find. When you want the output of the child process to be feed back into
the parent process you backtick the child process :


wrong way :

for curr_casefile in '(find | egrep '(anat1|rest1)')'; do
echo curr_casefile $curr_casefile
done


correct way :

for curr_casefile in `find | egrep '(anat1|rest1)' ` ; do
echo curr_casefile $curr_casefile
done


surround child processes with a pair of backticks, not single quotes,
backtick key typicaly is found near the top left just under escape key

Once you have this running OK then put inside the for loop whatever commands you want,
each on a separate line
When shell defines a variable like curr_casefile there is no leading $ sign, yet when
used it wants a $ or more flexibly a ${curr_casefile} which lets you construct strings from variables.
 
1 members found this post helpful.
Old 07-02-2014, 02:09 PM   #9
TalkingRice
LQ Newbie
 
Registered: Jun 2014
Posts: 10

Original Poster
Rep: Reputation: Disabled
Hi scottstensland, it seems work to some extend. To make the case easier, I changed to find only one variable first. The script turns out to be:
Code:
#!/bin/bash

for curr_casefile in `find | grep anat1`; do
echo curr_casefile $curr_casefile
echo 
"3dTagalign -master ~/Documents//Script_test/master_anat+orig. -affine -nokeeptags -matvec matvec-affine -prefix anat2_align -affine $curr_casefile"
done
However, it said:
"curr_casefile ./case1/anat1+orig.BRIK
Prel_align2anat_2.sh: line 6: 3dTagalign -master ~/Documents//Script_test/master_anat+orig. -affine -nokeeptags -matvec matvec-affine -prefix anat2_align -affine ./case1/anat1+orig.BRIK: No such file or directory"

I checked all the directory and file names. They were correct. So where is the problem?
 
Old 07-03-2014, 11:45 AM   #10
scottstensland
LQ Newbie
 
Registered: Feb 2010
Posts: 8

Rep: Reputation: 2
shell programming has been around MANY years and has limitations, especially around giving easy to understand error messages. Modern languages like python have supplated it for short and sweet scripts like your current project. Soldiering on using shell, your message says :

Prel_align2anat_2.sh: line 6: 3dTagalign -master ~/Documents//Script_test/master_anat+orig. -affine -nokeeptags -matvec matvec-affine -prefix anat2_align -affine ./case1/anat1+orig.BRIK: No such file or directory"

so evidently your above script is called : Prel_align2anat_2.sh

Assure the last echo command is on same line as the following single quoted text.

Since it could not find command : 3dTagalign
evidently the PATH where 3dTagalign lives is not in your environment variable called PATH
in your terminal window (console) type in this :

which 3dTagalign

also enter this :

type 3dTagalign

if it finds it it shows the full path to reach 3dTagalign
If it fails to find it it will say :


type 3dTagalign
bash: type: 3dTagalign: not found


You need to discover where in your filesystem 3dTagalign lives, lets say it lives at :

/usr/local/some/dir/3dTagalign

so then update your script using the full path as in :


#!/bin/bash

for curr_casefile in `find | grep anat1`; do

echo curr_casefile $curr_casefile

echo "/usr/local/some/dir/3dTagalign put this all on same line $curr_casefile"

/usr/local/some/dir/3dTagalign --help

done
 
Old 07-03-2014, 03:21 PM   #11
TalkingRice
LQ Newbie
 
Registered: Jun 2014
Posts: 10

Original Poster
Rep: Reputation: Disabled
I found the location of 3dTagalign and updated the script but not sure if in the right way. The new script is
Code:
#!/bin/bash

for curr_casefile in `find | grep anat1`; do
echo curr_casefile $curr_casefile
echo 
"/usr/local/mrrc/afni/3dTagalign 3dTagalign -master ~/Documents/Script_test/master_anat+orig. -affine -nokeeptags -matvec matvec-affine -prefix anat2_align -affine $curr_casefile"
/usr/local/mrrc/afni/3dTagalign --help

done
But the out put in terminal window is:
"curr_casefile ./case1/anat1+orig.HEAD
/usr/local/mrrc/afni/3dTagalign 3dTagalign -master ~/Documents/Script_test/master_anat+orig. -affine -nokeeptags -matvec matvec-affine -prefix anat2_align -affine ./case1/anat1+orig.HEAD No such file or directory
** Unknown option: --help"
And no output files in the folder where it used to be.
I also tried to remove the second 3dTagalign in the script but still got the same result.
 
Old 07-04-2014, 10:49 AM   #12
scottstensland
LQ Newbie
 
Registered: Feb 2010
Posts: 8

Rep: Reputation: 2
the command : echo
simply displays, it does not execute what comes after.

echo "this will just print out the quoted text with embedded variables like $curr_casefile"

Try it without echo and without quotes :

#!/bin/bash

for curr_casefile in `find | grep anat1`; do

echo curr_casefile $curr_casefile

/usr/local/mrrc/afni/3dTagalign -master ~/Documents/Script_test/master_anat+orig. -affine -nokeeptags -matvec matvec-affine -prefix anat2_align -affine $curr_casefile

done

NOTE - the above line containing your executable 3dTagalign must be all on same line - although the editor may wrap it for display only that's OK
 
1 members found this post helpful.
Old 07-08-2014, 01:42 PM   #13
TalkingRice
LQ Newbie
 
Registered: Jun 2014
Posts: 10

Original Poster
Rep: Reputation: Disabled
Hi scottstensland,

I think it works now. The 3dTagalign command will automatically save the output file into the master folder. So there is an error message about conflicts with existing file. I will figure out how to change the save directory. But the script itself should be working. Thanks a lot!!

Shiyu
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to modify files in sub-directories - Need help GottaGoFast Programming 1 12-29-2012 05:05 PM
Shell script to move through directories and run various commands. The_Seraphim Programming 1 10-15-2009 03:13 PM
Creating a script to move or copy files into multiple directories below the files matthes138 Linux - Newbie 5 08-25-2009 05:57 PM
how to add script files to /etc/csh.login Enigmatic_Desires Linux - Newbie 2 06-22-2005 07:45 AM
Script to run script in multiple directories... seabass55 Linux - General 5 07-06-2004 01:44 AM


All times are GMT -5. The time now is 12:44 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration