[SOLVED] General: selecting input files and applying a command automatically

Trd300 · 03-12-2012, 02:26 AM

Hi !

A very general question.
How could we select 2 input files based on their names and apply any command to them automatically?

I have a directory containing 2,000 files:

Code:

file1_red.tab
file1_red_info.temp
file2_blue.tab
file2_blue_info.temp
file3_green.tab
file3_green_info.temp
...
file1000_black.tab
file1000_black_info.temp

I want to apply automatically a command (let's say the "join" command) to specific pairs of files, both containing the same "number" and the same "color".
Just to avoid writing 1,000 times the same command with different files,

Code:

join file1_red.tab file1_red_info.temp
join file2_blue.tab file2_blue_info.temp
join file3_green.tab file3_green_info.temp
...
join file1000_black.tab file1000_black_info.temp

is there any way to use the names of files to run this join (or another) command?
Maybe using a conditional expression (in a for loop) in this spirit:

Code:

# if ( 2 file names contain the same "number" and "color")
    then (apply join command to them)

I know awk has a built-in variable "FILENAME", but it would mean that all the command we want to apply to these files must be written with awk, which is sometimes complicated .

grail · 03-12-2012, 04:06 AM

Do we know the pattern is always the same?
Example:

Code:

file1_red.tab
file1_red_info.temp

If the 2 pieces in red are always the same (obviously colour and number changing for new pairs), then simply use something like parameter substitution:

Code:

FILE=file1_red.tab

join $FILE ${FILE%.tab}_info.temp

Simple for loop can feed the variable and you may wish to test that the other file exists first.

Trd300 · 03-12-2012, 04:48 AM

When I write:

Code:

for FILE=file*_*.tab
do
    join $FILE ${FILE%.tab}_info.temp
done

or:

Code:

for FILE=file*_*.tab
do
    join ${FILE} ${FILE%.tab}_info.temp
done

in test.sh, and run the file:

Code:

sh test.sh

It returns for both scenario:

Code:

test.sh: line 4: `FILE=f*_*.tab': not a valid identifier

catkin · 03-12-2012, 04:55 AM

Try for FILE in file*_*.tab

It would be prudent to change the command to echo join $FILE ${FILE%.tab}_info.temp so you can see what it is going to do.

The { } in ${FILE} is not wrong but neither is it necessary.

grail · 03-12-2012, 05:11 AM

I think you need to read up a bit on things as your attempt was a guess.

Here are some links to help you:

http://tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/TitleIndex

Trd300 · 03-12-2012, 05:18 AM

When I try:

Code:

for FILE in file*_*.tab
do
join $FILE ${FILE%.tab}_info.temp > ${FILE%.tab}_final.tab
done

only with 4 files:

Code:

file1_red.tab
file1_red_info.temp
file2_blue.tab
file2_blue_info.temp

I get,
file1_red_final.tab which is correct,

but file2_blue_final.tab contains (with or without "echo"):

Code:

join $FILE ${FILE%.tab}_info.temp > ${FILE%.tab}_final.tab

Trd300 · 03-12-2012, 05:32 AM

Found it !

Code:

for i in file*_*.tab
do
    j=`basename ${i} .tab`
    join ${i} ${j}_info.temp > ${j}_final.tab
done

grail · 03-12-2012, 08:22 AM

Well, glad you found your answer although I see no difference between the 2 scripts and they both work as expected for me.