Bash script

towed · 05-01-2011, 01:46 PM

Hello!

I am trying to execute a .c program of mine through bash...

Problem is it takes two arguments (files) and am trying to use "for" loops to do it. I use the for loops to take the files (same number in both directories) and put them into arrays. I use the command "ls -1" in the "for loop" so they are sorted in such a way that they are in the same time and date order upon when the program is executed.

Here is the code:

Code:

for a in `ls -1 /analyses/data1/*_prt.txt`;
do  echo $a;
done

for sfc in `ls -1 /analyses/data2/*_ht.txt`;
do 
  ./plot_data $a $b >> log_file.log;
done

HOWEVER, $a in this case does not change with the loop. Thus, the program doesn't work.

So, I then tried:

Code:

for a in `ls -1 /analyses/data1/*_prt.txt`;
for sfc in `ls -1 /analyses/data2/*_ht.txt`;
do 
  ./plot_data $a $b >> log_file.log;
done

But that produces the error: syntax error near unexpected token `for'. As a "do" statement must come after the for loop call.

Any ideas how to do this? Just need to run the program with the two arrays and have them both advance together.

Reuti · 05-01-2011, 01:57 PM

Modify the second case so that also the outer loop uses a do and done.

BTW: `ls ...` isn’t necessary. A plain /analyses/data1/*_prt.txt will also do it.

towed · 05-01-2011, 02:01 PM

Such as:

Code:

for a in `ls -1 /analyses/data1/*_prt.txt`;
do
for sfc in `ls -1 /analyses/data2/*_ht.txt`;
do 
  ./plot_data $a $b >> log_file.log;
done
done

Only problem I worry about is that won't it increase the outer loop and then increase the inner loop unevenly? I mean it would execute it like:
./plot_data $a[0] $b[0] >> log_file.log;
./plot_data $a[0] $b[1] >> log_file.log;
./plot_data $a[0] $b[2] >> log_file.log;
...
and so on?

As I want it to be like:
./plot_data $a[0] $b[0] >> log_file.log;
./plot_data $a[1] $b[1] >> log_file.log;
./plot_data $a[2] $b[2] >> log_file.log;
...
and so on.

That make sense (using [] to represent the elements in the array, starting at 0).

Reuti · 05-01-2011, 02:30 PM

Aha, I thought you want to get all possible combinations. When it should be only one round, you can replace the path in the statement:

Code:

for FIRST in /analyses/data1/*_prt.txt; do
  SECOND=$(basename $FIRST)
  ./plot_data $FIRST /analyses/data2/${SECOND%prt.txt}ht.txt >> log_file.log;
done

basename will strip off the path, then the suffix will be replaces. If the names are too different, but in the same order you can also fill two arrays:

Code:

FIRST=(/analyses/data1/*_prt.txt)
SECOND=(/analyses/data2/*_ht.txt)
for ((i=0;i<${#FIRST[*]};i++)); do
  ./plot_data $FIRST[i] $SECOND[i] >> log_file.log
done

towed · 05-01-2011, 02:33 PM

Quote:

Originally Posted by Reuti

Aha, I thought you want to get all possible combinations. When it should be only one round, you can replace the path in the statement:

Code:

for FIRST in /analyses/data1/*_prt.txt; do
  SECOND=$(basename $FIRST)
  ./plot_data $FIRST /analyses/data2/${SECOND%prt.txt}ht.txt >> log_file.log;
done

basename will strip off the path, then the suffix will be replaces. If the names are too different, but in the same order you can also fill two arrays:

Code:

FIRST=(/analyses/data1/*_prt.txt)
SECOND=(/analyses/data2/*_ht.txt)
for ((i=0;i<${#FIRST[*]};i++)); do
  ./plot_data $FIRST[i] $SECOND[i] >> log_file.log
done

No worries... The files do have different notation, so the second way may be the way to go.. I am going to try that.

I guess though, what does: #FIRST[*] mean? I am guessing the number of elements in the array? total?

Reuti · 05-01-2011, 02:38 PM

Quote:

Originally Posted by towed

I guess though, what does: #FIRST[*] mean? I am guessing the number of elements in the array? total?

Yep. And it starts at 0.

When they have different names: do they have the same order? I mean: you might end up with different and wrong combinations otherwise.

towed · 05-01-2011, 02:40 PM

Quote:

Originally Posted by Reuti

Yep. And it starts at 0.

When they have different names: do they have the same order? I mean: you might end up with different and wrong combinations otherwise.

Yeah, they are in the same order and have the same number of the files... That is why I used the ls -1 to ensure that they would be paired togather... can I use that still in:

blend=(/data3/etownsend/analyses/blend/cases_reran/ppaes_join_200*_prt.txt)

like:
blend=(ls -1 /data3/etownsend/analyses/blend/cases_reran/ppaes_join_200*_prt.txt)

?

towed · 05-01-2011, 02:48 PM

Quote:

Originally Posted by Reuti

If the names are too different, but in the same order you can also fill two arrays:

Code:

FIRST=(/analyses/data1/*_prt.txt)
SECOND=(/analyses/data2/*_ht.txt)
for ((i=0;i<${#FIRST[*]};i++)); do
  ./plot_data $FIRST[i] $SECOND[i] >> log_file.log
done

It seems in the second way that: $SECOND[i] is getting parsed with the [i], which throws off the program... Any idea why? could it be because only FIRST is referenced when the for loop is started?

Actully, both [i] are getting passed in...

Reuti · 05-01-2011, 02:51 PM

You won't need it I think. `ls -1 ...` will list in one column on screen. But inside any assignment, it’s always used in this way by default.

NB: I forgot the curly braces in the call’s arguments: $FIRST[i] => ${FIRST[i]}

David the H. · 05-01-2011, 02:54 PM

Code:

for ((i=0;i<${#FIRST[*]};i++)); do
  ./plot_data ${FIRST[i]} ${SECOND[i]} >> log_file.log
done

There's a more convenient way to do this.

Code:

for i in ${!FIRST[*]}; do
  ./plot_data ${FIRST[i]} ${SECOND[i]} >> log_file.log
done

${!array[*]} and ${!array[@]} expand to a list of the index elements of the array. This makes it a snap to loop through them, and be sure that you only get elements that actually exist.

As for using ls to set an array, you need do this:

Code:

blend=( $(ls -1 /data3/etownsend/analyses/blend/cases_reran/ppaes_join_200*_prt.txt) )

The array is set to one element per "word" that exists inside the parentheses. So you have to use a command substitution to generate that list.

It's still not advisable nor necessary to use ls though. If the files are there, then simple globbing should generate exactly the same list as ls does.

towed · 05-01-2011, 02:54 PM

Quote:

Originally Posted by towed

It seems in the second way that: $SECOND[i] is getting parsed with the [i], which throws off the program... Any idea why? could it be because only FIRST is referenced when the for loop is started?

Actully, both [i] are getting passed in...

I believe they need ${FIRST[i]} to be used this way... Does that sound right? or else [i] gets added to the end...

Thus my script looks like:
[CODE]

FIRST=(/data1/analyses/*_prt.txt)
SECOND=(/data2/analyses/*_ht.txt)

echo ${#FIRST[*]} " elements in the data array" > log.log
echo ${#SECOND[*]} " elements in the data array" >> log.log

for ((i=0;i<${#BLEND[*]};i++)); do
./program ${FIRST[i]} ${SECOND[i]}
echo " " >> blend_images_reran.log
echo "--------------" >> blend_images_reran.log
echo " " >> blend_images_reran.log
done

towed · 05-01-2011, 03:01 PM

Quote:

Originally Posted by David the H.

Code:

for ((i=0;i<${#FIRST[*]};i++)); do
  ./plot_data ${FIRST[i]} ${SECOND[i]} >> log_file.log
done

There's a more convenient way to do this.

Code:

for i in ${!FIRST[*]}; do
  ./plot_data ${FIRST[i]} ${SECOND[i]} >> log_file.log
done

${!array[*]} and ${!array[@]} expand to a list of the index elements of the array. This makes it a snap to loop through them, and be sure that you only get elements that actually exist.

As for using ls to set an array, you need do this:

Code:

blend=( $(ls -1 /data3/etownsend/analyses/blend/cases_reran/ppaes_join_200*_prt.txt) )

The array is set to one element per "word" that exists inside the parentheses. So you have to use a command substitution to generate that list.

It's still not advisable nor necessary to use ls though. If the files are there, then simple globbing should generate exactly the same list as ls does.

Could you elaborate on "expand to a list of the index elements of the array".
Does that mean the for loops knows the filenames in the array?

David the H. · 05-01-2011, 03:13 PM

Why not try echoing it out and see for yourself?

But if you insist:

Code:

$ array=( Fry Bender Zapp Leela )

$ echo ${array[*]}
Fry Bender Zapp Leela

$ echo ${!array[*]}
0 1 2 3

$ unset array[2]

$ echo ${array[*]}
Fry Bender Leela

$ echo ${!array[*]}
0 1 3

towed · 05-01-2011, 03:36 PM

Yeah, I should have X_X... Don't know why I didn't.

Thanks though, props for the futurama reference in the example.