LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (http://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Read multiple files line by line (http://www.linuxquestions.org/questions/linux-newbie-8/read-multiple-files-line-by-line-766832/)

idaham 11-04-2009 10:14 AM

Read multiple files line by line
 
Hi!

I've just started writing a script to read multiple files and save the data in different variables. More precisely, I have two different files containing one column of data each (both files equally many rows):
Code:

file1:    file2:
0        3
1        5
2        20
3        3
.        .
.        .

I want to read line by line in a for or while loop and save the corresponding row value in two different variables (that I will later use as input into another function). It would work something like, in "semi-code" =):
Code:

for i=1 to "number of rows in files"
  variable1 = i:throw in file1
  variable2 = i:th row in file2
  function(variable1, variable2)
end

I've managed to do what I want when just reading one file:
Code:

#!/bin/bash
cat file1.dat | while read line
do
  variable1="$line"
  function($variable1)
done

Any ideas how to transform this to reading multiple files at the same time? Thanks!
Ida

catkin 11-04-2009 10:45 AM

It could be done by setting up the other files on file descriptors (other than 0, 1 and 2 which are already in use) and using them as inputs to reads within the loop reading file1.dat. There are some examples at The Linux Documentation Project. It would require input redirect rather than the cat idiom in your existing code, for example
Code:

read <&3

bsat 11-05-2009 12:00 AM

Hi if you are aware of awk, it can be done a little more easily. I am not sure what you want to do in function1, hope this helps.

put this code into a file say "awk1"
FILENAME==file1{
array1[i++] = $1
}
FILENAME==file2{
array2[j++]=$1
}
END{
#Do what ever you want to do with the two sets of arrays
}

run the script using the following way

awk -f awk1 file1 file2

evo2 11-05-2009 12:13 AM

You can use paste

Code:

#!/bin/bash
paste file1 file2 | (
while read col1 col2 ; do
echo "Data: ${col1}, ${col2}"
done
)

Evo2.

idaham 11-05-2009 02:33 AM

Thank you all for your great and quick tips!

I decided to follow the exec approach... I've now managed to read my files into arrays:
Code:

i=0
exec 3<"$file"
exec 4<"$file2"
while read value1 <&3
do
        read value2 <&4
        array1[$i]=$value1
        array2[$i]=$value2
        i=$(($i +1))
done

I now want to be able to actually use the values stored in the arrays like:
Code:

for j=0 to size of array-1
    start = array1[j]
    end = array1[j+1]
    some more stuff...
done

but I haven't been able to do this yet. I'm having problems actually retrieving the array data. I've tried with (and failed):
Code:

for ((j=0;j<=$(($i-1));j+=1))
do
    start=array1[$j]
    end=array1[$(($j+1))]
    echo $start - $end
done

Any more suggestions? thanks guys!
Ida

Disillusionist 11-05-2009 03:00 AM

Quote:

Originally Posted by idaham (Post 3745377)
I've tried with (and failed):
Code:

for ((j=0;j<=$(($i-1));j+=1))
do
    start=array1[$j]
    end=array1[$(($j+1))]
    echo $start - $end
done

Any more suggestions? thanks guys!
Ida

Try:

Code:

for ((j=0;j<=$(($i-1));j+=1))
do
    start=${array1[$j]}
    end=${array1[$j+1]}
    echo $start - $end
done


idaham 11-05-2009 03:19 AM

Thanks Disillusionist!
Such a simple solution that did just what I wanted! =)
/Ida
Quote:

Originally Posted by Disillusionist (Post 3745402)
Try:

Code:

for ((j=0;j<=$(($i-1));j+=1))
do
    start=${array1[$j]}
    end=${array1[$j+1]}
    echo $start - $end
done




All times are GMT -5. The time now is 03:49 PM.