LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Reply
 
Search this Thread
Old 11-04-2009, 11:14 AM   #1
idaham
LQ Newbie
 
Registered: Aug 2009
Posts: 26

Rep: Reputation: 0
Read multiple files line by line


Hi!

I've just started writing a script to read multiple files and save the data in different variables. More precisely, I have two different files containing one column of data each (both files equally many rows):
Code:
file1:    file2:
0         3
1         5
2         20
3         3
.         .
.         .
I want to read line by line in a for or while loop and save the corresponding row value in two different variables (that I will later use as input into another function). It would work something like, in "semi-code" =):
Code:
for i=1 to "number of rows in files"
   variable1 = i:throw in file1
   variable2 = i:th row in file2
   function(variable1, variable2)
end
I've managed to do what I want when just reading one file:
Code:
#!/bin/bash
cat file1.dat | while read line 
do 
   variable1="$line"
   function($variable1)
done
Any ideas how to transform this to reading multiple files at the same time? Thanks!
Ida
 
Old 11-04-2009, 11:45 AM   #2
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
It could be done by setting up the other files on file descriptors (other than 0, 1 and 2 which are already in use) and using them as inputs to reads within the loop reading file1.dat. There are some examples at The Linux Documentation Project. It would require input redirect rather than the cat idiom in your existing code, for example
Code:
read <&3
 
Old 11-05-2009, 01:00 AM   #3
bsat
Member
 
Registered: Feb 2009
Posts: 347

Rep: Reputation: 72
Hi if you are aware of awk, it can be done a little more easily. I am not sure what you want to do in function1, hope this helps.

put this code into a file say "awk1"
FILENAME==file1{
array1[i++] = $1
}
FILENAME==file2{
array2[j++]=$1
}
END{
#Do what ever you want to do with the two sets of arrays
}

run the script using the following way

awk -f awk1 file1 file2
 
Old 11-05-2009, 01:13 AM   #4
evo2
Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,591

Rep: Reputation: 1233Reputation: 1233Reputation: 1233Reputation: 1233Reputation: 1233Reputation: 1233Reputation: 1233Reputation: 1233Reputation: 1233
You can use paste

Code:
#!/bin/bash
paste file1 file2 | (
while read col1 col2 ; do
echo "Data: ${col1}, ${col2}"
done
)
Evo2.
 
Old 11-05-2009, 03:33 AM   #5
idaham
LQ Newbie
 
Registered: Aug 2009
Posts: 26

Original Poster
Rep: Reputation: 0
Thank you all for your great and quick tips!

I decided to follow the exec approach... I've now managed to read my files into arrays:
Code:
i=0
exec 3<"$file"
exec 4<"$file2"
while read value1 <&3
do
	read value2 <&4
	array1[$i]=$value1
	array2[$i]=$value2
	i=$(($i +1))
done
I now want to be able to actually use the values stored in the arrays like:
Code:
for j=0 to size of array-1
    start = array1[j]
    end = array1[j+1]
    some more stuff...
done
but I haven't been able to do this yet. I'm having problems actually retrieving the array data. I've tried with (and failed):
Code:
for ((j=0;j<=$(($i-1));j+=1))
do
    start=array1[$j]
    end=array1[$(($j+1))]
    echo $start - $end
done
Any more suggestions? thanks guys!
Ida
 
Old 11-05-2009, 04:00 AM   #6
Disillusionist
Senior Member
 
Registered: Aug 2004
Location: England
Distribution: Ubuntu
Posts: 1,013

Rep: Reputation: 83
Quote:
Originally Posted by idaham View Post
I've tried with (and failed):
Code:
for ((j=0;j<=$(($i-1));j+=1))
do
    start=array1[$j]
    end=array1[$(($j+1))]
    echo $start - $end
done
Any more suggestions? thanks guys!
Ida
Try:

Code:
for ((j=0;j<=$(($i-1));j+=1))
do
    start=${array1[$j]}
    end=${array1[$j+1]}
    echo $start - $end
done
 
Old 11-05-2009, 04:19 AM   #7
idaham
LQ Newbie
 
Registered: Aug 2009
Posts: 26

Original Poster
Rep: Reputation: 0
Thanks Disillusionist!
Such a simple solution that did just what I wanted! =)
/Ida
Quote:
Originally Posted by Disillusionist View Post
Try:

Code:
for ((j=0;j<=$(($i-1));j+=1))
do
    start=${array1[$j]}
    end=${array1[$j+1]}
    echo $start - $end
done
 
  


Reply

Tags
read, script


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH: read every line in the files and use the line as parameters as another program tam3c36 Programming 10 12-07-2010 02:42 PM
Search values within multiple files line to line Chrizzieej Programming 5 09-26-2008 05:11 PM
help with c program to read each line from text file, split line , process and output gkoumantaris Programming 12 07-01-2008 01:38 PM
php - Read file line by line and change a specific line. anrea Programming 2 01-28-2007 02:43 PM
linux scripting help needed read from file line by line exc commands each line read atokad Programming 4 12-26-2003 11:24 PM


All times are GMT -5. The time now is 05:56 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration