-   Linux - Newbie (
-   -   how to loop over text file lines within bash script for loop? (

johnpaulodonnell 01-24-2007 08:43 AM

how to loop over text file lines within bash script for loop?

I have a data file containing 4 rows of data say:

X1 Y1 Z1
X2 Y2 Z2
X3 Y3 Z3
X4 Y4 Z4

Associated with each individual row is a data file containing a subset of additional data for that row. These associated data files are single column files, but not all have the same number of columns. For example that data file associated with row 1 might be of form:


Is there a way that I can pass the data file above (R1a, etc) to a for loop within a bash script such that it loops over the lines of the data file and outputs the following:

X1 Y1 Z1 R1a
X1 Y1 Z1 R1b
X1 Y1 Z1 R1c
X1 Y1 Z1 R1d

- and so on for each record of the main data file.

I've seen examples with individual files of directories being passed into for loops, as in:

for i in *

But what is the equivalent code when you want to loop over text file lines?


colucix 01-24-2007 09:22 AM

There are many possible solutions. Here's mine, using gawk inside a while loop:




while [ $i -le `wc -l filename.txt | gawk '{print $1}'` ] ; do

    line=`head -$i filename.txt | tail -1`
    gawk -v prefix="$line" '{print prefix,$0}' $i.txt >> output.txt

    i=`expr $i + 1`

where filename.txt is the datafile and I assume that each "single column" file is called 1.txt (associated with first row) 2.txt (associated with second row) and so on. If the real "single column" files have progressive numbers in their name, you can easily build the name inside the loop. Otherwise you can do a list of these files and parse this list with the head/tail construct above.

Hope this will help.

scriptjunkie1 04-30-2009 07:28 PM

Much easier
Set the internal file separator:


IFS=$'\n'; for line in `ls`; do echo $line; done
(this just gets each line for you easier than head/tail. the rest is the same)

ghostdog74 04-30-2009 07:50 PM

this is an old thread.
first, no need to change IFS and using ls is "useless" .

for line in *
secondly, OP wants to loop over the lines in the files (based on his post's last sentence), not filenames.
lastly, the tool to use could be simple paste/join.

H_TeXMeX_H 05-01-2009 06:11 AM

Why not just use paste ?


paste file1 file2 > file 3

PMP 05-01-2009 08:27 AM

Try Something like this

exec 3>&0
exec 0<$filename
while read eachline
echo $eachline
exec 0>&3


downeyt 03-20-2011 03:48 PM

Use cat
Better late than never :)

I was looking for the same answer and saw this post.
Eventually, I came up with this:

for line in `cat fileName.txt`

akamikeym 03-20-2011 07:38 PM

My favorite:


while read line; do
    echo "$line"
done < "$filename"

mrm5102 04-05-2011 09:18 AM

Nice work

All times are GMT -5. The time now is 05:23 PM.