LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-24-2007, 08:43 AM   #1
johnpaulodonnell
Member
 
Registered: Jun 2006
Location: Galway
Distribution: Suse 10.1
Posts: 114

Rep: Reputation: 15
Arrow how to loop over text file lines within bash script for loop?


Hi.

I have a data file containing 4 rows of data say:

X1 Y1 Z1
X2 Y2 Z2
X3 Y3 Z3
X4 Y4 Z4

Associated with each individual row is a data file containing a subset of additional data for that row. These associated data files are single column files, but not all have the same number of columns. For example that data file associated with row 1 might be of form:

R1a
R1b
R1c
R1d

Is there a way that I can pass the data file above (R1a, etc) to a for loop within a bash script such that it loops over the lines of the data file and outputs the following:

X1 Y1 Z1 R1a
X1 Y1 Z1 R1b
X1 Y1 Z1 R1c
X1 Y1 Z1 R1d

- and so on for each record of the main data file.

I've seen examples with individual files of directories being passed into for loops, as in:

for i in *
do
........
done


But what is the equivalent code when you want to loop over text file lines?

Thanks.
 
Old 01-24-2007, 09:22 AM   #2
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
There are many possible solutions. Here's mine, using gawk inside a while loop:

Code:
#!/bin/sh

i=1

while [ $i -le `wc -l filename.txt | gawk '{print $1}'` ] ; do

    line=`head -$i filename.txt | tail -1`
    
    gawk -v prefix="$line" '{print prefix,$0}' $i.txt >> output.txt

    i=`expr $i + 1`
    
done
where filename.txt is the datafile and I assume that each "single column" file is called 1.txt (associated with first row) 2.txt (associated with second row) and so on. If the real "single column" files have progressive numbers in their name, you can easily build the name inside the loop. Otherwise you can do a list of these files and parse this list with the head/tail construct above.

Hope this will help.
 
Old 04-30-2009, 07:28 PM   #3
scriptjunkie1
LQ Newbie
 
Registered: Apr 2009
Posts: 1

Rep: Reputation: 0
Wink Much easier

Set the internal file separator:

Code:
IFS=$'\n'; for line in `ls`; do echo $line; done
(this just gets each line for you easier than head/tail. the rest is the same)

Last edited by scriptjunkie1; 04-30-2009 at 07:30 PM.
 
Old 04-30-2009, 07:50 PM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
this is an old thread.
first, no need to change IFS and using ls is "useless" .
Code:
for line in *
secondly, OP wants to loop over the lines in the files (based on his post's last sentence), not filenames.
lastly, the tool to use could be simple paste/join.
 
1 members found this post helpful.
Old 05-01-2009, 06:11 AM   #5
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301
Why not just use paste ?

Code:
paste file1 file2 > file 3
 
1 members found this post helpful.
Old 05-01-2009, 08:27 AM   #6
PMP
Member
 
Registered: Apr 2009
Location: ~
Distribution: RHEL, Fedora
Posts: 381

Rep: Reputation: 58
Try Something like this

exec 3>&0
exec 0<$filename
while read eachline
do
echo $eachline
done
exec 0>&3

Cheers
 
Old 03-20-2011, 03:48 PM   #7
downeyt
LQ Newbie
 
Registered: Feb 2011
Posts: 1

Rep: Reputation: 0
Use cat

Better late than never

I was looking for the same answer and saw this post.
Eventually, I came up with this:

for line in `cat fileName.txt`
do
---
done
 
Old 03-20-2011, 07:38 PM   #8
akamikeym
Member
 
Registered: May 2008
Posts: 112

Rep: Reputation: 21
My favorite:

Code:
while read line; do
    echo "$line"
done < "$filename"
 
Old 03-20-2011, 07:38 PM   #9
akamikeym
Member
 
Registered: May 2008
Posts: 112

Rep: Reputation: 21
My favorite:

Code:
while read line; do
    echo "$line"
done < "$filename"
 
1 members found this post helpful.
Old 04-05-2011, 09:18 AM   #10
mrm5102
Member
 
Registered: Apr 2011
Location: Philadelphia
Posts: 165

Rep: Reputation: 3
Nice work
 
Old 07-28-2015, 03:49 PM   #11
rhubarbdog
Member
 
Registered: Apr 2015
Location: Yorkshire, England
Distribution: Linux Mint
Posts: 145

Rep: Reputation: Disabled
Quote:
Originally Posted by mrm5102 View Post
Nice work
Whos done this nice work the OP doesn't ask for many derivatives of read every line in a group of files first line of all files, second line of all files, third line etc.
All the last few posts have done is create sone noise vaugely similar to whats required. This is possibly worse than going totally off post.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Loop through lines in file to find specified substring dresch Linux - Newbie 16 01-06-2007 05:34 PM
bash script loop bong.mau Programming 6 09-14-2005 07:38 PM
bash script for loop drisay Programming 5 12-25-2004 12:32 AM
Python - Set vars and loop over lines in file jnoller Programming 6 02-07-2004 10:32 AM
Loop over lines in text file? amaze Linux - General 2 08-12-2003 07:15 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:02 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration