LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-20-2011, 03:56 PM   #1
pathfinder57
LQ Newbie
 
Registered: Apr 2011
Posts: 4

Rep: Reputation: 0
for loop problem


I am having trouble with a "do for" loop that is supposed to read a file line by line and then give structured output. When I run it I get multiple copies of the output rather than one. Any help would be appreciated.

My script
for i in `cat "$filename"`
do
awk -F: '{print $1, $2, $3}' "$filename"
done

Like I said it works but gives me multiple copies of the output.

I actually set IFS=":" just before this as a colon is the field separator in the files I need to process. My question is the "for i in" correct to read the file one line at a time

Last edited by pathfinder57; 04-20-2011 at 04:26 PM. Reason: updated
 
Old 04-20-2011, 04:18 PM   #2
arizonagroovejet
Senior Member
 
Registered: Jun 2005
Location: England
Distribution: openSUSE, Fedora, CentOS
Posts: 1,093

Rep: Reputation: 197Reputation: 197
When posting code examples, wrap them in the CODE tag.

Your code contains at least two errors. There's no closing quote that matches the ' before cat. There's no { to match the } in the awk command.

Did you copy/paste this code or re-type it by hand? (Or to put it another way, is this actually the exact code you're using?)

What you've posted will not read one line at a time if there is any whitespace in any of the lines in the file. E.g. this

Code:
one
two three
would get read as three separate parts.

Can you post a sample of the input file and the output you're getting.
 
Old 04-20-2011, 04:27 PM   #3
pathfinder57
LQ Newbie
 
Registered: Apr 2011
Posts: 4

Original Poster
Rep: Reputation: 0
Please ss the updated original post. I only posted the part dealing with the for loop. When I sad multiple lines I mean I get all the files line multiple times. For instance the first file has 15 lines and I get each 4 times

Last edited by pathfinder57; 04-20-2011 at 04:29 PM. Reason: more info
 
Old 04-20-2011, 04:37 PM   #4
arizonagroovejet
Senior Member
 
Registered: Jun 2005
Location: England
Distribution: openSUSE, Fedora, CentOS
Posts: 1,093

Rep: Reputation: 197Reputation: 197
As I said, the code you've posted it has obvious errors in it. If that is the actual code you're using, fix the errors I mentioned and try again. If that is not the code you're using then post the code you are using.

Also post examples of the input you're using and the output you're getting.

It might also help if you posted an example of the output you are trying to get.


You're setting i to each 'line' but you're not using the $i variable inside the loop. So if you're trying to run an operation of each 'line', you're not doing that.
 
Old 04-20-2011, 04:54 PM   #5
konstan
LQ Newbie
 
Registered: Feb 2011
Posts: 4

Rep: Reputation: 0
You are invoking awk as many times as you have words in the file (for i in `cat $filename`;...). And then each time awk traverses the file and prints out your first three fields of each line of the file. Here is the simple example to validate this.

Code:
$ cat $filename 
1 2 a
3 4 b
5 6 c
$ CATED=`cat $filename`
$ echo $CATED 
1 2 a 3 4 b 5 6 c
$
~ konstan$ n=0;for i in $CATED;do echo == $i ==;n=$(($n+1));awk '{print '$n', "-", $1, $2, $3}' $filename;done
== 1 ==
1 - 1 2 a
1 - 3 4 b
1 - 5 6 c
...
== c ==
9 - 1 2 a
9 - 3 4 b
9 - 5 6 c
~ konstan$
In this particular case 27 lines.

Code:
$ wc -lw $filename | awk '{print $1 * $2}'
27
$ n=0;for i in $CATED;do echo == $i ==;n=$(($n+1));awk '{print '$n', "-", $1, $2, $3}' $filename;done|grep -v ^=|wc -l
      27
$
 
Old 04-20-2011, 07:38 PM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,505

Rep: Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890
Ultimately the question boils down to what is it you are trying to do?

Either bash OR awk can do what you seem to be trying on their own so currently the script
seems to not make any sense.

konstan has provided the why as far as what you are seeing occur, but to help with the how
of what you want to do, you will need to explain that first.
 
Old 04-20-2011, 07:54 PM   #7
bash-o-logist
LQ Newbie
 
Registered: Apr 2011
Posts: 2

Rep: Reputation: 0
Code:
while IFS=":" read -r a b c d
do
  echo "$a $b $c"
done < "filename"
 
Old 04-20-2011, 10:20 PM   #8
pathfinder57
LQ Newbie
 
Registered: Apr 2011
Posts: 4

Original Poster
Rep: Reputation: 0
I guess the question is

I guess my question is if the "i" reads the number of words what do I use to use the number of lines in the file. Like I originaly said the script works as written except it executes to many times. I understand this probably isn't the "advanced" way to do this, that is why I posted it in the NEWBIE section.

What I am trying to do is exacly as I originally stated, out put the columns from the file as it reads them. I entered the script here by hand as every time I try to copy and paste it I end up with a copy in the script and not here.
 
Old 04-20-2011, 10:40 PM   #9
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,362

Rep: Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377
I think it would be simpler if you showed us a few lines of the input file, and the expected output for those lines (all in code tags of course).
 
Old 04-20-2011, 10:49 PM   #10
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,505

Rep: Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890Reputation: 2890
Quote:
out put the columns from the file as it reads them.
Yes but in what format?
Quote:
I understand this probably isn't the "advanced" way to do this, that is why I posted it in the NEWBIE section.
None of the solutions being presented are particularly advanced, just there are plenty of alternatives.
Quote:
I guess my question is if the "i" reads the number of words what do I use to use the number of lines in the file
The confusion here is that 'i' is not reading any numbers at all, either words or lines.

What is happening is that 'i' is assigned a value from the file based on the IFS variable value for word splitting (normally white space).
As others have indicated, had you included an example of the input it would help to visualize what you are trying to do.
You are setting the IFS value to a colon so we are guessing this means the file is delimited as such.
So if we assume the following input file format:
Code:
a:b:c
x:y:z
'i' will be assigned each letter one at a time and whatever is inside the for loop will be executed for each, in this case 6 loops will be performed.

As your example then uses awk to read from the same file, it will now read the entire file and display as you have requested.
It will perform this task 6 times, ie. the entire file processed by awk 6 times.

So this leads me back to my response in post #6, either use bash or use awk.

Examples:
Code:
#awk (interestingly you already have your solution)
awk -F: '{print $1, $2, $3}' "$filename"
# assuming only 3 columns in the file, the advanced option here would be
awk -F: '$1=$1' "$filename"

#bash (there are while examples from others, so i will stick with your for loop)
for line in $(< "$filename")
do
    echo "${line//:/ }"
done
# or without the loop altogether
cat "$filename" | tr ':' ' '
 
Old 04-21-2011, 08:45 AM   #11
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
Use $(command) instead of backticks (`command`).

It's easier to read, connot be confused with quotes, and nests easily.
 
Old 04-21-2011, 09:31 PM   #12
pathfinder57
LQ Newbie
 
Registered: Apr 2011
Posts: 4

Original Poster
Rep: Reputation: 0
Thanks for all the help!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to loop over text file lines within bash script for loop? johnpaulodonnell Linux - Newbie 9 07-28-2015 03:49 PM
for loop or while loop to read the fields of a file.. visitnag Linux - Newbie 10 09-02-2010 08:47 PM
bash loop within a loop for mysql ops br8kwall Programming 10 04-30-2008 03:50 AM
Problem with C loop ! linuxlover1 Programming 3 11-02-2006 03:32 PM


All times are GMT -5. The time now is 01:55 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration