LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-19-2012, 04:24 PM   #1
vril
LQ Newbie
 
Registered: Apr 2012
Posts: 5

Rep: Reputation: Disabled
for...in


Let's say I need to loop through all lines of a string in bash to do the following:

Code:
for line in "$string"; do
	echo "$line aaa"
done
Why does it print "aaa" only once in the end instead of after each line of the string? Thank you.
 
Old 04-19-2012, 04:46 PM   #2
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
If you put $string between double quotes it won't be split in words but seen as a single string and the block of code inside the loop will be executed only once. What does $string contain, anyway? Is it a multiline string?

Last edited by colucix; 04-19-2012 at 04:47 PM.
 
Old 04-19-2012, 06:49 PM   #3
vril
LQ Newbie
 
Registered: Apr 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Yes, it 's a multiline string that contains the output of a unix command. So, would you suggest avoiding double quotes? I remember having ommited them, but I got the error that access to the string is forbidden or something like that.
 
Old 04-19-2012, 06:59 PM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,260

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
Try it without the quotes; if you have issues come back and show us exactly what it contains. Special chars (punctuation ) can have side effects and may need escaping.
 
Old 04-19-2012, 11:27 PM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
The syntax of a for loop is this:

Code:
for variable in <wordlist> ; do
	commands
done
Each iteration takes a "word" from the list into the variable and runs the sub-commands. Note that the definition of word depends on shell quoting and/or your IFS setting. So this:

foo bar "baz bum"

...is three words. "baz bum" is considered a single unit.

If the word list contains an unquoted variable, then the variable is first expanded, then word-split and glob expanded by the shell, to create the final word list. But if it's double-quoted, the expanded value is treated as a single unit. That's why you are getting the whole block of text at once; it's being seen as only one "word".

Code:
var='foo bar baz bum'
for word in hello $var "$var" goodbye; do
	echo "$word"
done

#output:
hello
foo
bar
baz
bum
foo bar baz bum
goodbye
Notice how unquoting it doesn't work either, because then your text is broken up by whitespace.


Now, to get to the meat of the matter, there's really no way for you to safely do exactly what you want with a for loop. The shell just isn't capable of safely delimiting variable contents into individual lines in cases like this. The closest you can get is by setting IFS to newline only, but even that means blank lines will be lost.

It's all explained here:

http://mywiki.wooledge.org/DontReadLinesWithFor


So you really need to use a while+read loop instead, or some other technique.

Code:
while read line; do
	echo "$line aaa"
done <<<"$string"

However, since a variable is really designed for holding single, short strings, a better technique may be to store your text in an array instead, with one line per index. Arrays are designed for holding lists of things, and so are much better suited for storing multi-lined data.

To convert your scalar variable into an array you can use the mapfile command (bash4+).

Code:
mapfile -t strarray <<<"$string"

for line in "${strarray[@]}"; do
	echo "$line aaa"
done

If you're wondering, both of the techniques above use here strings to send the contents of the variable to the command's stdin.

Last edited by David the H.; 04-19-2012 at 11:30 PM.
 
Old 04-20-2012, 08:03 AM   #6
vril
LQ Newbie
 
Registered: Apr 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by chrism01 View Post
Try it without the quotes; if you have issues come back and show us exactly what it contains. Special chars (punctuation ) can have side effects and may need escaping.
Ok I just tried it. The issue I come up with now is that it prints the string word by word, whereas I want line by line.
 
Old 04-20-2012, 09:16 AM   #7
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Quote:
Originally Posted by vril View Post
Ok I just tried it. The issue I come up with now is that it prints the string word by word, whereas I want line by line.
Of course! Being a multiple lines string both the blank spaces and the newlines act as Input Field Separator. Check the details about IFS in the bash man page. There are a lot of ways to accomplish this task. One of them has already been suggested by David the H. in post #5. Here it is again for your convenience:
Code:
while read line
do
  echo "$line aaa"
done <<< "$string"
Another one is to change the value of the IFS to a newline, so that blank spaces are preserved as part of a single line:
Code:
OLD_IFS="$IFS"
IFS="
" 
for line in $string
do
  echo "$line aaa"
done
IFS="$OLD_IFS"
Other not-pure-bash solutions:
Code:
echo "$string" | sed 's/$/ aaa/'
or
Code:
echo "$string" | awk '$0=$0 " aaa"'
You have something to learn, now! Here are some useful references:
http://www.gnu.org/software/bash/manual/bashref.html
http://www.grymoire.com/Unix/Sed.html
http://www.grymoire.com/Unix/Awk.html
http://www.gnu.org/software/gawk/manual/gawk.html
 
Old 04-22-2012, 06:18 PM   #8
AoZ.
LQ Newbie
 
Registered: Apr 2012
Posts: 1

Rep: Reputation: Disabled
Code:
for line in $(ls); do
  echo "$line aaa"
done
in this case, you would find "aaa" are printed several times.
Similarly,
Code:
export string='sss aaa ses'
for line in $(ls); do
   echo "$line aaa"
   echo "this is new"
done
You would find "this is new" would be printed out only once. That is, the loop only iterated once. However, for $(ls), which has multiple lines, "aaa" should be printed out several times.
But you mentioned "multi-lines". So I am not quite sure if this helps...
 
Old 04-22-2012, 11:29 PM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
Quote:
Originally Posted by AoZ. View Post
Similarly,
Code:
export string='sss aaa ses'
for line in $(ls); do
   echo "$line aaa"
   echo "this is new"
done
You would find "this is new" would be printed out only once. That is, the loop only iterated once. However, for $(ls), which has multiple lines, "aaa" should be printed out several times.
But you mentioned "multi-lines". So I am not quite sure if this helps...

Uh what? I think perhaps you meant to use:

Code:
export string='sss aaa ses'
for line in "$string"; do
	echo ...
That would execute the echos only once, since as I explained you'd get only one word. But that's due to the quoting, not the use of a variable. Unquote it and you'll get three iterations. Quote the "$(ls)" and you'll get one iteration again too.

But all this is just a distraction. I've already explained both the problem and the solution. But to say it once again, as clearly as possible:


DON'T READ LINES WITH FOR!


This includes the output of commands, variables, or text files. It just does not work safely. Go back and read the link I provided, and re-read it until you understand it completely. Then go change your code to use a while+read loop or an array.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off




All times are GMT -5. The time now is 07:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration