for...in

vril · 04-19-2012, 03:24 PM

Let's say I need to loop through all lines of a string in bash to do the following:

Code:

for line in "$string"; do
	echo "$line aaa"
done

Why does it print "aaa" only once in the end instead of after each line of the string? Thank you.

colucix · 04-19-2012, 03:46 PM

If you put $string between double quotes it won't be split in words but seen as a single string and the block of code inside the loop will be executed only once. What does $string contain, anyway? Is it a multiline string?

vril · 04-19-2012, 05:49 PM

Yes, it 's a multiline string that contains the output of a unix command. So, would you suggest avoiding double quotes? I remember having ommited them, but I got the error that access to the string is forbidden or something like that.

chrism01 · 04-19-2012, 05:59 PM

Try it without the quotes; if you have issues come back and show us exactly what it contains. Special chars (punctuation ) can have side effects and may need escaping.

David the H. · 04-19-2012, 10:27 PM

The syntax of a for loop is this:

Code:

for variable in <wordlist> ; do
	commands
done

Each iteration takes a "word" from the list into the variable and runs the sub-commands. Note that the definition of word depends on shell quoting and/or your IFS setting. So this:

foo bar "baz bum"

...is three words. "baz bum" is considered a single unit.

If the word list contains an unquoted variable, then the variable is first expanded, then word-split and glob expanded by the shell, to create the final word list. But if it's double-quoted, the expanded value is treated as a single unit. That's why you are getting the whole block of text at once; it's being seen as only one "word".

Code:

var='foo bar baz bum'
for word in hello $var "$var" goodbye; do
	echo "$word"
done

#output:
hello
foo
bar
baz
bum
foo bar baz bum
goodbye

Notice how unquoting it doesn't work either, because then your text is broken up by whitespace.

Now, to get to the meat of the matter, there's really no way for you to safely do exactly what you want with a for loop. The shell just isn't capable of safely delimiting variable contents into individual lines in cases like this. The closest you can get is by setting IFS to newline only, but even that means blank lines will be lost.

It's all explained here:

http://mywiki.wooledge.org/DontReadLinesWithFor

So you really need to use a while+read loop instead, or some other technique.

Code:

while read line; do
	echo "$line aaa"
done <<<"$string"

However, since a variable is really designed for holding single, short strings, a better technique may be to store your text in an array instead, with one line per index. Arrays are designed for holding lists of things, and so are much better suited for storing multi-lined data.

To convert your scalar variable into an array you can use the mapfile command (bash4+).

Code:

mapfile -t strarray <<<"$string"

for line in "${strarray[@]}"; do
	echo "$line aaa"
done

If you're wondering, both of the techniques above use here strings to send the contents of the variable to the command's stdin.

vril · 04-20-2012, 07:03 AM

Quote:

Originally Posted by chrism01

Try it without the quotes; if you have issues come back and show us exactly what it contains. Special chars (punctuation ) can have side effects and may need escaping.

Ok I just tried it. The issue I come up with now is that it prints the string word by word, whereas I want line by line.

colucix · 04-20-2012, 08:16 AM

Quote:

Originally Posted by vril

Ok I just tried it. The issue I come up with now is that it prints the string word by word, whereas I want line by line.

Of course! Being a multiple lines string both the blank spaces and the newlines act as Input Field Separator. Check the details about IFS in the bash man page. There are a lot of ways to accomplish this task. One of them has already been suggested by David the H. in post #5. Here it is again for your convenience:

Code:

while read line
do
  echo "$line aaa"
done <<< "$string"

Another one is to change the value of the IFS to a newline, so that blank spaces are preserved as part of a single line:

Code:

OLD_IFS="$IFS"
IFS="
" 
for line in $string
do
  echo "$line aaa"
done
IFS="$OLD_IFS"

Other not-pure-bash solutions:

Code:

echo "$string" | sed 's/$/ aaa/'

or

Code:

echo "$string" | awk '$0=$0 " aaa"'

You have something to learn, now!

Here are some useful references:
http://www.gnu.org/software/bash/manual/bashref.html
http://www.grymoire.com/Unix/Sed.html
http://www.grymoire.com/Unix/Awk.html
http://www.gnu.org/software/gawk/manual/gawk.html

AoZ. · 04-22-2012, 05:18 PM

Code:

for line in $(ls); do
  echo "$line aaa"
done

in this case, you would find "aaa" are printed several times.
Similarly,

Code:

export string='sss aaa ses'
for line in $(ls); do
   echo "$line aaa"
   echo "this is new"
done

You would find "this is new" would be printed out only once. That is, the loop only iterated once. However, for $(ls), which has multiple lines, "aaa" should be printed out several times.
But you mentioned "multi-lines". So I am not quite sure if this helps...

David the H. · 04-22-2012, 10:29 PM

Quote:

Originally Posted by AoZ.

Similarly,

Code:

export string='sss aaa ses'
for line in $(ls); do
   echo "$line aaa"
   echo "this is new"
done

You would find "this is new" would be printed out only once. That is, the loop only iterated once. However, for $(ls), which has multiple lines, "aaa" should be printed out several times.
But you mentioned "multi-lines". So I am not quite sure if this helps...

Uh what? I think perhaps you meant to use:

Code:

export string='sss aaa ses'
for line in "$string"; do
	echo ...

That would execute the echos only once, since as I explained you'd get only one word. But that's due to the quoting, not the use of a variable. Unquote it and you'll get three iterations. Quote the "$(ls)" and you'll get one iteration again too.

But all this is just a distraction. I've already explained both the problem and the solution. But to say it once again, as clearly as possible:

DON'T READ LINES WITH FOR!

This includes the output of commands, variables, or text files. It just does not work safely. Go back and read the link I provided, and re-read it until you understand it completely. Then go change your code to use a while+read loop or an array.