bash/sh: global or local variable with for (loop) isssue

frenchn00b · 11-04-2008, 05:08 PM

Hello,

I wanna prepare for growisofs, with taking the files, removing space, and to put them not in column but in line.

my code is:

Code:

 ls -1 | sedremovespace  | while read GL ; do FINAL="$FINAL $GL " ; done ; echo "$FINAL"

returns nothing because FINAL is empty

with /usr/bin/sedremovespace:

Code:

sed 's/\ /\\\ /g'

Would anyone have any ideas. Thank you!

jlinkels · 11-04-2008, 06:33 PM

Removed the post because of an embarrassing mistake.

jan61 · 11-05-2008, 03:08 PM

Moin,

every pipe creates a subshell and that's why nothing you assign to a variable after a pipe is visible outside. It's a common mistake: A variable can't be exported from a subshell to the parent shell.

You can do it like this (look here for explanations: http://www.jan-trippler.de/en/script...variables.html):

Code:

FINAL="`ls -1 | sedremovespace  | while read GL; do echo -n \"$GL \"; done`"

Jan

EDIT: But there's an even more simplier way: You don't need the read loop.

Code:

FINAL="`ls -1 | sedremovespace`"

frenchn00b · 11-05-2008, 03:55 PM

Code:

      #!/bin/bash 
         counter=0 ; ls -1 | while read n ; do let counter+=1 ; done ; 
         echo $counter

LOOKS IMPOSSIBLE !!

cuz echo $counter returns nothing

chrism01 · 11-05-2008, 05:45 PM

Like Jan61 said

Quote:

every pipe creates a subshell and that's why nothing you assign to a variable after a pipe is visible outside.

You need a different technique eg

Code:

counter=0
for file in `ls`
do
    let counter+=1
done
echo $counter

frenchn00b · 11-06-2008, 12:34 AM

Quote:

Originally Posted by chrism01

Like Jan61 said

You need a different technique eg

Code:

counter=0
for file in `ls`
do
    let counter+=1
done
echo $counter

The big problem is this one with files :

if cat /tmp/files.tmp

Code:

/home/french/the file with spaces.doc
/home/french/the file with spaces222 333.doc
/home/french/thefile.doc
/home/french/the file with spaces222 333 5555.doc

Code:

counter=0
for file in `cat /tmp/files.tmp`
do
    let counter+=1
    echo `du -b "$file" | tail -n 1 | cut -f 1`
done
echo $counter

then your script doesnt work because for / "$file" are messed up with spaces and not LF/chr(10)chr(13)

Help please

chrism01 · 11-06-2008, 01:40 AM

In that case, set the IFS to a newline only at the start.
http://tldp.org/LDP/abs/html/internalvariables.html

jlliagre · 11-06-2008, 02:33 AM

Quote:

Originally Posted by frenchn00b

Code:

      #!/bin/bash 
         counter=0 ; ls -1 | while read n ; do let counter+=1 ; done ; 
         echo $counter

LOOKS IMPOSSIBLE !!

cuz echo $counter returns nothing

Your script works fine if you replace /bin/bash by /bin/ksh or /bin/zsh.
In my opinion, bash developers took an unfortunate decision when choosing which part of a pipeline would be run in a subshell.

jlinkels · 11-06-2008, 06:17 AM

Quote:

Originally Posted by jlliagre

In my opinion, bash developers took an unfortunate decision when choosing which part of a pipeline would be run in a subshell.

Not so much the decision where to start the subshell is bad (*nix contains more very rigid orthogonalities which you have to learn to live with, either good or bad) but incompatibility between the shells is disastrous in my opinion. It doesn't only mean that this script runs the way it is intended in other shells while not in bash, it also means that scripts which run in bash using this property may crash or give unintentional results in other shells.

As for this particular script, I don't see the use for piping. I have seen a few of this similar problems over the past few months which all concerned piping into a loop. Piping the result of one program into another (like

Code:

cat myfile | grep foo | sed s/foo/bar/g

is logical and harmless. Piping into a loop

Code:

ls -1 | while read .... done

might be syntactically correct and executable, it still is not elegant programming. It is awkward to have one statement producing one result, and feed that into a repetitive loop using a part of that result bit by bit, without you are clearly aware of doing that. This is clearly different from, for example, producing a string and parse it by token in the next loop.
The example chrism01 provided is much cleaner, provides hooks for debugging and makes the code obvious and understandable. Maybe programmers should be discouraged of using pipes in this manner anyway.

jlinkels

jlliagre · 11-06-2008, 07:24 AM

Quote:

Originally Posted by jlinkels

Not so much the decision where to start the subshell is bad (*nix contains more very rigid orthogonalities which you have to learn to live with, either good or bad) but incompatibility between the shells is disastrous in my opinion.

These are different points.
Either you explicitely specify the shell you use (bash, ksh, zsh, whatever) and you are free to take advantage of their specific enhancements.
Either you want to write portable scripts and you need to stick to POSIX shell statements and syntax.
What is unfortunate in that specific pipe subshell case is POSIX didn't choose to follow ksh (which was the model for most features) and let that choice implementation dependent. Scripts that rely on variable sets in pipeline components are thus all non portable.

Quote:

It doesn't only mean that this script runs the way it is intended in other shells while not in bash, it also means that scripts which run in bash using this property may crash or give unintentional results in other shells.

True, but the interpreter should be specified to avoid the risk.

Quote:

As for this particular script, I don't see the use for piping.

Yes. I just pointed it as an example.

Quote:

I have seen a few of this similar problems over the past few months which all concerned piping into a loop. Piping the result of one program into another (like

Code:

cat myfile | grep foo | sed s/foo/bar/g

is logical and harmless.

Harmless but far to be optimized. Both cat and grep are unnecessary:

Code:

sed -n 's/foo/bar/gp' myfile