Character conversion combining cat and pipe
 

I have a script that I want to run several dozen times with varying command line arguments. I put these sets of command line args in a file, intending to cat the file and supply each line to the script. It did not work. each space in the feed line was converted to a newline "\n". I've duplicated this on Fedora Core 4, RH9, and I'll bet it does the same on my SUSE 9.3 at home tonight. Here is a one line file demo of what happens:

[scripts]$ cat test1
alpha:beta:gamma:delta:"bingo -":charlie
[scripts]$ od -bc test1
0000000 141 154 160 150 141 072 142 145 164 141 072 147 141 155 155 141
a l p h a : b e t a : g a m m a
0000020 072 144 145 154 164 141 072 042 142 151 156 147 157 040 055 042
: d e l t a : " b i n g o - "
0000040 072 143 150 141 162 154 151 145 012
: c h a r l i e \n
0000051
[scripts]$ for i in `cat test1`; do echo $i | od -bc -; done
0000000 141 154 160 150 141 072 142 145 164 141 072 147 141 155 155 141
a l p h a : b e t a : g a m m a
0000020 072 144 145 154 164 141 072 042 142 151 156 147 157 012
: d e l t a : " b i n g o \n
0000036
0000000 055 042 072 143 150 141 162 154 151 145 012
- " : c h a r l i e \n
0000013
[scripts]$

Does anyone have a solution to this? It seems that a simple "cat" displays the line correctly, but when pipe'd, it converts the space (octal 040) to an octal 012, a newline.
The effect is that the script only receives the first word in the argument line.

lda

This is because the line
$ for i in $(cat test1); do echo $i | od -bc -; done
is expanded by the shell and becomes
$ for i in 'alpha:beta:gamma:delta:"bingo ' '-":charlie'; do ...; done
This means that hexdumper od is invoked two times and gets input from echo, which appends a newline to its argument. What you probably want to do is:
$ for i in $(cat test1); do echo -n $i | od -bc -; done
-n prevents echo from adding a newline.

Hope I could help,
mschutte

You may be correct that the shell is taking the space and doing strange things with it. however, your suggestion still does not solve the problem. Just doing a cat within an inline script breaks things up. I changed the inline script to eliminate the echo and pipe and it still does the same thing - indicating shell influence:

[scripts]$ for i in $(cat test2)
> do
> echo $i
> done
alpha
beta
gamma
delta
"bingo
-"
charlie
[scripts]$ cat test2
alpha beta gamma delta "bingo -" charlie
[scripts]$

The question is how to make the shell ignore the spaces and treat the whole line as a single item... I'm not sure where to take this from here. i've tried putting single quotes at begin and end of each line, double quotes... has me totally baffled.

lda

I don't know if I understand you correctly, but I assume you want to process the file linewise. If this is your goal, a loop with read would be the right way:
If you want to split the file at the colons instead, do it like this:
Probably I could help you this time.
-- mschutte

The "while read" format works like I need - Thanks for the tip.

It still disturbs me that cat seems to be substituting newline for space. I've tried several dozen variations on the original cat and echo, every option to cat and echo in many different combinations, and nothing with those 2 in combination works (keeps all the words in the line, in the same line).

Thanks again

lda

Actually, cat doesn't »eat« the spaces. It's the shell which splits the cat output into tokens (at every space). So the body of the for loop is executed for every token in the file, which implies that ob is executed for each token. And for your purpose, this is too often.

-- mschutte