LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 08-04-2005, 12:20 PM   #1
lda_ga
LQ Newbie
 
Registered: Jul 2005
Posts: 3

Rep: Reputation: 0
Character conversion combining cat and pipe


I have a script that I want to run several dozen times with varying command line arguments. I put these sets of command line args in a file, intending to cat the file and supply each line to the script. It did not work. each space in the feed line was converted to a newline "\n". I've duplicated this on Fedora Core 4, RH9, and I'll bet it does the same on my SUSE 9.3 at home tonight. Here is a one line file demo of what happens:

[scripts]$ cat test1
alpha:beta:gamma:delta:"bingo -":charlie
[scripts]$ od -bc test1
0000000 141 154 160 150 141 072 142 145 164 141 072 147 141 155 155 141
a l p h a : b e t a : g a m m a
0000020 072 144 145 154 164 141 072 042 142 151 156 147 157 040 055 042
: d e l t a : " b i n g o - "
0000040 072 143 150 141 162 154 151 145 012
: c h a r l i e \n
0000051
[scripts]$ for i in `cat test1`; do echo $i | od -bc -; done
0000000 141 154 160 150 141 072 142 145 164 141 072 147 141 155 155 141
a l p h a : b e t a : g a m m a
0000020 072 144 145 154 164 141 072 042 142 151 156 147 157 012
: d e l t a : " b i n g o \n
0000036
0000000 055 042 072 143 150 141 162 154 151 145 012
- " : c h a r l i e \n
0000013
[scripts]$

Does anyone have a solution to this? It seems that a simple "cat" displays the line correctly, but when pipe'd, it converts the space (octal 040) to an octal 012, a newline.
The effect is that the script only receives the first word in the argument line.

lda
 
Old 08-04-2005, 12:57 PM   #2
mschutte
Member
 
Registered: Jan 2005
Location: Innsbruck, Austria
Distribution: Debian GNU/Linux Lenny
Posts: 68

Rep: Reputation: 15
This is because the line
$ for i in $(cat test1); do echo $i | od -bc -; done
is expanded by the shell and becomes
$ for i in 'alpha:beta:gamma:delta:"bingo ' '-":charlie'; do ...; done
This means that hexdumper od is invoked two times and gets input from echo, which appends a newline to its argument. What you probably want to do is:
$ for i in $(cat test1); do echo -n $i | od -bc -; done
-n prevents echo from adding a newline.

Hope I could help,
mschutte
 
Old 08-04-2005, 01:17 PM   #3
lda_ga
LQ Newbie
 
Registered: Jul 2005
Posts: 3

Original Poster
Rep: Reputation: 0
You may be correct that the shell is taking the space and doing strange things with it. however, your suggestion still does not solve the problem. Just doing a cat within an inline script breaks things up. I changed the inline script to eliminate the echo and pipe and it still does the same thing - indicating shell influence:

[scripts]$ for i in $(cat test2)
> do
> echo $i
> done
alpha
beta
gamma
delta
"bingo
-"
charlie
[scripts]$ cat test2
alpha beta gamma delta "bingo -" charlie
[scripts]$

The question is how to make the shell ignore the spaces and treat the whole line as a single item... I'm not sure where to take this from here. i've tried putting single quotes at begin and end of each line, double quotes... has me totally baffled.

lda
 
Old 08-04-2005, 02:19 PM   #4
mschutte
Member
 
Registered: Jan 2005
Location: Innsbruck, Austria
Distribution: Debian GNU/Linux Lenny
Posts: 68

Rep: Reputation: 15
I don't know if I understand you correctly, but I assume you want to process the file linewise. If this is your goal, a loop with read would be the right way:
Code:
while read LINE
do
	echo $LINE | od -bc -
done < test1
If you want to split the file at the colons instead, do it like this:
Code:
OIFS="$IFS"
IFS=:
for ITEM in $(cat file1)
do
	echo $ITEM | od -bc
done
IFS="$OIFS"
Probably I could help you this time.
-- mschutte
 
Old 08-04-2005, 02:38 PM   #5
lda_ga
LQ Newbie
 
Registered: Jul 2005
Posts: 3

Original Poster
Rep: Reputation: 0
The "while read" format works like I need - Thanks for the tip.

It still disturbs me that cat seems to be substituting newline for space. I've tried several dozen variations on the original cat and echo, every option to cat and echo in many different combinations, and nothing with those 2 in combination works (keeps all the words in the line, in the same line).

Thanks again

lda
 
Old 08-04-2005, 02:45 PM   #6
mschutte
Member
 
Registered: Jan 2005
Location: Innsbruck, Austria
Distribution: Debian GNU/Linux Lenny
Posts: 68

Rep: Reputation: 15
Actually, cat doesn't »eat« the spaces. It's the shell which splits the cat output into tokens (at every space). So the body of the for loop is executed for every token in the file, which implies that ob is executed for each token. And for your purpose, this is too often.

-- mschutte
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
combining pdfs RaI Linux - Software 1 09-09-2005 01:31 PM
RH-Firewall combining johnnydangerous Fedora 6 02-07-2005 06:00 AM
Write-combining thiagorobert Linux - Hardware 3 05-11-2004 09:38 PM
Pipe character ('|') on keyboard. caesius_01 Linux - General 5 10-18-2003 02:12 AM
Is the pipe (|) an illegal character for unix/linux filenames? Foebane Linux - General 7 07-08-2002 12:16 AM


All times are GMT -5. The time now is 11:40 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration