sed strange behaviour with line including backslash.
 

Hello,

I have a small script that takes input from one file, and uses this input to do a find/replace (with sed) to make an executable snmp trap.

The input file could look something like this :
JOHBURG SOUTH AFRICA
OSLO NORWAY
...
etc

The script takes the input from this file, and preserves all words in one line as a variable (so that the variables are "JOHBURG SOUTH AFRICA" and "OSLO NORWAY", instead of JOHBURG, SOUTH, AFRICA, OSLO, NORWAY.
This is working ok (in the given example, I get two variables).

sed is supposed to take this variable, and use it to substitute "HITME" with the new variable.


#!/bin/bash

N=0
cat tutticodes | while read LINE
do
N=$((N+1))
echo "Line $N = $LINE"
sed -e "s/HITME/${LINE}/g" template2 | sed -e 's/TERMINATE/\\/g'
done


The template2 file, looks like this :

/usr/bin/snmptrap -v 1 -c public 10.10.10.10 1.3.6.1.4.1.2378.1.1.1.1.4 10.10.10.11 0 2 ' ' \
1.3.6.1.4.1.2378.1.1.1.1.4.2.1 i 5 \
1.3.6.1.4.1.2378.1.1.1.1.4.2.2 i 6 \
1.3.6.1.4.1.2378.1.1.1.1.4.2.3 i 7 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.2 i IINNTT \
1.3.6.1.4.1.2378.1.1.1.1.4.3.3 s HITME \
1.3.6.1.4.1.2378.1.1.1.1.4.3.4 i 5 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.5 s "Some text" \
1.3.6.1.4.1.2378.1.1.1.1.4.2.4 s "timestamp"


When I run the script, against the template file, sed returns :
...snip...

1.3.6.1.4.1.2378.1.1.1.1.4.2.3 i 7 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.2 i IINNTT \
\3.6.1.4.1.2378.1.1.1.1.4.3.3 s ATPC ALM
1.3.6.1.4.1.2378.1.1.1.1.4.3.4 i 5 \
...snip...

As you can see, it has made the replacement, but the trailing "\" disappears (or rather, moves to the start of the same output line).
To me it appears irrational, and I cannot figure out why it does.

Any ideas?

Cheers,
Simon.

I'm finding it hard to figure out what you are trying to do. Maybe its because I'm not that into snmp.

Your code looks a bit inconsistent and its not the "standard way" of accessing and manipulating files.

Where did ATPC come from? and whats in the file tutticodes?

Post a snippet of all files with there names, the result you are getting and the result you want. Then I can hopefully help you.

Are there any unusual characters in "tutticodes"?
Also the second sed seems to also make no sense as neither file, based on current examples, would have this word.

You might need to explain a little further?

I too have trouble following this. For example in this statement:
sed -e 's/TERMINATE/\\/g'
Where does "TERMINATE" exist anywhere?

First, try the problem substitution by itself---without any of the other stuff---and show us the before and after

Ok, I realise I put this together a bit too fast...

tuttifiles is my input file.
Containing text strings (consisting of words), for example, "JOHBURG SOUTH*AFRICA",*"OSLO*NORWAY",*"ATPC*ALM" etc.
(I realise I should have given ATPC ALM in the example too, otherwise, it wouldn´t make sense...)

The piped sed shouldn´t be there (it was just one of my many attempts of trying to work around the problem).
So, the script should read ONLY:
sed -e "s/HITME/${LINE}/g" template2

Running the script, it is supposed to read tutticodes, take each line as one input, and substitute HITME with its value.
The problem is the trailing backslash (which I am not trying to substitute, but rather leave for line continuation in the end result).

What I expect as a result is :
...snip...
1.3.6.1.4.1.2378.1.1.1.1.4.2.3 i 7 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.2 i IINNTT \
1.3.6.1.4.1.2378.1.1.1.1.4.3.3 s ATPC ALM \
1.3.6.1.4.1.2378.1.1.1.1.4.3.4 i 5 \
...snip

but, there are two "problems" with the output, namely :
(This is the output that I get)
\3.6.1.4.1.2378.1.1.1.1.4.3.3 s ATPC ALM

The leading text is malformed (the 1. is missing, replaced by the very backslash that´s missing as the trailing one I had in my template file...

Hope that makes things a bit more clear, and apologies for not including enough info.
by copy :
to rical, never mind it´s SNMP, it is just text that happens to be an SNMP script.
to grail, there are no unusual characters in the tutticodes file, only words and spaces.

Cheers, and thanks for the response so far !
Simon.

Still don't make much sense to me.
Please give us example files like

input.txt
JOHBURG SOUTH
FOO BAR
BAR BAZ

script.sh
#!bin/bash
....

My result of command: ./script.sh input.txt
1.3.6.1.4.1.2378.1.1.1.1.4.2.3 i 7 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.2 i IINNTT \
\1.3.6.1.4.1.2378.1.1.1.1.4.3.3 s ATPC ALM
1.3.6.1.4.1.2378.1.1.1.1.4.3.4 i 5 \

expected result
1.3.6.1.4.1.2378.1.1.1.1.4.2.3 i 7 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.2 i IINNTT \
1.3.6.1.4.1.2378.1.1.1.1.4.3.3 s ATPC ALM \
1.3.6.1.4.1.2378.1.1.1.1.4.3.4 i 5 \


If you do this, it takes us 2 seconds to copy-paste it into files and try it ourself.

The fist post suggest the tuttifiles file is formated as:
JOHBURG SOUTH AFRICA
OSLO NORWAY

And you last post suggest:
SOUTH*AFRICA",*"OSLO*NORWAY",*"ATPC*ALM

Is this a regexp explanation of the files structure or is it the actual file? There is a huge difference.

I would love to help, but its still to unclear for me to easily follow what your trying to do, sorry.

Ok,

Here we go :
########
script.sh
########
#!/bin/bash

N=0
cat tutticodes | while read LINE
do
N=$((N+1))
echo "Line $N = $LINE"
sed -e "s/HITME/${LINE}/g" template2 # Only writes to screen, will be redirected to file once working
done


########
tutticodes (input file) save as tutticodes
########
NEW YORK
OSLO NORWAY
ONE TWO THREE
SOME OTHER WORDS


########
template2 (template file) save as template2
########
/usr/bin/snmptrap -v 1 -c public 10.10.10.10 1.3.6.1.4.1.2378.1.1.1.1.4 10.10.10.11 0 2 ' ' \
1.3.6.1.4.1.2378.1.1.1.1.4.2.1 i 5 \
1.3.6.1.4.1.2378.1.1.1.1.4.2.2 i 6 \
1.3.6.1.4.1.2378.1.1.1.1.4.2.3 i 7 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.2 i IINNTT \
1.3.6.1.4.1.2378.1.1.1.1.4.3.3 s HITME \
1.3.6.1.4.1.2378.1.1.1.1.4.3.4 i 5 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.5 s "Some text" \
1.3.6.1.4.1.2378.1.1.1.1.4.2.4 s "timestamp"

########
My result
########
...snip...
1.3.6.1.4.1.2378.1.1.1.1.4.2.3 i 7 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.2 i IINNTT \
\3.6.1.4.1.2378.1.1.1.1.4.3.3 s NEW YORK
1.3.6.1.4.1.2378.1.1.1.1.4.3.4 i 5 \
...snip...

########
Expected result
########
...snip...
1.3.6.1.4.1.2378.1.1.1.1.4.2.3 i 7 \
1.3.6.1.4.1.2378.1.1.1.1.4.3.2 i IINNTT \
1.3.6.1.4.1.2378.1.1.1.1.4.3.3 s NEW YORK \
1.3.6.1.4.1.2378.1.1.1.1.4.3.4 i 5 \
...snip...

Making these three files should make it reproducible
script.sh, tutticodes and template2
Place all in same directory, and execute script.sh

I believe (from memory) that sed is version 4.1.5

Cheers,
Simon.

Hi,

please use code tags when posting code. Makes it a lot easier to read.

I copy+pasted your samples and this is the output I got:
Looks like what you were expecting.

Hello,

ok, that confirms I'm still sane, but I get a different output. Can you please state your version of sed?

I am running exacly the same code, but output is malformed...

Cheers,
Simon.

GNU sed version 4.2.1. Do you get only malformed output on screen or also when you redirect it to a file?

I have tried this in GNU sed version 4.2.1 and 4.1.5 and I get the "correct" output:

What os and version of sed are you running?

I found out what the problem was!
tutticodes came from a windows box, ran dos2unix on the file, and volià!
(I know vi interprets DOS coding (8bit) ending with "^M").
That must have confused sed.

Silly, but I couldn't see the forest for the trees.
Thanks for all the contributions :)

Simon