[SOLVED] Bash - Replace a text with a variable containing new lines character

vikas027 · 05-11-2018, 06:04 AM

Hey All,

I am trying to replace a variable in sed, usually, it can be done by double quotes but my variable has new lines characters and backslashes which somehow is not working.

This is a sample variable

Code:

# echo $CLIENT_KEY
--BEGIN PRIVATE KEY--\nAAA\nBBB\nCCC\n--END PRIVATE KEY--
#

I want to replace this variable's value in a file

Code:

# cat myfile
ABC
CLIENT_KEY
#

# cat myfile | sed "s/CLIENT_KEY/${CLIENT_KEY}/"  (does not works)
ABC
--BEGIN PRIVATE KEY--\nAAA\nBBB\nCCC\n--END PRIVATE KEY--
#

Thanks!

Turbocapitalist · 05-11-2018, 06:18 AM

I'm not sure how it can be done with the multiple line replacement. It's easy with Perl 5 however.

Code:

cat myfile | perl -p -e "s|CLIENT_KEY|${CLIENT_KEY}|"

wpeckham · 05-11-2018, 06:23 AM

#1 this appears to be a SED question, not a BASH question, so your title is misleading.

#2 for what you want to do, I am not sure that SED is the correct tool. PERL would be optimal, GSAR would serve.

Could we have some detail about what you are really trying to achieve, and why you selected SED (not BASH) to approach the problem?

syg00 · 05-11-2018, 07:01 AM

The two character text string "\n" is just that - not a newline unless interpolated somehow. Perl, sed would make no difference.
Embedding 0x0A might work. Untested.

danielbmartin · 05-11-2018, 07:32 AM

Help us to help you. Start by constructing a simplified InFile such as ...

Code:

--BEGIN PRIVATE KEY--\nAAA\nBBB\nCCC\n\n--END PRIVATE KEY--

... and then construct the desired OutFile. Post both files here.

These files will facilitate understanding and give us an example for testing.

Daniel B. Martin

.

vikas027 · 05-11-2018, 08:16 AM

Quote:

Originally Posted by Turbocapitalist

I'm not sure how it can be done with the multiple line replacement. It's easy with Perl 5 however.

Code:

cat myfile | perl -p -e "s|CLIENT_KEY|${CLIENT_KEY}|"

Hey,

Thanks, but this did not work, it interpolates "\n" into new lines. I've tried with perl v5.16.3

vikas027 · 05-11-2018, 08:19 AM

Quote:

Originally Posted by wpeckham

#1 this appears to be a SED question, not a BASH question, so your title is misleading.

#2 for what you want to do, I am not sure that SED is the correct tool. PERL would be optimal, GSAR would serve.

Could we have some detail about what you are really trying to achieve, and why you selected SED (not BASH) to approach the problem?

Hey wpeckham,

I deliberately chose bash and not sed, as I am on bash shell and (as you correctly hinted) that sed may not be the correct tool. I have tried sed, because that was the first tool which came into my mind for replacing text and which has been working great for all my needs so far.

danielbmartin · 05-11-2018, 08:59 AM

With this InFile ...

Code:

ABC
CLIENT_KEY
DEF

... this code ...

Code:

str="--BEGIN PRIVATE KEY--\nAAA\nBBB\nCCC\n\n--END PRIVATE KEY--"
sed "s/CLIENT_KEY/$str/"<$InFile >$OutFile
echo "InFile ...";  cat $InFile;  echo "End Of InFile ($(wc -l <$InFile) lines)."
echo
echo "OutFile ..."; cat $OutFile; echo "End Of OutFile ($(wc -l <$OutFile) lines)."

... produced this result ...

Code:

InFile ...
ABC
CLIENT_KEY
DEF
End Of InFile (3 lines).

OutFile ...
ABC
--BEGIN PRIVATE KEY--
AAA
BBB
CCC

--END PRIVATE KEY--
DEF
End Of OutFile (8 lines).

Daniel B. Martin

.

Turbocapitalist · 05-11-2018, 09:10 AM

Quote:

Originally Posted by vikas027

... it interpolates "\n" into new lines. I've tried with perl v5.16.3

That's what you asked for above. If not please show the files as danielbmartin requested above so that we can see what you are trying to achieve.

danielbmartin · 05-11-2018, 09:20 AM

... Or maybe this is what you want ...

With this InFile ...

Code:

ABC
CLIENT_KEY
DEF

... this code ...

Code:

str="--BEGIN PRIVATE KEY--\nAAA\nBBB\nCCC\n\n--END PRIVATE KEY--"
sed "s/CLIENT_KEY/$str/"<$InFile   \
|paste -s -d~ -                    \
|sed 's/~/\\n/g'                   \
>$OutFile
echo "InFile ...";  cat $InFile;  echo "End Of InFile ($(wc -l <$InFile) lines)."
echo
echo "OutFile ..."; cat $OutFile; echo "End Of OutFile ($(wc -l <$OutFile) lines)."

... produced this OutFile ...

Code:

ABC\n--BEGIN PRIVATE KEY--\nAAA\nBBB\nCCC\n\n--END PRIVATE KEY--\nDEF

Daniel B. Martin

.

MadeInGermany · 05-12-2018, 03:22 AM

I think the two characters \ and n should be kept here. But a RE interprets it.
How about bash (as the title already suggests)?

Code:

str="--BEGIN PRIVATE KEY--\nAAA\nBBB\nCCC\n\n--END PRIVATE KEY--"
while IFS= read -r line
do
  case $line in
  (CLIENT_KEY*) line=$str;;
  esac
  printf "%s\n" "$line"
done <myfile

wpeckham · 05-12-2018, 06:45 AM

To prevent things like slash codes from being interpreted by the shell/command line and programs you would use single quotes instead of doble quotes. Somehow I doubt if that is really what you want. Normally a key file will contain embedded newlines at those locations. Why does the interpretation of the newline concern you?

vikas027 · 05-12-2018, 08:25 AM

Quote:

Originally Posted by MadeInGermany

I think the two characters \ and n should be kept here. But a RE interprets it.
How about bash (as the title already suggests)?

Code:

str="--BEGIN PRIVATE KEY--\nAAA\nBBB\nCCC\n\n--END PRIVATE KEY--"
while IFS= read -r line
do
  case $line in
  (CLIENT_KEY*) line=$str;;
  esac
  printf "%s\n" "$line"
done <myfile

Excellent, this solves my problem. Thanks much MadeInGermany.

Thanks everyone for your time.

MadeInGermany · 05-12-2018, 05:19 PM

Quote:

Originally Posted by wpeckham

To prevent things like slash codes from being interpreted by the shell/command line and programs you would use single quotes instead of doble quotes...

You mean

Code:

str='...'

Agreed.
Indeed within " " the shell does special treatment of $ and " and ` characters (and allows \ escape of them, therefore the \ character is a bit special, too).
BTW once stored in a variable, the shell does not do further special treatment of "$str"

vikas027 · 05-30-2018, 08:47 PM

I have also found a neat python solution for the same

Code:

f = open(file, 'r')
print f.read().replace('\n', '\\n')[:-1]