LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-20-2020, 04:48 AM   #1
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Rep: Reputation: 85
add literal string with simple and double quotes to a file


I'm trying to add the following string to a file:
Code:
sed 's|\$MYSQL_PASSWORD|'"$MYSQL_PASSWORD"'|'
I'm using the simple and double quote syntax so that sed would use the MYSQL_PASSWORD variable (in the second part; the first part is literally $MYSQL_PASSWORD) when sed is run in the file I'm trying to add the line to.

What I've tried to far is:
Code:
echo sed 's|\$MYSQL_PASSWORD|'"\$MYSQL_PASSWORD"'|' > file
But unfortunately it drops the quote altogether:
Code:
sed s|\$MYSQL_PASSWORD|$MYSQL_PASSWORD|
Somewhat closer is:
Code:
echo sed 's|\$MYSQL_PASSWORD|'"'"\$MYSQL_PASSWORD"'"'|'
But this drops the double quotes:
Code:
sed s|\$MYSQL_PASSWORD|'$MYSQL_PASSWORD'|
This is becoming a little bit retarded, so if there are more elegant solutions, I'm looking forward to it

Ok, I was actually able to find a horrible solution:
Code:
echo sed \''s|\$MYSQL_PASSWORD|'"'\""\$MYSQL_PASSWORD"'"\"'|'\'
But the question remains: is there any other way to achieve this?
Thanks!

Last edited by vincix; 05-20-2020 at 04:57 AM.
 
Old 05-20-2020, 05:06 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,618

Rep: Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755
that can be quite tricky. I don't know how did you check if the result is ok
Code:
TEXT="sed 's|\$MYSQL_PASSWORD|${MYSQL_PASSWORD}|'"
echo ">${TEXT}<"
would be probably the first step.
If that works you can safely redirect it into file too.
 
Old 05-20-2020, 05:21 AM   #3
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
Actually I made a mistake in the final result. It should have been:
Code:
echo sed \''s|\$MYSQL_PASSWORD|'"'\""\$MYSQL_PASSWORD\""'"'|'\'
I had misplaced the \ " (if I write \ " together, then the backslash is removed from the post, because it's interpreted as an escape character! )
I tested it by simply redirecting it into the file and it worked as expected:
Code:
root@oms:~# echo sed \''s|\$MYSQL_PASSWORD|'"'\""\$MYSQL_PASSWORD\""'"'|'\' > file
root@oms:~# cat file
sed 's|\$MYSQL_PASSWORD|'"$MYSQL_PASSWORD"'|'

The problem with this solution
Code:
TEXT="sed 's|\$MYSQL_PASSWORD|${MYSQL_PASSWORD}|'"
is that sed interprets the ${MYSQL_PASSWORD} literally, because of the simple quotes. This is why I was trying to use '"$MYSQL_PASSWORD"' (so between simple + double quotes; so I'm escaping the bash variable that way and sed will replace it with the value of the variable, instead of the literal name).

Can you explain what the meaning of the angle brackets in echo ">${TEXT}<" is?

Last edited by vincix; 05-20-2020 at 05:24 AM.
 
Old 05-20-2020, 05:37 AM   #4
shruggy
Member
 
Registered: Mar 2020
Posts: 899

Rep: Reputation: Disabled
Your first version should work with a HERE document:
Code:
cat <<EOF >file
sed 's|\$MYSQL_PASSWORD|'"\$MYSQL_PASSWORD"'|'
EOF
 
Old 05-20-2020, 05:57 AM   #5
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,571

Rep: Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874
Quote:
Originally Posted by shruggy View Post
Your first version should work with a HERE document:
Code:
cat <<EOF >file
sed 's|\$MYSQL_PASSWORD|'"\$MYSQL_PASSWORD"'|'
EOF
You need to use
Code:
cat <<'EOF' > file
sed 's|\$MYSQL_PASSWORD|'"\$MYSQL_PASSWORD"'|'
EOF
Note the quotes around EOF.

https://www.gnu.org/software/bash/ma...Here-Documents
Quote:
If any part of word is quoted, the delimiter is the result of quote removal on word, and the lines in the here-document are not expanded. If word is unquoted, all lines of the here-document are subjected to parameter expansion
 
1 members found this post helpful.
Old 05-20-2020, 06:09 AM   #6
shruggy
Member
 
Registered: Mar 2020
Posts: 899

Rep: Reputation: Disabled
^ With quotes around EOF you don't need to escape $ in the replacement string:
Code:
cat <<'EOF' > file
sed 's|\$MYSQL_PASSWORD|'"$MYSQL_PASSWORD"'|'
EOF
 
1 members found this post helpful.
Old 05-20-2020, 06:13 AM   #7
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,618

Rep: Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755
So your problem is that the password itself contains something which can be interpreted by sed.
You cannot protect it that way there is no fool-proof solution. You need to process the password string itself to escape all the suspicious chars.

> and < are just markers to see what is "inside". Nothing else.
 
Old 05-20-2020, 06:49 AM   #8
shruggy
Member
 
Registered: Mar 2020
Posts: 899

Rep: Reputation: Disabled
I assume the file you're writing your sed expression to is a shell script. A reasonably recent bash should have a nice facility to cope with the problem pan64 describes in form of ${var@Q}. From the Bash Reference Manual:
Quote:
${parameter@operator}
The expansion is either a transformation of the value of parameter or information about parameter itself, depending on the value of operator. Each operator is a single letter:

Q
The expansion is a string that is the value of parameter quoted in a format that can be reused as input.
 
1 members found this post helpful.
Old 05-20-2020, 09:23 AM   #9
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,618

Rep: Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755
Quote:
Originally Posted by shruggy View Post
I assume the file you're writing your sed expression to is a shell script. A reasonably recent bash should have a nice facility to cope with the problem pan64 describes in form of ${var@Q}. From the Bash Reference Manual:
It is something new, somewhere bash 4.4 was implemented.
 
Old 05-24-2020, 07:04 AM   #10
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
I wanted to restrict the thread only to this particular problem without giving the bigger context, but now the context has become much more relevant.
The point was to do something somewhat intentionally dumb, that is to say, to change an existing docker-entrypoint from an official docker image (of php-apache) through an additional Dockerfile, where I'd run the sed commands.
The here document would have been a great solution under normal circumstances, and it hadn't occurred to me anyway, but unfortunately it doesn't work in Dockerfile

What I did eventually (which I find much more sensible) was to actually copy only the entrypoint from github, add the sed lines that would replace the php configuration settings, and then copy it into the new image, ensuring, of course, that it has execution permissions (being an entrypoint).

@pan64
The issue with this: TEXT="sed 's|\$MYSQL_PASSWORD|${MYSQL_PASSWORD}|'" is that sed is not going to interpret the bash variable as it is, regardless of what the variable contains. This is why it needs to be escaped with '"${MYSQL_PASSWORD}"'. What the actual value of the variable is, is indeed a problem in itself and needs to be dealt with carefully, I agree, sure. That's why I chose a longer string (length of 40-50 characters) but with standard characters, as it were.


Anyway, thanks for the help, I've come away with lots of interesting information!

Last edited by vincix; 05-24-2020 at 07:21 AM.
 
Old 05-24-2020, 07:57 AM   #11
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,618

Rep: Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755
sed will never be able to use bash variables, that is a different language.
https://stackoverflow.com/questions/...itution-in-sed
If you wish to do that you need to use perl/awk/python/... something, which can handle environment variables properly.
 
Old 05-24-2020, 08:00 AM   #12
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
Ok, so what about this?
Code:
root@kube:~# var=crap
root@kube:~# echo "stuff" > file.txt
root@kube:~# sed 's/stuff/'"$var"'/' file.txt
crap
 
Old 05-24-2020, 08:14 AM   #13
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,618

Rep: Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755
In that case the shell will evaluate $var and will pass the result to sed. As always, the command evaluated first (before execution). You can check it by executing set -x (before executing your sed)
Code:
$ set -x
$ var=cap
+ var=cap
$ echo "stuff" > file.txt
+ echo stuff
$ sed 's/stuff/'"$var"'/' file.txt
+ sed s/stuff/cap/ file.txt
cap
black was entered
blue was actually executed (after evaluation)
green is the result

Also [re]read the link I posted.
 
1 members found this post helpful.
Old 05-24-2020, 08:17 AM   #14
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
Ok, I see what you mean (I also hadn't read the link entirely before answering), but that works for all intents and purposes. And that's the most important thing, so that's more of a theoretical problem (yes, not necessarily, because it depends on what the variable contains, but I've already cleared that up). There's no point in switching to another tool when doing such substitutions - they are being used widely.

Last edited by vincix; 05-24-2020 at 08:22 AM.
 
Old 05-24-2020, 08:22 AM   #15
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,618

Rep: Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755Reputation: 4755
the problem is when the variable contains something which can be interpreted by sed. And (if the content of the variable is unpredictable) this substitution is just unreliable.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Difference between echo of file within double quotes and without double quotes ankitpandey Programming 2 01-11-2013 09:02 AM
translate value from single quotes to double quotes venkateshrupineni Linux - Newbie 2 06-14-2012 03:03 PM
Double Quotes Inside Double Quotes youarefunny Programming 6 06-09-2010 10:21 PM
Problems with quotes and double quotes Andruha Slackware 6 01-02-2010 04:44 PM
Using single quotes vs double quotes in PHP strings vharishankar Programming 6 07-11-2005 11:41 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:20 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration