[SOLVED] add literal string with simple and double quotes to a file

vincix · 05-24-2020, 08:31 AM

Well, awk doesn't seem to be any smarter, to be honest. Or am I mistaken?

Code:

root@kube:~# cat file.txt
$
root@kube:~# echo $var
$
root@kube:~# echo $var2
crap2
root@kube:~# awk -v var="${var}" -v var2="${var2}" '{sub(var, var2)}1' file.txt
$crap2

So instead of replacing $ with crap2, it just suffixes, for some reason.
Of course, a normal string substitution works fine:

Code:

root@kube:~# cat file.txt
whatever
root@kube:~# var=whatever
root@kube:~# echo $var2
crap2
root@kube:~# awk -v var="${var}" -v var2="${var2}" '{sub(var, var2)}1' file.txt
crap2

shruggy · 05-24-2020, 08:59 AM

Quote:

Originally Posted by vincix

So instead of replacing $ with crap2, it just suffixes, for some reason.

For quite an obvious reason, actually. The first argument to sub/gsub is considered to be a regular expression. Putting it in a variable changes nothing in this respect.

pan64 · 05-24-2020, 09:03 AM

you may need to export variables, otherwise (as in your example) that will be again evaluated by the shell.
sub (in awk) works on regexp, that's why it replaced the end of the line (when var=$)
If you wish to use simple substitution/replacement you need to find another solution.

Code:

awk -v a="${var}" -v b="${var2}" {
  while(i=index($0,a))
    $0=substr($0,1,i-1) b substr($0,i+length(a))
  print $0
}

vincix · 05-24-2020, 09:06 AM

Right, that's correct, this is unrelated to any interpretation of the bash variables.
What about this? Now I'm curious, while we're at it:

Code:

root@kube:~# echo $var
$!:.%^&
root@kube:~# echo $var2
crap2
root@kube:~# awk -v var="${var}" -v var2="${var2}" '{sub(var2, var)}1' file.txt
$!:.%^crap2

Why is awk interpreting & that way? I guess it's like in sed, & means the whole matched regex, right?

EDIT: I've posted it without seeing pan64's latest reply.

pan64 · 05-24-2020, 09:13 AM

which awk is it exactly? what is in file.txt?

vincix · 05-24-2020, 09:18 AM

It's gawk (comes with most Linux distros - here running under Ubuntu 18.04)
In file.txt there's only "crap2".
I couldn't get your command to work, I'm not sure why. I get this:

Code:

awk: cmd. line:1: {
awk: cmd. line:1:  ^ unexpected newline or end of string
awk.sh: line 2: syntax error near unexpected token `('
awk.sh: line 2: `  while(i=index($0,a))'

shruggy · 05-24-2020, 09:23 AM

Quote:

Originally Posted by vincix

Why is awk interpreting & that way? I guess it's like in sed, & means the whole matched regex, right?

Exactly. There's a pretty long treatment of it in the Gawk Manual.

shruggy · 05-24-2020, 09:24 AM

Quote:

Originally Posted by vincix

I couldn't get your command to work, I'm not sure why.

The whole awk expression should be quoted:

Code:

awk -v a="${var}" -v b="${var2}" '{
  while(i=index($0,a))
    $0=substr($0,1,i-1) b substr($0,i+length(a))
  print $0
}'

vincix · 05-24-2020, 09:28 AM

It doesn't work either way

Adding double quotes shouldn't work because they are already being used, and it might screw up the interpretation. Single quotes don't work anyway. I placed the whole command in a file, and it's not working.

pan64 · 05-24-2020, 09:35 AM

Quote:

Originally Posted by vincix

It doesn't work either way

Adding double quotes shouldn't work because they are already being used, and it might screw up the interpretation. Single quotes don't work anyway. I placed the whole command in a file, and it's not working.

so check it again. The last post from shruggy looks correct, I missed the two ' .

shruggy · 05-24-2020, 09:40 AM

I've just tried it out, it works:

Code:

$ cat file.txt
crap2
$ awk -v b='$!:.%^&' -v a='crap2' -f pan64.awk file.txt
$!:.%^&

vincix · 05-24-2020, 09:45 AM

Yes, I missed that. It does work as expected, indeed!

Code:

root@kube:~# cat file.txt
crap2
root@kube:~# echo $var
crap2
root@kube:~# echo $var2
$!:.%^&
awk -v a="${var}" -v b="${var2}" '{
  while(i=index($0,a))
    $0=substr($0,1,i-1) b substr($0,i+length(a))
  print $0
}' file.txt
$!:.%^&

So that would be a completely literal substitution, and I wouldn't care about what characters are found in the variable.

But if I wanted to use regex for matching the pattern and then substitute it with a literal string, like the sub command does, then all of a sudden I have a problem, in that I yet again have to be careful about what the variable contains, right?

shruggy · 05-24-2020, 09:54 AM

Yes, and you should be aware of how regex in a string differs from a regex literal between slashes.

pan64 · 05-24-2020, 10:01 AM

and additionally you may also kill the awk process (with an invalid regexp).

vincix · 05-24-2020, 10:02 AM

Right, but then I return to the initial problem related to sed and the interpretation of bash variables and I see that awk is susceptible to very similar issues, regardless of when or by what the variable is converted into its value.

Maybe perl would do the job, but I don't see lots of people writing perl substitutions in their docker entrypoints, to be honest. Installing python to create entrypoint containers could be ok, depending on the situation, but normally you don't want your image to grow by hundreds of mb just for a substitution.