LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-24-2020, 08:31 AM   #16
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85

Well, awk doesn't seem to be any smarter, to be honest. Or am I mistaken?

Code:
root@kube:~# cat file.txt
$
root@kube:~# echo $var
$
root@kube:~# echo $var2
crap2
root@kube:~# awk -v var="${var}" -v var2="${var2}" '{sub(var, var2)}1' file.txt
$crap2
So instead of replacing $ with crap2, it just suffixes, for some reason.
Of course, a normal string substitution works fine:
Code:
root@kube:~# cat file.txt
whatever
root@kube:~# var=whatever
root@kube:~# echo $var2
crap2
root@kube:~# awk -v var="${var}" -v var2="${var2}" '{sub(var, var2)}1' file.txt
crap2
 
Old 05-24-2020, 08:59 AM   #17
shruggy
Member
 
Registered: Mar 2020
Posts: 994

Rep: Reputation: Disabled
Quote:
Originally Posted by vincix View Post
So instead of replacing $ with crap2, it just suffixes, for some reason.
For quite an obvious reason, actually. The first argument to sub/gsub is considered to be a regular expression. Putting it in a variable changes nothing in this respect.

Last edited by shruggy; 05-24-2020 at 09:14 AM.
 
Old 05-24-2020, 09:03 AM   #18
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,663

Rep: Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777
you may need to export variables, otherwise (as in your example) that will be again evaluated by the shell.
sub (in awk) works on regexp, that's why it replaced the end of the line (when var=$)
If you wish to use simple substitution/replacement you need to find another solution.
Code:
awk -v a="${var}" -v b="${var2}" {
  while(i=index($0,a))
    $0=substr($0,1,i-1) b substr($0,i+length(a))
  print $0
}
 
1 members found this post helpful.
Old 05-24-2020, 09:06 AM   #19
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
Right, that's correct, this is unrelated to any interpretation of the bash variables.
What about this? Now I'm curious, while we're at it:
Code:
root@kube:~# echo $var
$!:.%^&
root@kube:~# echo $var2
crap2
root@kube:~# awk -v var="${var}" -v var2="${var2}" '{sub(var2, var)}1' file.txt
$!:.%^crap2
Why is awk interpreting & that way? I guess it's like in sed, & means the whole matched regex, right?

EDIT: I've posted it without seeing pan64's latest reply.

Last edited by vincix; 05-24-2020 at 09:09 AM.
 
Old 05-24-2020, 09:13 AM   #20
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,663

Rep: Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777
which awk is it exactly? what is in file.txt?
 
Old 05-24-2020, 09:18 AM   #21
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
It's gawk (comes with most Linux distros - here running under Ubuntu 18.04)
In file.txt there's only "crap2".
I couldn't get your command to work, I'm not sure why. I get this:
Code:
awk: cmd. line:1: {
awk: cmd. line:1:  ^ unexpected newline or end of string
awk.sh: line 2: syntax error near unexpected token `('
awk.sh: line 2: `  while(i=index($0,a))'
 
Old 05-24-2020, 09:23 AM   #22
shruggy
Member
 
Registered: Mar 2020
Posts: 994

Rep: Reputation: Disabled
Quote:
Originally Posted by vincix View Post
Why is awk interpreting & that way? I guess it's like in sed, & means the whole matched regex, right?
Exactly. There's a pretty long treatment of it in the Gawk Manual.

Last edited by shruggy; 05-24-2020 at 09:26 AM.
 
Old 05-24-2020, 09:24 AM   #23
shruggy
Member
 
Registered: Mar 2020
Posts: 994

Rep: Reputation: Disabled
Quote:
Originally Posted by vincix View Post
I couldn't get your command to work, I'm not sure why.
The whole awk expression should be quoted:
Code:
awk -v a="${var}" -v b="${var2}" '{
  while(i=index($0,a))
    $0=substr($0,1,i-1) b substr($0,i+length(a))
  print $0
}'

Last edited by shruggy; 05-24-2020 at 09:25 AM.
 
1 members found this post helpful.
Old 05-24-2020, 09:28 AM   #24
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
It doesn't work either way Adding double quotes shouldn't work because they are already being used, and it might screw up the interpretation. Single quotes don't work anyway. I placed the whole command in a file, and it's not working.
 
Old 05-24-2020, 09:35 AM   #25
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,663

Rep: Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777
Quote:
Originally Posted by vincix View Post
It doesn't work either way Adding double quotes shouldn't work because they are already being used, and it might screw up the interpretation. Single quotes don't work anyway. I placed the whole command in a file, and it's not working.
so check it again. The last post from shruggy looks correct, I missed the two ' .
 
Old 05-24-2020, 09:40 AM   #26
shruggy
Member
 
Registered: Mar 2020
Posts: 994

Rep: Reputation: Disabled
I've just tried it out, it works:
Code:
$ cat file.txt
crap2
$ awk -v b='$!:.%^&' -v a='crap2' -f pan64.awk file.txt
$!:.%^&

Last edited by shruggy; 05-24-2020 at 09:50 AM.
 
Old 05-24-2020, 09:45 AM   #27
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
Yes, I missed that. It does work as expected, indeed!
Code:
root@kube:~# cat file.txt
crap2
root@kube:~# echo $var
crap2
root@kube:~# echo $var2
$!:.%^&
awk -v a="${var}" -v b="${var2}" '{
  while(i=index($0,a))
    $0=substr($0,1,i-1) b substr($0,i+length(a))
  print $0
}' file.txt
$!:.%^&
So that would be a completely literal substitution, and I wouldn't care about what characters are found in the variable.

But if I wanted to use regex for matching the pattern and then substitute it with a literal string, like the sub command does, then all of a sudden I have a problem, in that I yet again have to be careful about what the variable contains, right?
 
Old 05-24-2020, 09:54 AM   #28
shruggy
Member
 
Registered: Mar 2020
Posts: 994

Rep: Reputation: Disabled
Yes, and you should be aware of how regex in a string differs from a regex literal between slashes.

Last edited by shruggy; 05-24-2020 at 09:59 AM.
 
Old 05-24-2020, 10:01 AM   #29
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,663

Rep: Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777
and additionally you may also kill the awk process (with an invalid regexp).
 
1 members found this post helpful.
Old 05-24-2020, 10:02 AM   #30
vincix
Senior Member
 
Registered: Feb 2011
Distribution: Ubuntu, Centos
Posts: 1,149

Original Poster
Rep: Reputation: 85
Right, but then I return to the initial problem related to sed and the interpretation of bash variables and I see that awk is susceptible to very similar issues, regardless of when or by what the variable is converted into its value.

Maybe perl would do the job, but I don't see lots of people writing perl substitutions in their docker entrypoints, to be honest. Installing python to create entrypoint containers could be ok, depending on the situation, but normally you don't want your image to grow by hundreds of mb just for a substitution.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Difference between echo of file within double quotes and without double quotes ankitpandey Programming 2 01-11-2013 09:02 AM
translate value from single quotes to double quotes venkateshrupineni Linux - Newbie 2 06-14-2012 03:03 PM
Double Quotes Inside Double Quotes youarefunny Programming 6 06-09-2010 10:21 PM
Problems with quotes and double quotes Andruha Slackware 6 01-02-2010 04:44 PM
Using single quotes vs double quotes in PHP strings vharishankar Programming 6 07-11-2005 11:41 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 04:53 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration