LinuxQuestions.org - [SOLVED] Use same input string twice with sed?

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Use same input string twice with sed? (https://www.linuxquestions.org/questions/linux-newbie-8/use-same-input-string-twice-with-sed-4175430693/)

Use same input string twice with sed?

I have been working with sed lately and I have for the first time come unstuck I am trying to update two different files in two different directories with two different outputs and only one input:

Here is as far as I have got (I know it can be shorter but I am trying to get it working first):

Code:

read -p "Please Enter your VC1 codec license key :" vc1

sed -i "s|#decode_WVC1=.*|decode_WVC1=$vc1|" /dir1/file.a ; sed -i 's|<setting id="sys.config.decode.wvc1" value="" />|<setting id="sys.config.decode.wvc1" value="$vc1" />|' /dir2/file.b

OK It is updating the /dir1/file.a correctly with the inputed $vc1 and it is updating the /dir2/file.b, BUT instead of using the inputted $vc1 (as with file.a) it is actually giving me this as the output, IE: it isn't using the string again:

Code:

<setting id="sys.config.decode.wvc1" value="$vc1" />

Any help or pointers would be great.....

TIA

Mark

Hi.

Welcome to LQ!

The problem is in quotes. Consider this

Code:

$ var='hello'

$ echo "$var"

hello

$ echo '$var'

$var

Variables are not expanded within a string in single quotes. You must use double quotes here. To enter double quote as a character you should escape it with backslash: \".

So you end up with something like

Code:

sed -i "s|<setting id=\"sys\.config\.decode\.wvc1\" value=\"\" />|<setting id=\"sys\.config\.decode\.wvc1\" value=\"$vc1\" />|" /dir2/file.b

Note that I escaped dots as well, because as such a dot in regular expression means any character.

Hi wow, that was quick and more importantly THAT WAS CORRECT!!
I never realised about the '' "" & \ I just assumed if I had "" as a character then I had to enclose in ''
Thankyou so much and thanks for the welcome too, TBH I have always used this site as a lurker, it's a great site full of peeps doing great work!
Cheers,
Mark

If this is for a script, I want to suggest that you might find it slightly easier to use ed. It's an actual text editor and often more flexible in this kind of situation.

Assuming I'm decoding the desired actions correctly:

Code:

ed -s /dir1/file.a <<HEREDOC

/#decode_WVC1=.*/ s//#decode_WVC1=$vc1/

w

HEREDOC



ed -s /dir2/file.b <<HEREDOC

/decode[.]wvc1/ s/value=""/value="$vc1"/

w

HEREDOC

The above also assumes there's only one line to match in each file. If there could be more, you'll have to use a global matching prefix (just stick a "g" in front of the address field).

Notice too how the heredoc format makes quoting the expressions unnecessary.

To be fair though, ed's syntax can sometimes be a bit trickier to use (the "s" command can only use "/" as the delimiter, for example), and the regex ability is a bit more primitive, so you may be equal either way.

How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)

(Actually, if this is xml/html, as it appears to be, you should probably really be using a tool with a dedicated parser for those formats, but I'm not going to get into that just now.)

Many thanks for the heads up I had never even heard of ed, I will check it out! the problem I have is I am working with multiplatform OS's some are maxed out with commands but some have a very customised shell so not all commands are present, I chose sed as it was present on them all, its certainly been a learning curve :)

OK I have another question, but wasn't sure should I start a new thread as it is again related to sed, please correct me if I am wrong to post here?

I need to alter a line that contains multiple strings 10+, sometime all of those strings will be present and sometimes none will be present so I have no constant knowledge of what to find at the minute I am repeating the same sed command about 10 times to solve my problem, I was wondering if anyone could help with a one liner...

line to edited:

Code:

"$string1$string2$string3$string4$string5" >/

now when strings are added they automatically add a ';' in between each string like so:

Code:

"thisis1;thisis2;thisis3;" >/etc,etc

but certain parts of the program (but not all) when an option is skipped will automatically add a ';' at the end of the line and as such I can (but not always) end up with:

Code:

"thisis1;thisis2;thisis3;;;;;;;;;;;;;;;;;;;;;;" /> the ammount varies

at the minute I am using this command multiple times to cover myself & solve my problem, obviously it could be being run far to many times so isn't really pratical:

Code:

sed -i "s|\;\"|\"|" > /path_to_file_to_update

I need to make sure that no matter what the variable of the strings are that the line ends with " and NOT ;" and the overall line ending has to " >/

TIA

Mark

Hi. Have a read on & with sed. It might help.

Quote:

Originally Posted by N0hAT (Post 4800177)

at the minute I am using this command multiple times to cover myself & solve my problem, obviously it could be being run far to many times so isn't really pratical:

Code:

sed -i "s|\;\"|\"|" > /path_to_file_to_update

I need to make sure that no matter what the variable of the strings are that the line ends with " and NOT ;" and the overall line ending has to " >/

TIA

Mark

Instead of using &, you could use:

Code:

sed 's/;\+"/"/' file

The solution to the new problem isn't really about the tool used, but the regex applied. Since the ";" is a literal character that matches only once, you need to run the thing over and over. What you need is to match a string of identical characters, and for that you use the "+" operator (match the previous character one or more times).

Code:

sed -r -i -e '/thisis/ s|;+" />$|" />|' file.txt



ed -s file.txt <<HEREDOC

g/thisis/ s/;\+" \/>$/" \/>/

w

HEREDOC

Here you can see an example of a situation in which ed can be trickier to use than sed. We have to backslash several characters to do it, due to the lack of a configurable delimiter and default extended regex ability.

In sed, I used the -r option to avoid having to backslash the "+" sign, but this option isn't available in ed. See the regex sections of both programs for more.

On the other hand, both ed and sed can use multiple expressions at once. sed with the -e option, and ed just by putting it into the script. So you can combine this with your other edits if you want and avoid some more extra processes.

In both cases I used "thisis" as an address string to target the line. You'll probably want to change it to something more exacting. I recommend always using an address field like this if you can, BTW. It helps to avoid errors and makes it more efficient by skipping the processing of unnecessary lines. And I made the substitution expression a bit longer, again to avoid any possible false positives. Remember too that sed works globally by default, but ed doesn't.

Also notice how single quotes escape double quotes, and vice-versa. This helps make it easier when you need to include one kind inside the text itself. Just remember that double quotes are needed in order to expand parameters and command substitutions (anything that begins with "$", basically).

Here's some vital info on arguments and quoting:
http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

Finally, it definitely pays off in the long run to really familiarize yourself with regular expressions. It's about the most useful single tool you can study. Get yourself a good tutorial and work through it.

Here are a few regular expressions resources:
http://mywiki.wooledge.org/RegularExpression
http://www.grymoire.com/Unix/Regular.html
http://www.regular-expressions.info/

Many thanks to everyone in this thread I have added it my Favourites as a brilliant reference page, what with the info and external links!!

I ended up using sycamorex's one liner with the 'in place' suffix, so, so simple yet I can't believe it replaced lots and lots of pointless code and saved valuable resources especially as it all being run on a Raspberry...

Again many thanks to everyone in here, once I suss how to give everyone brownie points I will don't worry! :)

It appears I can not use the reputation system just yet, oh well I'm sure I will be posting more often, so will eventually get the chance to repay you all with points :)

Happy to hear it helped out.

I think you need to enable the reputation system in your profile settings. But as my signature says, you can also mark posts a helpful.