sed replace first occurrence but do subsequent

docker_it_is · 10-01-2018, 12:56 PM

Hello, I am a newbie to Shell scripting so please forgive me for any nonsense things

My requirement:

Input file (Docker compose - yml format) has following:
version: "3.2"
services:
eurekaA:
image: dtr.abc.com/11658-us-eos/registry-service-qa:develop-4
eurekaB:
image: dtr.abc.com/11658-us-eos/registry-service-qa:develop-4
configserver:
image: dtr.abc.com/11658-us-eos/config-service-qa:develop-69
gatewayservice:
image: dtr.abc.com/11658-us-eos/gateway-service-qa:develop-26
groupcustomerservice:
image: dtr.abc.com/11658-us-eos/grpcust-service-qa:develop-149

Need to Retag all images with appending "-promote" keyword and I have following script working well until encountering a use case of repeated image name in the input file "registry-service"

Code:
========================================================================

Code:

images_to_promote=(`grep image docker-compose.yml | cut -d":" -f2,3`)

#Loop through the array and execute SED to search and replace

for i in "${images_to_promote[@]}"
        do
                new_Img_name="$i-promote"
                echo "Updating $i ---> $new_Img_name"
                eval sed -i 's#$i#$new_Img_name#g' docker-compose.yml
        done

========================================================================

The Output expected is :
dtr.abc.com/11658-us-eos/registry-service-qa:develop-4-promote
dtr.abc.com/11658-us-eos/registry-service-qa:develop-4-promote
dtr.abc.com/11658-us-eos/config-service-qa:develop-69-promote
dtr.abc.com/11658-us-eos/gateway-service-qa:develop-26-promote
dtr.abc.com/11658-us-eos/groupcustomer-service-qa:develop-149-promote

But instead I get:
dtr.abc.com/11658-us-eos/registry-service-qa:develop-4-promote-promote
dtr.abc.com/11658-us-eos/registry-service-qa:develop-4-promote-promote
dtr.abc.com/11658-us-eos/config-service-qa:develop-69-promote
dtr.abc.com/11658-us-eos/gateway-service-qa:develop-26-promote
dtr.abc.com/11658-us-eos/groupcustomer-service-qa:develop-149-promote

I have tried exact search using \< \>
I have tried getting rid of \g to limit search for only first occurrence

But none of these are helping.

Could someone please provide me direction to achieve this ?

scasey · 10-01-2018, 02:05 PM

Please use code tags when posting commands, scripts or outputs.

I suspect the problem is with value in the images-to-promote variable, not with sed.
Put a set -x at the beginning of your script to see the actual values being processed.

Because you have duplicate file names in your input, those two names are both processed each time, resulting in the duplicated update.

Modify images_to_promote to remove duplicates:

Code:

images_to_promote=(`grep image lqtest | sort -u | cut -d":" -f2,3`)

docker_it_is · 10-01-2018, 02:25 PM

Quote:

Originally Posted by scasey

Please use code tags when posting commands, scripts or outputs.

I suspect the problem is with value in the images-to-promote variable, not with sed.
Put a set -x at the beginning of your script to see the actual values being processed.

Because you have duplicate file names in your input, those two names are both processed each time, resulting in the duplicated update.

Modify images_to_promote to remove duplicates:

Code:

images_to_promote=(`grep image lqtest | sort -u | cut -d":" -f2,3`)

Just what I needed (A different perspective to look at the problem)
THANKS Sean for this and yes this has resolved the problem !

l0f4r0 · 10-02-2018, 03:14 AM

Out of curiosity, are you doing other things with your var images_to_promote or inside your for loop?
Because IMHO your script, as it is, seems pretty complicated for what it does. Maybe I do not have all the context, but why not use the following?

Code:

sed -n -r '/image/ s/develop-[0-9]+/&-promote/p' docker-compose.yml

docker_it_is · 10-02-2018, 12:47 PM

Quote:

Originally Posted by l0f4r0

Out of curiosity, are you doing other things with your var images_to_promote or inside your for loop?
Because IMHO your script, as it is, seems pretty complicated for what it does. Maybe I do not have all the context, but why not use the following?

Code:

sed -n -r '/image/ s/develop-[0-9]+/&-promote/p' docker-compose.yml

Yes, I have multiple other things to do in that loop but this is great too and in fact with your help I have modified and separated the loop to do rest of the stuff and achieved the desired result using following one line sed:

Code:

sed -i '/image:/s/$/-Promote/' docker-compose.yml

Yet another perspective

A Big thanks to you too!

l0f4r0 · 10-03-2018, 02:30 AM

Perfect then, everybody is happy

Maybe you can mark your thread as resolved?

MadeInGermany · 10-03-2018, 07:21 AM

With shell builtins it's efficient and flexible

Code:

fn=docker-compose.yml
#loop thru the lines
while IFS=":" read w1 i
do
  # branch on keyword
  case $w1 in
  (image)
    new_Img_name="$i-promote"
    echo "Updating $i ---> $new_Img_name"
    echo "$new_Img_name" >&3
  esac
done < $fn 3> $fn.new