sed command to change variable names globally

shridhar22 · 01-14-2013, 02:37 AM

I wish to replace following variable name like this:
hnlVerilogpattern == _hnlVlogCxtpattern
hnlVerilogpattern == _hnlVlogCxtpattern

these are just 2 examples, there are about 100 such variables that need to be changed like this i.e. change hnlVerilog to _hnlVlogCxt rest should be same.

For ease of use I have copied all the variables that need to be changed into 1 separate file. Now the sed script should
1.read each variable in the file and change it accordingly (globally)
NOTE: there are various occurance of thse variables in su-directories...that all need to be changed with a backup in place.

Code:

sed -i.bak 's/\<hnlVerilog/_hnlVlogCxt/g'

should be the way.... but then how do i tell sed that read from file (containing all variables). I hope im clear enough to all what i want
.

bijo505 · 01-14-2013, 06:58 AM

Hi shridhar22,

What you can do that, first find out the files which needs to be replaced.

Code:

egrep -R -l <pattern> *

So you will get all the files, which is having the pattern.
Then I created one file which is having all the variables that needs to be replaced. here it is sed.txt

Code:

$cat sed.txt
s/hnlVerilogpattern/_hnlVlogCxtpattern/g
s/bijo/mano/g
s/bino/ram/g
s/sam/Thomas/g

$cat 1.txt # This file should be updated
hnlVerilogpattern ==
hnlVerilogpattern ==
hnlVerilogpattern ==
hnlVerilogpattern ==
hnlVerilogpattern ==
hnlVerilogpattern ==
bijo
sam
bino

Then you can run the following to change the value of files under the subdirectory too

Code:

egrep -R -l hnlVerilogpattern * | egrep -v 'sed.txt' | xargs -L 1 -I {} sed -i .bak -f sed.txt {};

$cat 1.txt
_hnlVlogCxtpattern ==
_hnlVlogCxtpattern ==
_hnlVlogCxtpattern ==
_hnlVlogCxtpattern ==
_hnlVlogCxtpattern ==
_hnlVlogCxtpattern ==
mano
Thomas
ram

PS:- After run the command, you have to check the files. If you re run the command, your backup files will be replace, so please take a note on this.

shridhar22 · 01-14-2013, 08:02 AM

Hi bijo505, thanks for the reply.
Actually the number of variables that need change are approx 100

So writing s/old/new/g for all variables is tedious and may be mistake prone because i have variables name like hnlVerilogabcdecghiblablabla and doing this doesnt help my cause. Because what i tried before was
foreach file(*/*)
sed -i.bak 's/old/new/g' and so on for all variables.
Is there some other way? hnlVerilogSomeThing is common in all variables that need to change to hnlVlogSomeThing globally.
Thanks

TB0ne · 01-14-2013, 08:48 AM

Quote:

Originally Posted by shridhar22

Hi bijo505, thanks for the reply.
Actually the number of variables that need change are approx 100

So writing s/old/new/g for all variables is tedious and may be mistake prone because i have variables name like hnlVerilogabcdecghiblablabla and doing this doesnt help my cause. Because what i tried before was
foreach file(*/*)
sed -i.bak 's/old/new/g' and so on for all variables.
Is there some other way? hnlVerilogSomeThing is common in all variables that need to change to hnlVlogSomeThing globally.
Thanks

A bit confused by what you've posted. Based on this, you say that you want to replace "hnlVeriLog" with "hnlVlog"...should be simple, and replacing the part of a string won't change the other parts.

Writing a simple script file to look in all the subdirectories for files that match a pattern, then running a replace on them isn't hard. What have you done/tried so far to accomplish this? And can you post a few lines of the sample input data, and sample output data, EXACTLY as they look/you want them to look??

shridhar22 · 01-14-2013, 09:18 AM

@TBOne

Quote:

hnlVerilogpattern changes to _hnlVlogCxtpattern
hnlVerilogpattern changes to _hnlVlogCxtpattern

and so on
I used

Quote:

sed -i.bak 's/\<hnlVerilog/_hnlVlogCxt/g'

but this works for one file i.e. I open file in vim type : and the command
I dont want to open each file and do it everywhere, SO THEREFORE I HAVE COPIED ALL VARIABLES THAT NEED TO BE CHANGED IN A FILE. Now how do i apply such substitution at every occurrence of such variables in subdirectories.

File1 content:
hnlVerilogpattern #change globally to hnlVlogCxtpattern in every subdirectory file that has this variable
hnlVerilogblabla
hnlVerilogsomething

NOw i want sed to read each of these variables in file1 and apply sed 's/firstVariableReadFromFile1/whatIwANT/g' for all variables in file1
I hope I'm clear enough. Thanks

allend · 01-14-2013, 10:36 AM

Quote:

hnlVerilogSomeThing is common in all variables that need to change to hnlVlogSomeThing globally.

Is this correct? If so, as TB0ne said, sed will only substitute the part of the string that you specify.

Quote:

NOTE: there are various occurance of thse variables in su-directories...that all need to be changed with a backup in place.

Those typos do not help in understanding your requirement.

Quote:

Now how do i apply such substitution at every occurrence of such variables in subdirectories.

So why would this not work when run from the top directory of the tree you want to change?

Code:

find . -type f -exec sed -i.bak 's:hnlVerilog:hnlVlog:g' '{}' \;

shridhar22 · 01-14-2013, 11:16 AM

Guys sorry for making troubles, plz bear with me. Let me try again, What I only want is
------------------------------------------------------------------------------------
Scenerio:I am at root
there are lot of files (in SUB-DIRECTORIES) having variables like
hnlVerilogA
hnlVerilogB
hnlVerilogC

Out of these only some variables had to changed, suppose only hnlVerilogA and hnlVerilogC.
hnlVerilogB SHOULD NOT CHANGE
-----------------------------------------------------------------------------------
So i copied all those that needed to be changed into 1 FILE i.e.
hnlVerilogA
hnlVerilogC

Now i want sed to read this FILE and do following changes

change hnlVerilogA to _hnlVlogCxtA AND hnlVerilogC to _hnlVlogCxtC

NOTE :hnlVerilogB doesnt change because it was not in FILE

Algo: foreach variable in FILE ,change it to my requirement

TB0ne · 01-14-2013, 11:22 AM

Quote:

Originally Posted by shridhar22

Guys sorry for making troubles, plz bear with me. Let me try again, What I only want is
------------------------------------------------------------------------------------
Scenerio:I am at root
there are lot of files (in SUB-DIRECTORIES) having variables like
hnlVerilogA
hnlVerilogB
hnlVerilogC

Out of these only some variables had to changed, suppose only hnlVerilogA and hnlVerilogC.
hnlVerilogB SHOULD NOT CHANGE
-----------------------------------------------------------------------------------
So i copied all those that needed to be changed into 1 FILE i.e.
hnlVerilogA
hnlVerilogC

Now i want sed to read this FILE and do following changes

change hnlVerilogA to _hnlVlogCxtA AND hnlVerilogC to _hnlVlogCxtC

NOTE :hnlVerilogB doesnt change because it was not in FILE

Algo: foreach variable in FILE ,change it to my requirement

OK...so again, it seems like you only want to change a part of the string. And again, all you're needing is a she'll script to loop through files. So show us what you've done so far

Otherwise, post real sample data as requested.

shridhar22 · 01-14-2013, 11:43 AM

Code:

shridhar@ubuntu:~$ for file in hnlVerilogPattern hnlVerilogAnother hnlVerilogOnemore
> do
> sed -i.bak 's/"$file"/_"$file"Cxt/g'
> done
sed: no input files
sed: no input files
sed: no input files

but see how stupid this code is
I'm not able to make it understand that foreach variable make change. I couldnt do it with making sed read a file that has all variables that need change. so i have tried giving it in for loop but still not working.
TBone i hope im posting better now?

OMG... how do i do this...

TB0ne · 01-14-2013, 02:02 PM

Quote:

Originally Posted by shridhar22

Code:

shridhar@ubuntu:~$ for file in hnlVerilogPattern hnlVerilogAnother hnlVerilogOnemore
 do
 sed -i.bak 's/"$file"/_"$file"Cxt/g'
 done

but see how stupid this code is I'm not able to make it understand that foreach variable make change. I couldnt do it with making sed read a file that has all variables that need change. so i have tried giving it in for loop but still not working.
TBone i hope im posting better now?

Yes, but do you see how we all got confused?? You want to replace ONE static string for another in DIFFERENT FILES...which is different from replacing a dynamic string with another dynamic string...which is what you posted.

Also, you STILL have not posted any sample data, although it's been requested several times.

Quote:

OMG... how do i do this...

You read one of the MANY easily-found bash scripting tutorials, and see the examples about how to loop through a file and run command(s). You also need to think about what you're DOING with the files. In your example above, you're telling it to look in three different files, then replace a string that doesn't match what you were saying above...which is why we've repeatedly asked for SAMPLES OF YOUR DATA.

Code:

for file in /some/directory/*
do
    sed -i.bak 's/\<hnlVerilog/_hnlVlogCxt/g'
done

This looks in /some/directory, and reads ALL the files in it. For each file, it runs the sed command.

allend · 01-14-2013, 09:33 PM

I set up these files to be changed

Code:

bash-4.2$ cat junk/t3.txt
hnlVerilogA
hnlVerilogB
hnlVerilogC

bash-4.2$ cat junk/junk1/t4.txt
hnlVerilogA
hnlVerilogB
hnlVerilogC

Then I created FILE1.txt

Code:

bash-4.2$ cat FILE1.txt 
hnlVerilogA
hnlVerilogC

and a short bash script t.sh

Code:

bash-4.2$ cat t.sh
#!/bin/bash

file=$1

while read ; do
  sed -i.bak s/"$REPLY"/_hnlVlogCxt"${REPLY#hnlVerilog}"/g "$file";
done < FILE1.txt

I then executed

Code:

find ./junk -type f -exec t.sh '{}' \;

with the following results

Code:

bash-4.2$ cat junk/t3.txt
_hnlVlogCxtA
hnlVerilogB
_hnlVlogCxtC

bash-4.2$ cat junk/t3.txt.bak
_hnlVlogCxtA
hnlVerilogB
hnlVerilogC

bash-4.2$ cat junk/junk1/t4.txt
_hnlVlogCxtA
hnlVerilogB
_hnlVlogCxtC

bash-4.2$ cat junk/junk1/t4.txt.bak
_hnlVlogCxtA
hnlVerilogB
hnlVerilogC

Perhaps this is the sort of result that you desire.
Please note that the above does not retain backups of the original files, but only backups of the files prior to the last change.

celticdevildog · 01-14-2013, 10:45 PM

Quote:

Originally Posted by shridhar22

Code:

shridhar@ubuntu:~$ for file in hnlVerilogPattern hnlVerilogAnother hnlVerilogOnemore
> do
> sed -i.bak 's/"$file"/_"$file"Cxt/g'
> done
sed: no input files
sed: no input files
sed: no input files

but see how stupid this code is
I'm not able to make it understand that foreach variable make change. I couldnt do it with making sed read a file that has all variables that need change. so i have tried giving it in for loop but still not working.
TBone i hope im posting better now?

OMG... how do i do this...

If I understand you correctly, you have a bunch of files, and subdirecotries of files that all have a variable that needs to be changed.

How about something like this:

Code:

for file in `find . -type f` ; do
   sed -i.bak -e "s/\(hnlVerilogPattern\)/_\1Cxt/g" -e "s/\(hnlVerilogAnother\)/_\1Cxt/g" -e "s/\(hnlVerilogOnemore\)/_\1Cxt/g" $file
done

shridhar22 · 01-15-2013, 12:03 AM

@Allend, thanks for the try but your code does something like this , content of file that needed to be changed now look like

_hnlVlogCxt hnlVerilogCrossViewCheck
_hnlVlogCxt hnlVerilogGetInstCDFPropLenWidValue
_hnlVlogCxt hnlVerilogNetBitSelect
_hnlVlogCxt hnlVerilogCreateMSTestFixtureFile

But i wanted to replace hnlVerilogCrossViewCheck and change it with _hnlVlogCxtCrossViewCheck and so on.
Note: There did exist some trailing tabs/spaces before hnlVerilogCrossViewCheck

allend · 01-15-2013, 02:22 AM

I created files to be changed

Code:

bash-4.1$ cat junk/t3.txt
_hnlVlogCxt hnlVerilogCrossViewCheck
_hnlVlogCxt hnlVerilogGetInstCDFPropLenWidValue
_hnlVlogCxt hnlVerilogNetBitSelect
_hnlVlogCxt hnlVerilogCreateMSTestFixtureFile

bash-4.1$ cat junk/junk1/t4.txt
_hnlVlogCxt hnlVerilogCrossViewCheck
_hnlVlogCxt hnlVerilogGetInstCDFPropLenWidValue
_hnlVlogCxt hnlVerilogNetBitSelect
_hnlVlogCxt hnlVerilogCreateMSTestFixtureFile

Then I created FILE1.txt

Code:

bash-4.1$ cat FILE1.txt 
hnlVerilogCrossViewCheck
hnlVerilogGetInstCDFPropLenWidValue
hnlVerilogNetBitSelect
hnlVerilogCreateMSTestFixtureFile

and then executed (with the bash script I posted previously)

Code:

find ./junk -type f -exec t.sh '{}' \;

with the results

Code:

bash-4.1$ cat junk/t3.txt
_hnlVlogCxt _hnlVlogCxtCrossViewCheck
_hnlVlogCxt _hnlVlogCxtGetInstCDFPropLenWidValue
_hnlVlogCxt _hnlVlogCxtNetBitSelect
_hnlVlogCxt _hnlVlogCxtCreateMSTestFixtureFile

bash-4.1$ cat junk/junk1/t4.txt
_hnlVlogCxt _hnlVlogCxtCrossViewCheck
_hnlVlogCxt _hnlVlogCxtGetInstCDFPropLenWidValue
_hnlVlogCxt _hnlVlogCxtNetBitSelect
_hnlVlogCxt _hnlVlogCxtCreateMSTestFixtureFile

Why is this not right for you?

shridhar22 · 01-15-2013, 02:37 AM

@Allend I wanted

Quote:

bash-4.1$ cat junk/t3.txt
_hnlVlogCxtCrossViewCheck
_hnlVlogCxtGetInstCDFPropLenWidValue
_hnlVlogCxtNetBitSelect
_hnlVlogCxtCreateMSTestFixtureFile

bash-4.1$ cat junk/junk1/t4.txt
_hnlVlogCxtCrossViewCheck
_hnlVlogCxtGetInstCDFPropLenWidValue
_hnlVlogCxtNetBitSelect
_hnlVlogCxtCreateMSTestFixtureFile

not the additional prefexed _hnlVlogCxt

And on thing i also noticed that you created files to be changed=

Quote:

bash-4.1$ cat junk/t3.txt
_hnlVlogCxt hnlVerilogCrossViewCheck
_hnlVlogCxt hnlVerilogGetInstCDFPropLenWidValue
_hnlVlogCxt hnlVerilogNetBitSelect
_hnlVlogCxt hnlVerilogCreateMSTestFixtureFile

bash-4.1$ cat junk/junk1/t4.txt
_hnlVlogCxt hnlVerilogCrossViewCheck
_hnlVlogCxt hnlVerilogGetInstCDFPropLenWidValue
_hnlVlogCxt hnlVerilogNetBitSelect
_hnlVlogCxt hnlVerilogCreateMSTestFixtureFile

but in my case each of them DONT HAVE have the prefexed _hnlVlogCxt.
i.e.

Quote:

bash-4.1$ cat junk/t3.txt
hnlVerilogCrossViewCheck
hnlVerilogGetInstCDFPropLenWidValue
hnlVerilogNetBitSelect
hnlVerilogCreateMSTestFixtureFile

bash-4.1$ cat junk/junk1/t4.txt
hnlVerilogCrossViewCheck
hnlVerilogGetInstCDFPropLenWidValue
hnlVerilogNetBitSelect
hnlVerilogCreateMSTestFixtureFile