awk, sed find and replace recursively from files

jschiwal · 02-13-2010, 07:35 PM

If you use
sed "s#pattern#replacepattern#", then the pattern and replacement can have slashes in them. In my solution, I did this as well.

The exclamation point can be a problem if you are working interactively. It is used for recalling commands from the history buffer. It needs to be escaped then typing commands on the command line interface but not if it's in a script. This can sometimes make debugging or designing a program more difficult.

Looking closer at one of your solutions,

Code:

sed "#$line#s#$line\/#!$line\/#g" file2

The first two octothorpes won't work. You need to use slashes for pattern matching before the sed command.
Since $line matches the entire line, you don't need to match a pattern to locate the line. The LHS contains the entire line.
Also, since the entire line is in the LHS, you don't need "g" at the end.

You can use have a pattern test in the beginning determine whether to execute the following
sed command.
/pattern/s ...
/pattern/!s ...,

The second one is run if the pattern isn't found.

---

By the way, your new information about the data causes the particular sed command I used not to work. Working with sed, awk & grep, the pattern of the input can be very important. Special conditions need to be tested for as well.

bluewind · 02-14-2010, 06:32 AM

Hi All,

Thanks jschiwal for your great input really helpful.
I have managed to solve the problem with this script finally

Soon will try to adapt your method of creating sed command using sed itself

I am posting this because it may be useful for others

and as i mentioned before in my previous post, my
input file contains meta-characters in various places. Hence the solution I used:

#!/bin/sh

while read line; do
search="$line"
sed "s#$search#!$search#" file2 >tempfile.tmp
mv tempfile.tmp file2
done < file1

Input file1

a+b=c+d 1e105
g+h=o+p abcdefg
rev/ 0.35 / h 35/
h2 / 20 / he / ar

file2 (going to be changed file)

a+b=c+d 1e105
x+y=z+s 5e105
g+h=o+p abcdefg
t+r=w+q xvyderf
rev/ 0.35 / h 35/
h2 / 20 / he / ar

file2 (after changes made)

!a+b=c+d 1e105
x+y=z+s 5e105
!g+h=o+p abcdefg
t+r=w+q xvyderf
!rev/ 0.35 / h 35/
!h2 / 20 / he / ar

So with this script the problem with regards to meta-characters is solved

bluewind · 02-26-2010, 11:06 AM

Quote:

Originally Posted by jschiwal

Your program will run sed for each line of input in file1.

Each time you run it you are running it on the same input file instead of saving what was done so far.

My idea is to use sed to process file1 to create a sed program.

sed '...' file1 >process.sed

This should produce these lines given your specific example:
s/g+h=o+p/!g+h=o+p/
s/a+b=c+d/!a+b=c+d/

Then you would run:
sed -f process.sed file2 >file3

There will be as many lines in process.sed as in your file1 file. If some lines in file1 are repeated, you could use "sort file1 | uniq" to eliminate redundancy.

I have left out the sed command I used to process your input file "file1".

I have used this technique at work to generate a script to delete files from a list of names (without the extension). There are sometimes up to 2000 files I need to delete from about 60 devices. Doing this using a graphical interface would take forever.

One script I make is run in Cygwin/X to delete files in a shared directory on a Windows server. Another is a script I upload to a number of devices. These devices have a custom OS so the command lines are different. I just need to produce the list and then convert it to each style of script.
So your example isn't a far fetched one. You may find it useful in the future.

Hi jschiwal,

I have been trying your suggestion for some time, but I keep get through the "wall"

At the moment i could get up to:
s/g+h=o+p/!
s/a+b=c+d/!

Using these commands:
#!/bin/sh
sed 's/^/s#/' <file1 >file2
sed 's/$/#!/' <file2 >file3

But I really dont know how to make it get the 1st part back, g+h=o+p/ back at the end of the line to finally get like you have mentioned:

s/g+h=o+p/!g+h=o+p/
s/a+b=c+d/!a+b=c+d/

Thanks in advance