Search and replace line in multiple files

dnoy · 07-13-2010, 10:45 AM

Hello, I need some help searching through multiple files, finding a line and replacing that line. The line I am searching for is:

password key ******* 1222554

ultimately I want to be able to delete the numbers after the asterisks . my thoughts are to create a script that will search for the line password key ******* and delete it then replace it with password key *******

my files are located in /opt and they are all txt files.

Please let me know the best scripting method to get this accomplished.

Thank you,

colucix · 07-13-2010, 10:59 AM

It is not clear from your example if the file contains real asterisks or you used them to mask real passwords. Anyway you can accomplish this task with a simple sed command. It might be something like this:

Code:

sed -i -r 's/^(password key .+) [0-9]+$/\1/' file

Please, pay attention to the fact that the -i option edit the file in place, actually changing its content. Better if you test it on a safe copy of the original files.

Then you can repeat the command over a bunch of files either by embedding the sed line in a while loop or by passing all the file names as multiple arguments to the sed command itself. In the first case you can do something like:

Code:

while read file
do
  sed -i -r 's/^(password key .+) [0-9]+$/\1/' "$file"
done < <(find /opt <your search criteria here>)

where the loop is fed by the result of a find command through process substitution. Hope this helps.

dnoy · 07-13-2010, 11:30 AM

colucix thanks for your help but that didnt work. The password does contain real asterisks. should i be using a -e instead of -r? i tried the -e but there are problems with that.

Thank you,

colucix · 07-13-2010, 11:59 AM

Nope. The -r option means "make use of extended regexp" and let the + sign be interpreted as "match one or more occurrences of the previous pattern". In the suggested command line I used anchors to match the beginning ^ and the end $ of the line. Maybe there are extra characters in your lines that prevent the exact matching. What if you try without them?

Code:

sed -r 's/(password key .+) [0-9]+/\1/' file

Please, show us the results (or error messages) of your commands together with the sed version you're running, e.g.

Code:

$ cat testfile
password key ******* 1222554
password key ******* 5362578
$ sed -r 's/(password key .+) [0-9]+/\1/' testfile
password key *******
password key *******
$ sed --version
GNU sed version 4.1.5
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.

pixellany · 07-13-2010, 12:04 PM

If you use "*" in a regex, it has to be escaped. (Otherwise, it means "any number of the preceding expression or character")

Untested (some pseudo-code):

Code:

for fil in ls -R; do
    if it is a file; then
         sed -i -r 's/(passwordkey\*+)[0-9]+/\1/g' $fil
    fi
done

\*+ means a group of at least one asterisks

dnoy · 07-13-2010, 12:15 PM

cat me.txt

password key ******** 225541A222221111F8

$ sed -r 's/(password key .+) [0-9]+/\1/' me.txt

password key ********A222221111F8

$ sed --version
GNU sed version 4.1.5
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.

pixellany · 07-13-2010, 12:24 PM

That is working as expected.....

Earlier you said you wanted to "delete the numbers after the asterisks".....
Do you want to:
---delete everything after the asterisks?
---delete a specific pattern after the asterisks?
---delete everything after the asterisks but before another pattern?
---other?

dnoy · 07-13-2010, 12:30 PM

i would like to delete everything after the asterisks on that line. I have multiple lines under there but do not wish to delete any of those.

example

blah blah blah
text
password key ******** 225541A222221111F8
blah blah blah

i want this to be:

blah blah blah
text
password key ******** 225541A222221111F8
blah blah blah

pixellany · 07-13-2010, 12:36 PM

Delete everything after the asterisks:

sed -r 's/(password key \*+).*/\1/' me.txt

".*" means any number of any character---if you kept it that way, it would by itself go to the end of the line, and the second ".*" would have no effect.

dnoy · 07-13-2010, 12:44 PM

pixellany, great thank you so much, this is what i needed.

dnoy · 07-13-2010, 12:51 PM

OK, another question. I have over 100 files that i need to run through. Can you also help me create a while loop for this?

pixellany · 07-13-2010, 12:59 PM

We already gave some examples of how to do the loops.....

The easiest case---if the files are all in one directory, and there are no sub-directories:

Code:

for fil in *; do
    <sed stuff here>
done

Try something like this first---to get familiar with the concept:

Code:

for fil in *; do
    echo $fil
done

The above loops are setup to be run in the directory where the files are. To run from anywhere, add a full pathname in front of the *.

dnoy · 07-13-2010, 01:01 PM

this seems to search multiple files:

sed -i -r 's/(password key \*+).*/\1/' *.txt

looks like i dont need a while loop

pixellany · 07-13-2010, 01:11 PM

"If it works, it's OK"