LinuxQuestions.org - [SOLVED] search and replace string with special character perl or sed.

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - search and replace string with special character perl or sed. (https://www.linuxquestions.org/questions/programming-9/search-and-replace-string-with-special-character-perl-or-sed-945781/)

search and replace string with special character perl or sed.

I am trying to use perl or sed to edit some directory paths in configuration files. The problem is I want to delete part of the path (so replace BOBDOG/ with nothing). Following is some of the code I have tried.

This leaves 0 byte files behind :(

Code:

#!/bin/bash

#



echo "Type your find text followed by [Enter]:"

read strFind



perl -e "s/"$strFind"/""/g;" -pi $(find test -type f);

This runs from within the directory and isn't recursive. It also leaves 0 byte files behind.

Code:

#!/bin/bash

#



SCRIPTNAME=`echo $0|cut -f2 -d/`



echo "Type your find text followed by [Enter]:"

read strFind

#echo "Type your replace text followed by [Enter]:"

#read strReplace



for file in `ls *`

do

#echo $file

#echo $SCRIPTNAME

        if [ "$file" <> "$SCRIPTNAME" ]; then

                sed "s|$strFind|\"\"|g" $file > temp; mv temp $file;

        fi

done

I'd like something generic that will ask for the search and the replace text but be able to handle special characters or just the [Enter] (chr(13) I believe) if I want to delete the searched string.

Any ideas?

Sed can use many different delimiters for substitutions. Pipes for example. This is particularly handy for working with paths, and other code.
for example.

Code:

echo '/home/bobdog/billdog/documents' | sed "s|bobdog/||g"

/home/billdog/documents

You could wrap this in a for loop that iterated through a list of config files to change.

Quote:

for file in `ls *`

`ls *` is a bad idea, and will break for a bunch of reasons shown here
It is better (and easier) to let BASH expand * itself..

Code:

for file in * ; do

It would help a lot if you would provide an actual example of the input that needs to be worked on, and the desired output.

Code:

perl -e "s/"$strFind"/""/g;" -pi $(find test -type f);

1) The output of the "$()" at the end is word-split and glob-expanded after the substitution is made. Any filenames with whitespace or globbing characters will result in unreadable names or other undesirable behavior.

2) The proper way to use find with external tools is through the -exec option. Also, note that the syntax is "find <directories> <tests> <action>"

Code:

find . -type f -exec perl -e "s/$strFind/\"\"/g;" -pi '{}' \+

If the above can't handle multiple file input at once, use "\;" at the end instead.

http://mywiki.wooledge.org/UsingFind
http://www.grymoire.com/Unix/Find.html

Code:

SCRIPTNAME=`echo $0|cut -f2 -d/`

1) Parameter substitution can handle this more cleanly:

Code:

scriptname=${0##*/}

2) basename is a better external command to use than cut anyway.

3) $(..) is highly recommended over `..`

4) Since environment variables are generally all upper-case, it's good practice to keep your own user variables in lower-case or mixed-case to help differentiate them.

Code:

if [ "$file" <> "$SCRIPTNAME" ]; then

1) When using bash or ksh, it's recommended to use ((..)) for numerical tests, and [[..]] for string/file tests and other complex expressions. Avoid using the old [..] test unless you specifically need POSIX-style portability.

http://mywiki.wooledge.org/ArithmeticExpression
http://mywiki.wooledge.org/BashFAQ/031
http://wiki.bash-hackers.org/commands/classictest
http://wiki.bash-hackers.org/syntax/...nal_expression

2) "<>" is a file redirection, not a comparison test. In bash at least, it opens up the filename following it for both reading and writing, and the parameter in front of it should be the number of the file descriptor. To compare two strings, use "=" in the traditional test, or "==" in the double-bracket test.

This is also another reason not to use "[..]". Since it's just a simple command, the redirector is allowed to be processed first, and the file will be created if it doesn't exist. But inside the "[[..]]" keyword, it's considered improper syntax instead, generating an error.

3) A better way to exclude the script name from the loop is with the -ef comparison anyway.

Code:

if [[ ! $file -ef $0 ]] ; then

Code:

sed "s|$strFind|\"\"|g" $file > temp; mv temp $file;

1) QUOTE ALL OF YOUR VARIABLE SUBSTITUTIONS. You should never leave the quotes off a parameter expansion unless you explicitly want the resulting string to be word-split by the shell (globbing patterns are also expanded). This is a vitally important concept in scripting, so train yourself to do it correctly now. You can learn about the exceptions later.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

Refer back to my very first point as well.

2) If you want to replace a string with nothing, as you stated, don't use quotes in the RHS replacement. Just leave it empty. Of course if you do want the quotes to be inserted, it's just fine.

gnu sed has "-i" as an option for editing a file in place, letting you skip the tempfile.

Code:

sed -i "s|$strFind||g" "$file"

More detailed information.

First thanks for the replies. I have a lot to go through. Second, I will give some more information about what I am trying to do.

I need to loop through a number of directories and sub-directories looking for a particular string (in a path) in various files and deleting it. A typical file might look like this.

Code:

#!/bin/sh

CUR_DIR=$(pwd)

ADD_LANG_DIR=`dirname $0`

cd $ADD_LANG_DIR

HOME=/oracle/app/TEST/agent/core/12.1.0.1.0

HOMENAME=testinst1

./runInstaller -addLangs HOME=$HOME HOME_NAME=$HOMENAME $* 

cd $CUR_DIR

I want to loop through these files and find all instances of TEST/ and delete them or modify them to something else. Modifying would actually be easier as I wouldn't need to change the /. Right now though I am trying to remove the string and it winds up deleting all the contents of the file.

I was able to get this working in a single directory using the sed commands as you specified, David, but I would need to move the script to every directory and there are just too many, leading me to using the *find* which is recursive.

Let me know if anything else would be helpful in understanding the problem.

Thanks again for your help.

Sorry I forgot to give you the new find/perl command I was trying to use.

Code:

#!/bin/bash

#

echo "Type your find text followed by [Enter]:"

read strFind



find test -type f -exec perl -e "s/$strFind//g;" -pi '{}' \+

So I am doing a find on a subdir named test finding files and executing the perl command.

This gives me *Illegal division by zero at -e line 1, <> line 1.* and it doesn't change any of the files.

Thanks,
Kevin