SED with variables

RocketJumper · 02-03-2011, 10:58 PM

Hello everyone, I have this problem below.

Quote:

b="./list"
echo './list ./out' | sed "/$b/d"

Which comes out as an error.

Quote:

sed: -e expression #1, char 5: extra characters after command

Anyone have any idea how this can this solved?

grail · 02-04-2011, 12:08 AM

Have a look at your delimiter in sed, it is a slash (/). Now have a look at the string being passed in:

Code:

b="./list"

If you combine this altogether then it looks like (once expanded):

Code:

echo './list ./out' | sed "/./list/d"

If your ever not sure or cannot see the issue plainly, try doing the following:

Code:

echo './list ./out' | echo sed "/$b/d"

xeleema · 02-04-2011, 12:37 AM

Greetingz!

Yes, grasshopper, you must master shell escapes, or forever STDERR will dominate you.

Code:

b="./list"
echo './list ./out' | sed "/\$b/d"

Remember to give thanks when earned, and mark any solved threads as such via the "Thread Tools"
Nevermind!

grail · 02-04-2011, 03:25 AM

@xeleema - sorry to be the bearer of bads new but escaping the $ sign is causing only lines with '$b' to be removed. Again the echo addition would yield:

Code:

$ echo './list ./out' | echo sed "/\$b/d"
sed /$b/d

Which I am guessing will still not help OP to remove this line.

Personally I am not sure there is a solution based on this scenario as the escape would actually need to look like the following, assuming no variable being used:

Code:

echo './list ./out' | sed "/\.\/list/d"

So both the dot (.) and the slash (/) need to be escaped to get an accurate removal.

xeleema · 02-04-2011, 04:42 AM

@grail
Hey, but at least I got past the error. In my defense, OP never said what it was they were trying to actually accomplish. :P
So is there any way to feed sed a variable and have it act on it? I'm kinda curious about that myself.

Basically, if the value of b is unknown, how can we write a sed statement that will delete b from the string?

grail · 02-04-2011, 06:50 AM

I am guessing, as we still need OP to come back, that the idea initially was to simply use a variable in a sed statement. I think that maybe the unintentional fact that the value of this particular
variable is causing so much angst was not the initial intention.

Guess we will wait and see what they say

crts · 02-04-2011, 08:33 AM

Quote:

Originally Posted by grail

Personally I am not sure there is a solution based on this scenario as the escape would actually need to look like the following, assuming no variable being used:

Code:

echo './list ./out' | sed "/\.\/list/d"

So both the dot (.) and the slash (/) need to be escaped to get an accurate removal.

Well, there is but it's (surprise) ugly. Generally, for using variables which contain a delimiter

Code:

b="./list"
echo './list ./out' | sed "/${b////\/}/d"

Better: Also escaping the 3. slash. Though not necessary in the above case. However, NOT escaping it will cause trouble when handling the '.'.
So finally

Code:

b="./list"
echo './list ./out' | echo sed "/${b//.\//\.\/}/d"

However, this technique also has it's limitations, e.g., when there are too many characters that would have to be escaped in order to not be interpreted as RegEx.

Ramurd · 02-04-2011, 08:52 AM

It might be prettier to escape the search string in the first place:

Code:

export b="\.\/list" # ./list
echo './list ./out'| sed "s/${b}.//"

search and replace by nothing (=effectively removal)

crts · 02-04-2011, 09:07 AM

Quote:

Originally Posted by Ramurd

It might be prettier to escape the search string in the first place:

Code:

export b="\.\/list" # ./list
echo './list ./out'| sed "s/${b}.//"

search and replace by nothing (=effectively removal)

Well, yes, but sometimes you do not know in advance how the variable looks like. And sometimes when you read a text-file you do not want to change it's contents. In those cases escaping the delimiter is a good choice.

theNbomr · 02-04-2011, 06:47 PM

Maybe this is an example of using the wrong tool for the job. Perl has the quotemeta() function to handle cases just like this.
--- rod.

grail · 02-05-2011, 02:28 AM

I am with rod on this one. Whilst methods have been shown to solve the problem, I can easily see issues arising from both methods.

As usual, the OP will need to explain what there requirement is a bit more as we may be solving issues that are not really there but that the chosen
example made it particularly complicated

MTK358 · 02-06-2011, 08:31 AM

Just use a delimiter other than '/'?

grail · 02-06-2011, 09:09 AM

@MTK358 - to the best of my knowledge your suggestion is only good when being used with the replace option. Happy for you to correct me if I am wrong?

MTK358 · 02-06-2011, 09:20 AM

Quote:

Originally Posted by grail

@MTK358 - to the best of my knowledge your suggestion is only good when being used with the replace option. Happy for you to correct me if I am wrong?

I didn't notice that it wasn't a replacement. There probably is a way to use a different delimiter, but I'm not familiar enough with sed to know.

You can also escape the slashes, the advantage of that is that the filename can contain any characters you want.

Code:

b=./list
b=$(echo "$b" | sed 's#/#\\/#g')
echo './list ./out' | sed "/$b/d"

crts · 02-06-2011, 09:49 AM

Quote:

Originally Posted by MTK358

You can also escape the slashes, the advantage of that is that the filename can contain any characters you want.

Code:

b=./list
b=$(echo "$b" | sed 's#/#\\/#g')
echo './list ./out' | sed "/$b/d"

That is what the bash substitution is for in my earlier post

Code:

echo './list ./out' | sed "/${b////\/}/d"

When dealing with variables it is good practice to escape the delimiter if the content is unknown.

Quote:

There probably is a way to use a different delimiter, but I'm not familiar enough with sed to know.

There is

Code:

sed '\#pattern# d' file

But as it has already been stated, it is always problematic when the variable contains characters which can be interpreted as part of a RegEx.