[SOLVED] sed: spaces, quotes, alternative patterns, substitution

rm_-rf_windows · 03-12-2012, 04:42 PM

Hi all,

I've been struggling with sed for over 2 hours now and thought I'd post my problem.

I need the following to be changed into the following:

Code:

THIS --> THAT
"hello       " --> "hello" (quotations included)
"       hello" --> "hello" (quotations always included)
"    hello    " --> "hello" ...
"      John F. Kennedy    " --> "John F. Kennedy"
"  Secret Agent 007    " --> "Secret Agent 007" 
"(space)+(anything but a space)+((space)?(not a space)+)*(space)+" --> "(anything but a space)+((space)?(not a space)+)*"

I just can't figure it out! Not with sed, I've done it in SQL scripts because I can regroup patterns with parentheses. However I'm not sure how to do this with sed.

Many thanks,

rm

Dark_Helmet · 03-12-2012, 04:52 PM

Code:

sed -r 's@("[[:space:]]+|[[:space:]]+")@"@g'

For example:

Code:

user@localhost$ echo '"hello       "' | sed -r 's@("[[:space:]]+|[[:space:]]+")@"@g'
"hello"
user@localhost$ echo '"    hello    "' | sed -r 's@("[[:space:]]+|[[:space:]]+")@"@g'
"hello"
user@localhost$ echo '"      John F. Kennedy    "' | sed -r 's@("[[:space:]]+|[[:space:]]+")@"@g'
"John F. Kennedy"

colucix · 03-12-2012, 04:54 PM

Tried this?

Code:

sed -r 's/"[[:blank:]]*(.+[^[:blank:]])[[:blank:]]*"/"\1"/g' file

The part highlighted in green is mandatory to take in account spaces in the middle of the string. It means a non-blank character followed by a sequence of blanks (if any) immediately before the closing quotes.

rm_-rf_windows · 03-12-2012, 06:41 PM

Ciao ragazzi,

Thanks for the replies. Dark Helmet's solution works with the g option, not without, colucix's works with or without the g option, I don't know why.

Code:

$ echo '"     Ciao ragazzi!!     "' | sed -r 's@("[[:space:]]+|[[:space:]]+")@"@g'
"Ciao ragazzi!!"
$ echo '"     Ciao ragazzi!!     "' | sed -r 's@("[[:space:]]+|[[:space:]]+")@"@'
"Ciao ragazzi!!     "
$ echo '"     Ciao ragazzi!!     "' |  sed -r 's/"[[:blank:]]*(.+[^[:blank:]])[[:blank:]]*"/"\1"/'
"Ciao ragazzi!!"
$ echo '"     Ciao ragazzi!!     "' |  sed -r 's/"[[:blank:]]*(.+[^[:blank:]])[[:blank:]]*"/"\1"/g'
"Ciao ragazzi!!"
$

In any case, the problem is solved, this thread can be closed. I don't know how to mark this "[SOLVED]". That was quick, effective.

Many thanks!

rm

colucix · 03-13-2012, 03:20 AM

Quote:

Originally Posted by rm_-rf_windows

Ciao ragazzi,
Thanks for the replies. Dark Helmet's solution works with the g option, not without, colucix's works with or without the g option, I don't know why.

Ciao. The solution by Dark_Helmet requires the g option to do more than one substitution, since it has an alternate pattern, that is it substitutes a pattern OR another pattern. Without g the first matched pattern is substituted and the rest is ignored. My solution has a unique pattern that spans all over the input line.

Furthermore, I noticed that Dark's solution makes extra (and maybe unwanted) substitutions if the quoted text is inside a longer line, e.g.

Code:

$ echo 'Io li vidi da lontano e dissi "   Ciao ragazzi!! " e lei si voltò verso di me' | sed -r 's@("[[:space:]]+|[[:space:]]+")@"@g'
Io li vidi da lontano e dissi"   Ciao ragazzi!!" e lei si voltò verso di me

Notice the space before the opening quote has been removed and the spaces after it has been preserved. Just for the sake of exactness!

Dark_Helmet · 03-13-2012, 06:45 AM

Both colucix's solution and mine operate on assumptions regarding the data set. Both assumptions are valid given the sample data you provided.

As colucix correctly pointed out, a string outside of those assumptions will give unexpected/undesireable results. The same is true for his solution as well.

Code:

echo 'He said,   "    where are they?  ", and she responded with, "   right there!  "' | sed -r 's/"[[:blank:]]*(.+[^[:blank:]])[[:blank:]]*"/"\1"/g'
He said,   "where are they?  ", and she responded with, "   right there!"

My solution assumes that there will be no instance where spaces need to be removed both before and after a double quote whereas colucix's solution assumes there is only one double quote pair on the line.

That's the thing about regular expressions. The more detail you give about the data set, the more accurate the solution.

colucix · 03-13-2012, 08:46 AM

Quote:

Originally Posted by Dark_Helmet

colucix's solution assumes there is only one double quote pair on the line.

That's the thing about regular expressions. The more detail you give about the data set, the more accurate the solution.

Totally agreed!