[SOLVED] Using a long Bash command including single quotes and pipes in a Bash script

antcore · 07-22-2009, 08:42 AM

This long command works fine from the command line.

Code:

lynx -dump http://www.ip-adress.com/ip_tracer/?QRY=$1|grep address|egrep 'city|state|country'|awk '{print $3,$4,$5,$6,$7,$8}'|sed 's\ip address flag \\'|sed 's\My\\' | sed 's\address \\' |  perl -ne 'if (3..3) {print "$_"; last;}' |sed 's\city: \\'

However, if I want to use it in a script as such:

Code:

location=`lynx -dump http://www.ip-adress.com/ip_tracer/?QRY=$1|grep address|egrep 'city|state|country'|awk '{print $3,$4,$5,$6,$7,$8}'|sed 's\ip address flag \\'|sed 's\My\\' | sed 's\address \\' |  perl -ne 'if (3..3) {print "$_"; last;}' |sed 's\city: \\'`

I get

Code:

sed: -e expression #1, char 19: unterminated `s' command
sed: -e expression #1, char 5: unterminated `s' command
sed: -e expression #1, char 11: unterminated `s' command
sed: -e expression #1, char 9: unterminated `s' command

using

Code:

location=`'COMMAND'`

does not work because there are single quotes in the command. Trying

Code:

location=`'lynx -dump http://www.ip-adress.com/ip_tracer/?QRY=$1|grep address|egrep '\'' # etcetera

gives similar errors. Using double quotes and escaping all $'s also gives errors. There must be a smarter way to do this. Who'll educate me?

David the H. · 07-22-2009, 09:18 AM

It's recommended not to use backticks to embed commands. It's been superseded by the $() form. I don't get any errors when I use location=$(<your_long_command>).

Edit: By the way, you can usually use "sed -e 'expression1' -e 'expression2', to combine your filters into a single command.

Also, I can't be 100% certain because I don't know what your input value is supposed to be, but I'm guessing the error is coming from a '/' value being in the sed input somewhere, messing up the reading of the separators. Or something like that. You can change the sed separator to another character instead (e.g. sed -e 's|ip address flag ||') which also eliminates the error messages.

By the way, just what exactly are you trying to extract from the lynx output anyway? It's likely we can clean up the command and make it more efficient. I tried running it with the ip address of google.com and got nothing.

i92guboj · 07-22-2009, 09:29 AM

Besides that, using all of perl, awk, sed, egrep and grep is, to say the least, an overkill. Either awk or perl should be enough to do all the parsing, or even sed. I'd also drop lynx in favor of wget -O - if possible, it seems to be much standard these days, but that's just my opinion.

I haven't looked deeply into the purpose of that one-liner, but it seems to me that you are creating complication for nothing.

antcore · 07-22-2009, 09:31 AM

This works. Thank you.

antcore · 07-22-2009, 09:34 AM

Thanks for the solution regarding the syntax, David the H.

With regard to the command itself. It is not a beauty, yet. With regard to what it does, it gives back the name of the city your provider is in, based on your IP.

David the H. · 07-22-2009, 09:41 AM

Well, I'd help you to find a better way, but all I'm getting now is a "You reached your limit of 10 lookup queries per day" message, so I can't test it anymore. If you could provide a sample of the output we could work from instead, I'm sure we could give you a cleaner way to get that info.

antcore · 07-22-2009, 09:57 AM

My goal was to get a string representing the current city in which I am. (I push this string to a list which serves as the source for a web page on which some colleagues of mine can read my latest location update.) Looking for a simple way to extract my current location, I came across a working one liner at http://www.commandlinefu.com/ which produced relevant output for me:

Code:

address country: Netherlands
address state: Zuid-Holland   
address city: Leiden

Since I did not need the first two lines and only the third word of the last line, I added:

Code:

 | sed 's\address \\' |  perl -ne 'if (3..3) {print "$_"; last;}' |sed 's\city: \\'

which gives me:

Code:

Leiden

i92guboj · 07-22-2009, 10:16 AM

I am not testing this, but, the first one you posted is grep'ing for "address", then whatever else.

Why not just grep for "address city" instead if that's all you want? Then just sed it to remove "address city: ".

Code:

<whatever to fetch the page> | grep "address city:" | sed -e 's/address city: //'

Much simpler. Just test it step by step, first the grep, then based on the output, look what do you exactly need to cut with sed, and so on.

David the H. · 07-22-2009, 10:22 AM

I'd need to see the exact input in order to make sure that we can always match the city name. For example, is there only one line with the phrase "address city" in it, or could there be multiple hits?

If, for example, there's only one line with "address city: cityname" in it, and nothing following it but a newline, then it could be as simple as

Code:

<input>|sed -nr s/'^.*address city: (.+)$/\1/p'

I'm thinking it might not be as easy as that though, or else the original line probably wouldn't be as complex as it is.

antcore · 07-22-2009, 11:10 AM

David the H. is right:

Code:

lynx -dump http://www.ip-adress.com/ip_tracer/?QRY=$1|sed -nr s/'^.*My IP address city: (.+)$/\1/p'

and all is fine.

With regard to i92guboj's remark about lynx: w3m also does the trick, wget gives a 403.