replace space with newline and save result in a variable
Hi there!
My script is supposed to do the following: 1) wget google search result for $MYSEARCH 2) grep interesting parts with regexp (URIs in this case) $URIPATTERN and i want this to happen with variables, so my hdd is not writing small data all the time (there are other parts in my script which download about 100-300 rather small files just to scan it's content. But i hope to adopt the solution to these parts by myself) Here's what I've got: Code:
DATAPB=`wget -O - -q -U "Mozilla" 'http://www.google.com/search?q='$MYSEARCH'&tbs=qdr:d'` Code:
echo $DATAPB | sed 's/ http/\nhttp/g' Code:
DATAPB=`echo $DATAPB | sed 's/ http/\nhttp/g'` Do i need to double-escape the \n or something? Thanks for your help. PS: none of the following works either: Code:
DATAPB=$(echo $DATAPB | sed 's/ http/\nhttp/g') |
What does the output of echo "$DATAPB" look like? bash will convert any consecutive whitespace characters (tabs, spaces and newlines) to a single space without the quotes
|
Hi,
I ran the following command: Code:
wget -O - -q -U "Mozilla" 'http://www.google.com/search?q='linux'&tbs=qdr:d' Code:
<a href="http://www.howtoforge.com/installing-windows-software-with-wine-linux-mint-11" class=l>Installing Windows Software With Wine (<em>Linux</em> Mint 11) | HowtoForge <b>...</b></a></h3><div class="esc" id="poS12" style="display:none">You +1'd this publicly. <a href="#">Undo</a></div><div class="s"><span class="f std" >17 hours ago</span> - Installing Windows Software With Wine (<em>Linux</em> Mint 11)<br><span class=f><cite><span class=bc>www.howtoforge.com › <a href="/url?q=http://www.howtoforge.com/howtos&sa=X&ei=ArqvTpa7JY2KhQeEp-nXAg&ved=0CFoQ6QUoADAM&usg=AFQjCNFSiAx5T6Yz-VGC-q5t2g3fKcbTsA">Howtos</a> › <a href="/url?q=http://www.howtoforge.com/howtos/linux&sa=X&ei=ArqvTpa7JY2KhQeEp-nXAg&ved=0CFsQ6QUoATAM&usg=AFQjCNGZqNywPfdWyS1VJZe67rTKkG3lRQ">Linux</a> › Code:
http Code:
DATAPB=$(wget -O - -q -U "Mozilla" 'http://www.google.com/search?q='linux'&tbs=qdr:d'|sed -nr 's/<a href="/\n&/gp'|grep 'http') Also, have a look at this link here why you should avoid backticks for command substitution. Hope this helps. |
You need a "here file"
Basically shell variables are not designed to take newlines.
You might want to use something like this: DATAPB=$(wget -O - -q -U "Mozilla" 'http://www.google.com/search?q='$MYSEARCH'&tbs=qdr:d') for x in $DATAPB do echo $x done | some-command You can define a shell function some-command and make it as complex as you like. |
Quote:
|
Quote:
Code:
$ variable='foobar See her for more on how the shell handles arguments and whitespace: http://mywiki.wooledge.org/Arguments http://mywiki.wooledge.org/WordSplitting http://mywiki.wooledge.org/Quotes To change spaces into newlines inside a variable, you can use a simple parameter expansion. Code:
$ variable='foobar foobar foobar' BUT FINALLY, in cases like this, you really should be using an ARRAY to store the urls, rather than a scalar variable. Then you won't have to worry about dealing with newlines at all. http://mywiki.wooledge.org/BashGuide/Arrays http://mywiki.wooledge.org/BashFAQ/005 Assuming you can get your wget/grep command to output a clean list of urls, separated by whitespace of some kind (space-tab-newline), then you set them into an array like this: Code:
urls=( $( wget.... | grep .... ) ) P.S. @huge; Please use [code][/code] tags around your code, to preserve formatting and to improve readability. |
All times are GMT -5. The time now is 11:12 AM. |