Use "curl" to get the returned page that contains your search results. Store its contents in a variable or to a file and then pipe the contents to "grep" to extract only hyper links:
You have given "Cherry" above. I right-clicked and view the source and copied those lines then saved them to a file for this example:
Code:
[demo@localhost ~]$ cat data
<div id="post_message_4604415"><!-- google_ad_section_start -->An Example, a search for the word cherries, renders"<br />
<br />
<a href="http://en.wikipedia.org/wiki/Cherry" target="_blank">Cherry Wikipedia, the free encyclopedia</a><br />
<br />
<a href="http://www.choosecherries.com" target="_blank">Cherries. America's "Super Fruit"</a><br />
<br />
<a href="http://www.choosecherries.com/health/main.aspx" target="_blank">Cherry Health and Cherry Nutrition</a><br />
<br />
<br />
I'd like to extract the text with the url embedded.<!-- google_ad_section_end --></div>
<!-- / message -->
Code:
cat data | grep -oE '<a href.*</a>'
<a href="http://en.wikipedia.org/wiki/Cherry" target="_blank">Cherry Wikipedia, the free encyclopedia</a>
<a href="http://www.choosecherries.com" target="_blank">Cherries. America's "Super Fruit"</a>
<a href="http://www.choosecherries.com/health/main.aspx" target="_blank">Cherry Health and Cherry Nutrition</a>
So, you get URLS/Links/Hyperlinks with the text.
Another example:
Code:
echo "$curled" | grep -oE '<a href.*</a>'
Note: The above example assumed that you're storing the output of "curl" command to the named variable, i.e. "curled".
You can further use "sed" to instert "<li>" and "</li>" tags to embed each line so that you have a list of Links.
Code:
cat data | grep -oE '<a href.*</a>' | sed -e 's/^/<li>/' -e 's/$/<\/li>/'
<li><a href="http://en.wikipedia.org/wiki/Cherry" target="_blank">Cherry Wikipedia, the free encyclopedia</a></li>
<li><a href="http://www.choosecherries.com" target="_blank">Cherries. America's "Super Fruit"</a></li>
<li><a href="http://www.choosecherries.com/health/main.aspx" target="_blank">Cherry Health and Cherry Nutrition</a></li>
Now, you can put "<ol>" and "</ol>" or "<ul>" and "</ul>" to surround the entire output to suit your fancy requirement.
Cheers!
