How to extract domains, links from one webpage.
Hi,
I'd like to know how to extract domain names from one site.
with CLI or third party programs.
I want to add these domains to my squid list.This is easy but I have to compile the list first.
I can do this by hand but it is very time consuming.
For example:
One site
has subfolders and every subfolder is for categories.
Every page contains hundreds of links.
domainname.com/subfolder/page1
domainname.com/subfolder/page2
domainname.com/subfolder/page3.html
How can I do this?
I am using Debian but I am open to any suggestions.
Last edited by neopandid; 02-12-2013 at 11:14 PM.
Reason: Info added
|