How to remove forward slash and everything after with sed or awk?
Ok guys, Ive got a list of urls and I need to parse just the domains from them and get rid of all the trailing crap
Ive got a list of these lines like this
And I need to end up with
Also this is a massive list of over 2 million urls, so, its not limited to .com, .ru. .ch, .pl, and .biz. So it needs to implicitly remove the first firward slash and everything after it on each line
Thank you to the wizard who knows the answer in advance.
If your list is formatted with no http:// at the beginning then this should work.
sed 's:/.*::' file
Did exactly what I needed
also I didnt know about the -i option thats awesome,
Hmm I have another more complicated one, any care to have a swing at it?
Ok, so this huge list of sites is like this.
I need a way to remove subdomains, but without destroying items like .co.uk,
So I figure I need to remove anything before .*.* but only on lines that do not contain multiple entries like .co.uk and .co.nz
The whole idea is to have a list of top level domains. But retaining .co.nz, and .co.uk and all similar extensions
Unfortunately, I think you'll need a list of all 2 level domain codes to keep ie your program needs to know if eg '.com.br' is a TLD (ie only want .br) or an allowable 2 level one?
Similarly, given .it is ccTLD for italy, is there an equiv to .co.uk eg .co.it?
For tha I'd use Perl, although others might use awk.
I think sed would be a stretch for this more general problem.
This is a problem where the code needs to know stuff, as opposed to just truncating at a known marker.
|All times are GMT -5. The time now is 09:12 AM.|