How To: Block All Internet Ads in Any Web Browser
 

Hello, everyone. I'm visiting over from the Ubuntu forums, and I'd like to share a quick tutorial about blocking malicious software in any browser whatsoever. :)

Firefox has a great add-on called AdBlock, which blocks (you guessed it!) ads. But, not everyone uses Mozilla Firefox. This isn't really fair to the Opera users, Midori users, etc.

So, how can you block ads with these browsers? Simply use your /etc/hosts file.

The /etc/hosts file is the predecessor to modern DNS. It is a local mapping of IP address to hostname. At this time, since we more or less rely on DNS, the /etc/hosts file is probably pretty empty. You’ll likely just have something along the lines of:

So, how are you supposed to block all of the internet's advertisements if the /etc/hosts file looks like that?

Well, you're not, if it only contains that, which it likely does.

Oh well, right?

Wrong!

I have managed to find an /etc/hosts file that is 15,169 lines long, without comments.

...

:eek:
That's a ton of mapped IP addresses! What the original author of the file has done is compose every bad thing he was able to find into the hosts file, and map it to 127.0.0.1. (Alternatively, you can map it to 0.0.0.0)

Do you see the genius in this?

If not, I'll explain.

When your browser is told to display an ad it needs to look up the address. The /etc/hosts file is checked before DNS, which will go on to tell it to ask the local machine. The local machine, of course, does not have that information to display, so, simply, nothing is displayed.

WHOOOOOOOOOOHOOOOOO!

And guess what? This doesn't only apply to being able to block ads, but also ad banners, 3rd party page counters, 3rd party cookies, web bugs, and even most hijackers. You’re not only blocking ads anymore, you’re simply just blocking thousands of known malicious websites.

That's right.

And this works with every browser.

So, without further ado, here is the script to end malware.

I am not the original author of this file, but it has been published under a CC-BY-SA license and I am therefore allowed to redistribute it. For more information, check out the information contained in the file.

Open up your favorite shell, and run

These commands will firstly back up your /etc/hosts file, and then wget the contents to fill the new /etc/hosts.

No restart required, you'll be ready to try it out immediately.

Enjoy your ad-free browsers. I hope that I've helped everyone! :)

//moderator: links judged to be good for removal even in the event of being reminded to be cautious.

My understanding is having a huge hosts file slows down address lookups since every time you enter an address, your browser has to scan through 15,000 lines before realising www.google.com isn't a dodgy address. I may be wrong on this, but that is my understanding.

I prefer privoxy - a local filtering proxy which I believe filters based on a series of regular expression-based rules rather than this brute-force approach.

To each their own, I suppose.

If you want to do the same thing without having the hosts file slowdown, dnsmasq offers similar functionality and it seems to be pretty snappy. Also, it allows you to use DNS for your LAN to boot!

For me , this is not the purpose of the host file, use firefox with plugins, or dsnmasq

Using /etc/hosts for redirecting lookups like that is a crude, pre-Junkbuster era method of dealing with things. It is crude because it is a static listing and because it only deals with domainnames. it can't parse code to resolve Javascript fun nor can it do path-based filtering. Opera, in earlier incarnations, already had the possibility of creating a "filter.ini" for regex-based exclusions built in. The current Opera-9.x allow you to fine-tune Javascript and other things on a site-by-site basis, again built in.

Networked applications that don't incorporate a resolver rely on Glibc (stub resolver) to do the resolving for them. Related to that is the nscd which caches requests to some extent. This still is likely to be far less performant than using a persistent caching nameserver like say Pdnsd. On top of that Pdnsd allows you to reload configuration files on the fly, so for instance a wee BaSH script to parse and update entries from say malwaredomains.com or malware.com.br is easily made. For blocking domainnames I use Pdnsd and Privoxy on top of that (I agree, way more versatile method) for its regex-based filtering.

I would use:
http://www.privoxy.org/

I use the host file from http://www.mvps.org/winhelp2002/hosts.txt as well as AdblockerPlus in iceweasel.

I haven't noticed any slow downs surfing the web. Screw ads and tracking cookies.

I think this is a very quick and easy way to block a lot of crap. Are there better ways? No doubt. Are they as easy as replacing one file in your system? Heck no. :)

FYI, the site-by-site JavaScript filtering that you mentioned in Opera unSpawn is expanded in the NoScript add-on for Firefox. It filters JavaScript, Java, Flash, and so forth on a site-by-site basis. It only works in FireFox though, and the rules are on a per-machine basis.

I like this approach because it filters your entire network in one fell swoop and would be extremely easy and fast to set up. Thanks for sharing BslBryan.

Easy yes, crude even more so. Maybe reread the reasons against using it more closely. And what makes you think I don't know about NoScript?..

I have read and understand the reasons for not using it. I still think it's very nice of the OP to post in such detail how to do it, along with a link to an appropriate hosts file. I also believe that an awful lot of people (noobs especially) would be willing (and able) to replace one file on their system to filter out crap for all the boxes being fed by the linux system no matter what browser used rather than install and configure software to do so.

Would I use this method when setting up a production box? Probably not. Would I if I were pinched for time and just needed to get something semi-effective up and going ASAP? Sure.



Maybe because you didn't mention it while devoting half of your first paragraph to such a feature present in Opera??? :)

FYI, I view posts on here as not only helping inform the person you're directing the comment to but also the community at large. If somebody does not happen to know about NoScript and reads this thread, they will because of what I posted and I accomplished something good. You don't have to get defensive; I was merely pointing out that such functionality is easily available for FireFox, a much nicer browser than Opera IMO.

How does this pertain to "security?"

Personally I think using a hosts file is an obsolete method of doing this. There are many better ways of doing it.

If you are using a package based system, then installing privoxy is as easy as installing the package, and changing the path to the web to point to it.

The default privoxy configuration works pretty well right out of the box, and its flexibility is far, far greater than a hosts file can give you - and it doesn't slow the system down, and it can be deployed to guard an entire network.

Further, if your protection DOES give problems with a site, it is trivial in most browsers to change to a direct connect, thus bypassing privoxy, if that is necessary to deal with some specific issue. With a hosts file, you have to point to a different hosts file.

Anyone who wants to continue to defend the old-fashioned and very crude hosts file means of blocking is welcome to do so, but I am sure there are still a few people out there who think the Model A Ford was the absolute pinnacle of automotive technology, too.

A) Simply downloading and copying a single file is much quicker, easier, and less intimidating than installing software for many people.

B) I don't believe I was defending this method, only pointing out that it has it's uses. It is a good method for certain circumstances, just like everything.



My apologies for trying to thank and be appreciative to the OP for something many people may not know. I learned from him that there are hosts files out there that people have built to streamline this type of thing and am grateful to him; he allowed me to add one more tidbit of junk to my already overflowing brain to pick out at an opportune time in the future. From now on, when somebody visits from another forum to share information, I'll not say anything in response. Please, continue saying how worthless his post is and how much the method presented sucks - I'll no longer hinder your efforts to make him not want to post anything that might be helpful to somebody.

And you need to climb off your high horse.

No one said it was "worthless". Everyone said: "there is a much better and more modern way."

I'll just repeat it. "There's a much better and more modern way."

And you also - before you rant - should take the time to look at who "everyone" in this context is. To put it simply, the weight of experience on this board is all saying: "there's a much better and more modern way".

Learn, or not. Your choice. But do NOT think you will get away with misquoting or misrepresenting what we say or said.

Wow, I didn't mean to start a flame war. Sorry, guys! :o

Thanks, Ruler2112 for your kind words, and to everyone else, yeah, there are certainly easier ways to block ads. ;) I just found this hosts file and thought it was neat.

I might add in a little disclaimer that it's only for knowledge. I'm not recommending this tutorial, because there really are simpler ways of doing this, but there certainly isn't any harm in posting it here, or even following the steps.

Also, some people might find this comes in useful if they are using a very new, unstable browser that has no add-ons, etc. :)

EDIT: Just realized that the link to the hosts file has been modded out. :(
Anyway, thanks for reading, and I'm glad most of you seemed to enjoy it. I suppose it really is worthless now. :)

Since Adblock was mentioned I'll throw in that the Untangle Gateway has adblock now, so it can do ad blocking for your entire network while it's sitting at the gateway. Plus all the other nifty features of the UTM device..

cool
 

I'm definitely one for knowing how to do something. many ways. This is yet another way to blacklist ip addresses from toturing you. Thanks for the post!

Not really... name lookups will have to do this anyhow for new domain names. For other, cacheable names, the browser will remember. For non cacheables (ads for instance) it will have to do the lookup every time, either in the hosts file or via DNS.

And you're right, it is...

I, for one, am not a fan of loading layer upon layer of software on my network stack and/or browser. I really have no interest in privoxy, adblock and so on... they cause sluggishness, IMO.

This, simple, lightweight addition gives many of the benefits with little of the bloat.

I'm betting a hosts file is a lot slower actually.

Since you say "gives many of the benefits" please explain how you would do this / block these using only your precious "simple, lightweight addition":
- incrementally update your blocklist,
- block ad-tracking cookies (for those using Firefox have a look at the www.ghostery.com plugin),
- block in-page ads residing in a path on the same server you visit,
- block ads from a hostname of which the domainname is the same as the server you visit,
- block ads presented through Javascript or Flash,
- block ads by host or path substring match,
- block only webbugs (you know, those tiny 1 pixel transparent images),
- set session-only cookies for a range of sites,
- selectively block popups, refresh-tags and redirects,
- keep images with specific sizes from displaying,
- block visiting domains based on content (like those parked sites).
Well, actually you don't need to try because you can't.
As such your "many of the benefits with little of the bloat" phrase can not based on reading (and actually understanding) what's been said in this thread already.


Also it turned out that the owner of the site where the hosts file resided is one of the Ubuntu people. That particular "hosts tutorial" got plastered over several web logs in the month it got posted here. Since this is not some radically new idea and the site owner does not make clear the content is not his but originates from mvps.org it must have been done purely for promotional purposes. He should have either linked to it or made clear it's not his work. File under plagiarism.

...and since this isn't about Linux Security I'll move it elsewhere.

I have a suggestion.

"I'm betting a hosts file is a lot slower actually."

I have and still use some pretty huge hosts files. There were some versions of OS's and some browsers that didn't work correctly but they are all now faster than a dns lookup is (assuming your hard drive isn't from 1990).

To reverse the issue, you can put a hosts file with only the few sites that you use. Don't use any dns ip. There are some ad's that are on an ip so they will show. Not sure you can fully block all ads ever.

Sad we have to pay for this but it is also a choice to consider. http://www.opendns.com/home-solution...ntal-controls/

using the hosts file this way is a very old trick. It still has it's uses but once it grows too large it can contribute some lag. And of course there are things it can't catch. But as a means of blocking specific IP's or host names it's fairly efficient, especcially for a short list.

I do have to chuckle at someone actually applying a licence of any kind to this as these lists have been around the net forever and are virtually all in the public domain. Trying to licence a particular list (which is undoubtedly a collation of other such lists strikes me as a bit over the top.

For many years I only could get a very slow dial up connection. It was usable on bulletin boards if you didn't mind paying long distance charges every night for a week to get a few floppies worth of data. When the internet took over I still had to use a text based browser. If you really want to speed up things you can try them. Once you get used to it, you wonder what the big deal is with other forms.
It may be that a person born into the web can't easily use a text based browser but at least try it.

FCOL not again...
 

Easy to test.


Yeah, likely the stuff you wouldn't encounter the past millennium.


http://www.linuxquestions.org/questi....php?p=3585760

I view the subject of ad's or unwanted content on a web page the same way I view OS/application security. I say my OS will be subject to an attack at some point no matter what I do. I can only limit my exposure. If you feel you have found some way to block content then there will be some new means to send junk to your browser. The worse offenders rely on new found holes in security. The really bad ones might get past any controls, because they know someone is trying to prevent them.

Post the suggestion.

Close the thread, it's old and there are, 3 years later, better and easier (for some) ways of doing what the OP tried to tell people about.

I had one, and it never ever slowed my browsing experience down (dial up did that for me) that was way over his 15k lines. I even had a post on Ubuntu forums asking how to remove duplicates from it. Hosts files have their place and so do other tools, educating the masses about each and every one so they can choose for themselves is the key. I don't use it anymore but learning about them helped me to become more knowledgeable of what happens as soon as you open a browser and what continues to happen as you use the net.

Sure, but there's a difference between just "believing" ("I think", "don't worry", "I guess") something is secure and proper hardening and regularly auditing exposure. And since you're linking this to security maybe you can understand that relying on /etc/hosts is just like valuing Psionic PortSentry over Snort. PS is everything Snort is not: of limited use, inefficient, prone to FP's, obsolete, no longer maintained, etc, etc. But just like there's still people stubbornly advertising /etc/hosts as an efficient and accurate method of blocking ads there's still people stubbornly installing PortSentry, even in 2012. Part of that may be due to people not realizing that some HOWTO's on the 'net are utterly deprecated and part of that may be due to, what shall we call it generally speaking, some form of perverse ignorance?

I just can not understand why, with all that we can use these days, people still maintain blocking IP addresses is "good enough" when every simple test shows it just is not...

It's still maintained the hosts at this site,
I'm lazy and I use it on Windows machines of clients and some friends which tend to get full of viruses within a week from clean install.
It gets rid quite a lot of unwanted malware (but I never tested the speed because of that large hosts file).

Well, to be exact the OP was talking about blocking ad's. The scope of security was not part of that question. One can do a number of tasks to help secure a system. Anyone who thinks they can fully secure a system from hackers and connect it to the internet may be a bit out of step with reality. One can greatly limit vulnerability but I have not seen in my 40 years of technical work any OS that is immune.

A hosts file is only a part of a multi-part solution for ad's and some malware sites.


One could try the removal of dns and use a host file for only their common web pages. But a complete blocking of all other ip addresses is still needed and all other security features need to be up to date.

Some of my favorite web pages have ad's that I am interested so there is good and bad to this. One might find out about a free unix book or such from an ad. They are not all bad.

ROTFL. Actually it was you who started talking about security, likening it the way your did:

...then all I did was try to fit in a comparison you might be able to understand.


...and since this thread isn't about security what are you trying to accomplish talking about it anyway?

Well, since the thread is 3 years old - as previously noted - AND things may have changed since then AND the ongoing discussion seems to lead nowhere, let it rest in peace. Closed.