[SOLVED] Adblock using /etc/hosts script not working for no obvious reason.
SlackwareThis Forum is for the discussion of Slackware Linux.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Adblock using /etc/hosts script not working for no obvious reason.
I would like to use the following script, but have been unable to get it to work, despite not seeing any obvious reason why it shouldn't. I have previously used it with Debian based systems by using yad and installing the .deb file version of the script. I have installed zenity and libgksu, but with no result in my hosts file. https://github.com/swiftlinuxold/z-o...lock-advert.sh
I have tried this with both Slackware and Salix, but neither seems to work - Slackware using root and Salix using sudo...
Output Slackware 14.1:
Code:
# ./block-advert.sh
Beginning Advert Blocker
./block-advert.sh: line 85: gksu: command not found
Success - Block Advert updated your hosts file.
Output Salix64 14.1 MATE:
Code:
$ sudo ./block-advert.sh
Beginning Advert Blocker
sudo: Advert: command not found
Success - Block Advert updated your hosts file.
Does anyone have an idea how to get this to work on these different systems?
Last edited by globetrotterdk; 03-07-2016 at 10:17 AM.
This script seems to have dependencies on gnome programs (which aren't included in Slackware). Having a quick look through the source, I see at least zenity and gksu.
I don't think the zenity lines will cause you any issues, so you could just try removing the "gksu -D "$title"" from line 85 so it looks like the below, but, I haven't done any testing on this or a more thorough examination of the script, so more may be required (you might need to also remove lines 108-111 and remove the trailing backslash "" on line 107).
Code:
mv "/tmp/hosts-temp" "/etc/hosts"
Why don't you just use a browser based adblock solution like ublock origin?
EDIT: If you do want to keep using hosts file blocking, you can use the extremely simple script provided on this blog post.
Last edited by bassmadrigal; 03-07-2016 at 10:31 AM.
Reason: Added quick alternative
Thanks for the quick reply. Both zenity and libgksu are available in Slackware repositories, which is why I thought that after install, the script should work. Gksu is included by default in Salix OS and it didn't seem to work there either for some reason, despite installing zenity.
You ask why I don't just use an adblocker plugin for my web browser. The reality is that I am almost militant in my desire not to see ads on the web pages that I visit, nor do I want to get all of those "accept the cookies that we are trying to shove down your throat" messages. In my experience, adblocker plugins for web browsers don't always cut it.
Thanks for the quick reply. Both zenity and libgksu are available in Slackware repositories, which is why I thought that after install, the script should work. Gksu is included by default in Salix OS and it didn't seem to work there either for some reason, despite installing zenity.
They don't come on a stock Slackware installation. Did you install the required dependencies for each? gksu requires libgksu which requires libgtop. And zenity requires libgnomecanvas. And did it prompt you for your password when running it? Did you try it with my previously suggested modifications?
But, maybe it'd be worth digging a little more into how the script functions on Slackware, however, I am not at home and am unable to work it on my system (work runs Windows). Taking another glance through, try commenting out line 212 (the cleanup) on the original script (or if you make my suggested changes above, you'd need to figure out which line calls the cleanup function at the bottom), and then afterwards, check the contents of /tmp/adlist* and /tmp/hosts-temp to see the rest of the script is functioning and building the file correctly. It might just be having an issue with the mv command, or maybe it is something prior, like downloading the source lists.
Quote:
Originally Posted by globetrotterdk
You ask why I don't just use an adblocker plugin for my web browser. The reality is that I am almost militant in my desire not to see ads on the web pages that I visit, nor do I want to get all of those "accept the cookies that we are trying to shove down your throat" messages. In my experience, adblocker plugins for web browsers don't always cut it.
Adblocker plugins actually have more flexibility than a hosts list because it can block more than just hostnames. They can (and do) block elements within the webpage or certain subfolders that tend to contain ads (for example, if LinuxQuestions had all their ad stuff under http://www.linuxquestions.org/ads/, you can't use a hostname blocker for that, because you could either block all of LinuxQuestions or allow all the ads in there). Adblockers can also prevent javascript popups that would open to a blocked hostname (I get that on Android quite frequently on certain domains... the actual ads are blocked on those popups, but the hosts file doesn't prevent the popups themselves). It is also handy, because you can block divs that contain the ads, which can remove the space they use on the page.
Just something to consider, but we'll still get this other method working, if that's your desire
You ask why I don't just use an adblocker plugin for my web browser. The reality is that I am almost militant in my desire not to see ads on the web pages that I visit, nor do I want to get all of those "accept the cookies that we are trying to shove down your throat" messages. In my experience, adblocker plugins for web browsers don't always cut it.
I too recommend uBlock Origin, together with uMatrix. A combination every bit as good as, if not better than, adblock and noscript.
Ok, I got home and checked. I created the following patch and if you apply it, the script should work (although, the output is kinda ugly).
Code:
--- block-advert.sh.orig 2016-03-07 17:45:57.734938273 -0500
+++ block-advert.sh 2016-03-07 17:50:49.206273982 -0500
@@ -77,12 +77,12 @@
cat /tmp/adlist-all >> /tmp/hosts-temp
echo "$markerend" >> /tmp/hosts-temp
else
- zenity --info --title "$title" --text "Empty ad list. No change made in /etc/hosts."
+ echo "Empty ad list. No change made in /etc/hosts."
exit 1
fi
# replace the original /etc/hosts with the new version
# require root privilege
- gksu -D "$title" mv "/tmp/hosts-temp" "/etc/hosts"
+ mv /tmp/hosts-temp /etc/hosts
}
@@ -103,12 +103,7 @@
# sed to delete the ending '%' sign
# sed -u to flush the output of sed
# changed -t 0 (tries) to -t 20
- wget -c -4 -t 20 -T 10 --progress=dot -O $2 "$1" 2>&1 | \
- awk '{print $7}; system("")' | sed -u 's/%//' | \
- zenity --title "$title" --progress --width $WIDTH \
- --text "Loading adlist from $domain" \
- --percentage=0 \
- --auto-close
+ wget -c -4 -t 20 -T 10 --progress=dot -O $2 "$1" 2>&1
}
# download the ads lists
(This is not a very pretty script and has a bunch of useless commented out stuff... which is probably why the github author has it listed under their obsolete folder.)
Last edited by bassmadrigal; 03-07-2016 at 04:42 PM.
They don't come on a stock Slackware installation. Did you install the required dependencies for each? gksu requires libgksu which requires libgtop. And zenity requires libgnomecanvas. And did it prompt you for your password when running it?
I was not prompted for my password as I have been in the past. Good to know about the deps. I am missing some of those as well.
Quote:
Originally Posted by bassmadrigal
Adblocker plugins actually have more flexibility than a hosts list because it can block more than just hostnames. They can (and do) block elements within the webpage or certain subfolders that tend to contain ads (for example, if LinuxQuestions had all their ad stuff under http://www.linuxquestions.org/ads/, you can't use a hostname blocker for that, because you could either block all of LinuxQuestions or allow all the ads in there). Adblockers can also prevent javascript popups that would open to a blocked hostname (I get that on Android quite frequently on certain domains... the actual ads are blocked on those popups, but the hosts file doesn't prevent the popups themselves). It is also handy, because you can block divs that contain the ads, which can remove the space they use on the page.
Interesting. I would have thought that modifying /etc/hosts would "hard code" the ad blocking into the system. The plugins don't seem to be effective for very long before a way around them is developed. Just today I was reading about anti ad blocking scripts that Google and others inject into pages where they want ads to appear.
Quote:
Originally Posted by bassmadrigal
Just something to consider, but we'll still get this other method working, if that's your desire
Thanks. I am a bit under the weather today, so I will try and take a look at all of your script changes tomorrow.
Interesting. I would have thought that modifying /etc/hosts would "hard code" the ad blocking into the system. The plugins don't seem to be effective for very long before a way around them is developed. Just today I was reading about anti ad blocking scripts that Google and others inject into pages where they want ads to appear.
All it does is prevent access to those domains completely. You can do the same thing with just about any adblocker browser plugin, but they'll include extra features. I've been really happy with ublock origin, and its default blocklists (that it keeps up to date itself). You can pick additional blocklists, add your own custom blockings if needed, and create whitelists for things that may be accidentally blocked.
One thing you could do if you dual boot Windows. Get a copy of Spybot Search and Destroy, have it immunize the system, and then copy paste the blacklisted URLs and IPs from it to your Linux hosts file. It's cheesy, but it's effective.
Here is some information I found about using /etc/hosts files: http://pgl.yoyo.org/adservers/
The rational appears that using the /etc/hosts file blocks the entre domain, in a similar fashion to a name server:
Quote:
You can of course also just put the contents of the list in hosts file format in /etc/hosts, and it'll have the same effect as running a local nameserver (as long as the machine is configured to check /etc/hosts of course).
Quote:
Using the list of ad servers with a name server is one of the most efficient methods. It allows you to block entire domains, rather than just individual host by configuring your nameserver to act as master for the zone. For example, by setting your DNS server to be master for "doubleclick.net", all requests for hostnames under "doubleclick.net" will fail, instead of having to add entries for "ad1.doubleclick.net", "ad2.doubleclick.net", etc.
I am trying to learn more about this issue, so I am curious as to whether plugins can do the same thing or they only can deal with sub domains?
Also, if the entire domain is blocked, wouldn't that block anything emanating from that domain including pop-ups, scripts, etc?
Last edited by globetrotterdk; 03-08-2016 at 02:04 AM.
Yep. If it's blocked, any traffic trying to access it gets rerouted back to the 127.0.0.1 dead zone.
So using the /etc/hosts strategy could be described as more effective, efficient and Unix like than using a browser plugin, or does it come down to the same thing in practice?
Last edited by globetrotterdk; 03-08-2016 at 02:32 AM.
Hosts file: works on a system level, no need to modify your browser(s) in any way, huge filter files.
Browser plugin: per browser settings, able to use regular expressions (much smaller filter files), much more flexible.
Considering the fact that both methods use the same lists, the second one is more convenient. +1 for ublock origin.
I am trying to learn more about this issue, so I am curious as to whether plugins can do the same thing or they only can deal with sub domains?
Also, if the entire domain is blocked, wouldn't that block anything emanating from that domain including pop-ups, scripts, etc?
You can block both domains and subdomains with both browser plugins and hosts blocking. There is no argument there. In fact, hosts blocking would probably be a bit more efficient, since it is at the system level rather than a browser plugin (although, on our current computers, you might not notice the difference). Blocking the domain will block anything that might come from the domain (including pop-ups, scripts, etc). But, that doesn't prevent the parent site to run those things. My previously mentioned example on Android is a prime example. The site I visit has code that generates a popup, and that popup points to an ad site. Since the popup is generated by the site I'm visiting the hosts blocking does nothing to prevent it from getting generated, however, it does prevent the content from loading in the popup. But the hosts blocking does nothing to prevent the annoyance of the popup itself, even if I don't view the actual ad.
Hosts blocking limits you to just blocking domains and subdomains. That's it. You can't block anything extra, like subfolders or webpage elements. I suppose if you're used to it, it may not seem very limiting, but after using heavily customized filters with browser plugins, hosts-only blocking seems really limited (and it is a common frustration I experience on my Android phone since I haven't found a better adblocking experience than using hosts blocking -- personally, Firefox is an annoyance on Android because of their idiotic scrolling implementation).
Ad content can exist on domains that have other uses (like the Akami content delivery network). If you block akami.net in your hosts file, you won't get any ads served by them, but you also won't get anything else served by them (unless things have changed, Yahoo relies heavily on akami for content). You're more likely to see this type of thing on smaller sites, as many will use something like www.mylittledomain.com/ads/ to house all their ad code. Just imagine if Google implemented that... if all their ads were served under google.com/ads, you wouldn't be able to block ANY of them with using your hosts file without blocking all of Google.
Another big benefit that I personally use a lot is being able to block individual elements on a webpage. A common thing I will block is those stupid video players on some sites that start automatically (usually news sites), but you can also block other webpage elements, like the "viral pictures" box on imgur, or the boxes that say "advertisement" (which are hopefully empty now, with your hosts blocking, but the box still exists and takes up space). With hosts blocking, you also can't block IP addresses, since ad content will sometimes just be referenced by IP rather than a domain name.
Finally, using the patched script, I get ~25k entries in the hosts file. On the default install, using the default adlists, ublock origin has 53,919 network filters and 38,148 cosmetic filters. This is using only 10 of the 28 optional lists (including several hosts lists), so enabling more would block a lot more (which increases the chance of false positives, but it is really easy to whitelist sites if needed).
Overall, it is certainly your choice, but for me, I wouldn't limit myself to just hosts blocking. Good luck!
EDIT: One thing I forgot to mention, you can use both methods without issue. So you're free to continue using the hosts file while installing the browser plugin. They won't conflict with each other (however, two browser plugins will likely conflict with each other and cause excessive memory consumption).
Last edited by bassmadrigal; 03-08-2016 at 08:34 AM.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.