LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 08-18-2019, 08:40 AM   #1
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,372

Rep: Reputation: 51
Question Is this a correct behaviour? Edited /etc/hosts, but Firefox still downloads content from a domain


In my current linux computer, i have 2 lines like these in my /etc/hosts file:

Code:
127.0.0.1 gsa.com
127.0.0.1 www.gsa.com
I put these lines there to avoid *all* content from that domain being downloaded in this computer. But i recently noted that a webpage which body contains the lines

Code:
<script
    type="text/javascript"
    src="http://page2.gsa.com/page/show_content.js">
</script>
really *has* content showed in Firefox! I and trying to directly open the address http://page2.gsa.com/page/show_content.js in my computer shows content.

What am i doing wrong? I tested this when i made a big set of those changes.

There is no proxy in use - if that would make any difference.

Last edited by dedec0; 08-18-2019 at 08:42 AM.
 
Old 08-18-2019, 09:07 AM   #2
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
My guess: Your computer gives DNS priority over /etc/hosts. This behaviour can be changed in /etc/nsswitch.conf, if your computer uses it.
 
1 members found this post helpful.
Old 08-18-2019, 09:20 AM   #3
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,372

Original Poster
Rep: Reputation: 51
Quote:
Originally Posted by berndbausch View Post
My guess: Your computer gives DNS priority over /etc/hosts. This behaviour can be changed in /etc/nsswitch.conf, if your computer uses it.
My /etc/nsswitch.conf is:

Code:
# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         compat
group:          compat
shadow:         compat
gshadow:        files

hosts:          files mdns4_minimal [NOTFOUND=return] dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis
Does that line mean it should look in /etc/hosts before doing DNS in a server?

The page

https://www.dummies.com/programming/...tch-conf-file/

says:

Quote:
The order in which you list these keywords determines the order in which the data sources are searched. Thus, if you want host names to be resolved first by the local Hosts file and then by DNS, you should include the following line in nsswitch:

Quote:
hosts: files dns
The OS is Debian, but searching with its name in query brought the same results.
 
Old 08-18-2019, 10:26 AM   #4
permaroot
Member
 
Registered: Aug 2019
Location: Arden, NC
Distribution: Arch Linux
Posts: 129

Rep: Reputation: 48
You don’t have any type of dns caching enabled do you? And I’m assuming you tried clearing Firefox cache as well as rebooting? Just trying to cover al the bases here.
 
Old 08-18-2019, 10:43 AM   #5
ehartman
Senior Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 1,674

Rep: Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888
Quote:
Originally Posted by dedec0 View Post
In my current linux computer, i have 2 lines like these in my /etc/hosts file:

Code:
127.0.0.1 gsa.com
127.0.0.1 www.gsa.com
I put these lines there to avoid *all* content from that domain being downloaded in this computer. But i recently noted that a webpage which body contains the lines
/etc/hosts works for HOSTnames only, not for full domains, so gs2.gsa.com is NOT covered by your hosts file. And I would put them in a single line
Code:
127.0.0.1 gsa.com www.gsa.com gs2.gsa.com <all other .gsa.com hosts you want covered>
 
3 members found this post helpful.
Old 08-18-2019, 11:03 AM   #6
permaroot
Member
 
Registered: Aug 2019
Location: Arden, NC
Distribution: Arch Linux
Posts: 129

Rep: Reputation: 48
Quote:
Originally Posted by ehartman View Post
/etc/hosts works for HOSTnames only, not for full domains, so gs2.gsa.com is NOT covered by your hosts file. And I would put them in a single line
Code:
127.0.0.1 gsa.com www.gsa.com gs2.gsa.com <all other .gsa.com hosts you want covered>
This is good to know!

So it’s just a “block them as you see them” approach you have to do? Is there a way to find all of their domains with a dig command or anything like that?
 
Old 08-18-2019, 11:15 AM   #7
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,372

Original Poster
Rep: Reputation: 51
Red face

Quote:
Originally Posted by permaroot View Post
You don’t have any type of dns caching enabled do you? And I’m assuming you tried clearing Firefox cache as well as rebooting? Just trying to cover al the bases here.
It is important to cover such things. I would not know how to.

The computer rebooted with both files as they are now. I made no modifications to them since the last boot.

But something weird happened. Follow these details to know my context to use Firefox:

1. I have 4 config profiles for it. Two of these profiles are used practically all the time. Their need is due different settings i use for some sets of websites - and my wish to given them the least possible liberties. The 2 other profiles are for eventual uses, and one of them is used/created/recreated in a folder inside /dev/shm (RAM), so it can be quickly destroyed or anything i eventually want to experiment with it.

2. The profile NOJS has javascript disabled (among other details).

3. The profile JS has javascript enabled (among other details).

4. The profile INRAM is frequently (re)created with all default settings, since i usually do not copy anything to the folder it is configured to be (like copying the NOJS or the JS profile folders contents to it).

With that context, NOJS and JS profiles being used for days, with normal and private windows opened for *both*, i did these steps:

1. I used the INRAM profile, opened the page with those unwanted parts, and they were not there. So your guess found the problem?

2. I deleted *all* files, hidden and normal ones, inside the INRAM profile folder. Started FF again (the command used by my window manager shortcut is /usr/lib/firefox-esr/firefox-esr --ProfileManager --new-instance%u, which shows me the profile choice window) choosing INRAM profile (all other windows of other profiles being left intact), tried to open the *script* URL (which i took from the source code i had opened in a JS profile window), and FF asked me where to save the file - in other words, it found the domain that should be blocked.

3. I tried to open that script URL in the NOJS profile, which uses only pages that can be completely (or acceptably) used without javascript. But it found the script too! I did not expect that.

/-:
 
Old 08-18-2019, 11:26 AM   #8
permaroot
Member
 
Registered: Aug 2019
Location: Arden, NC
Distribution: Arch Linux
Posts: 129

Rep: Reputation: 48
Quote:
Originally Posted by dedec0 View Post
It is important to cover such things. I would not know how to.

The computer rebooted with both files as they are now. I made no modifications to them since the last boot.

But something weird happened. Follow these details to know my context to use Firefox:

1. I have 4 config profiles for it. Two of these profiles are used practically all the time. Their need is due different settings i use for some sets of websites - and my wish to given them the least possible liberties. The 2 other profiles are for eventual uses, and one of them is used/created/recreated in a folder inside /dev/shm (RAM), so it can be quickly destroyed or anything i eventually want to experiment with it.

2. The profile NOJS has javascript disabled (among other details).

3. The profile JS has javascript enabled (among other details).

4. The profile INRAM is frequently (re)created with all default settings, since i usually do not copy anything to the folder it is configured to be (like copying the NOJS or the JS profile folders contents to it).

With that context, NOJS and JS profiles being used for days, with normal and private windows opened for *both*, i did these steps:

1. I used the INRAM profile, opened the page with those unwanted parts, and they were not there. So your guess found the problem?

2. I deleted *all* files, hidden and normal ones, inside the INRAM profile folder. Started FF again (the command used by my window manager shortcut is /usr/lib/firefox-esr/firefox-esr --ProfileManager --new-instance%u, which shows me the profile choice window) choosing INRAM profile (all other windows of other profiles being left intact), tried to open the *script* URL (which i took from the source code i had opened in a JS profile window), and FF asked me where to save the file - in other words, it found the domain that should be blocked.

3. I tried to open that script URL in the NOJS profile, which uses only pages that can be completely (or acceptably) used without javascript. But it found the script too! I did not expect that.

/-:
So are you saying the only way it DOESNT load the unwanted page is if you only open FF with the INRAM profile, with no other FF profile windows having been loaded since boot?
 
Old 08-18-2019, 11:47 AM   #9
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,372

Original Poster
Rep: Reputation: 51
Quote:
Originally Posted by ehartman View Post
/etc/hosts works for HOSTnames only, not for full domains, so gs2.gsa.com is NOT covered by your hosts file. And I would put them in a single line
Code:
127.0.0.1 gsa.com www.gsa.com gs2.gsa.com <all other .gsa.com hosts you want covered>
Awwww... that is a really sad discovery (except for the file organization tip). But it will do, I guess. Let me test it...

Note: i am still puzzled to understand how a brand new profile had not opened the unwanted parts... i will do that test again (from zero, since i immediatelly deleted the INRAM profile files created)
 
Old 08-18-2019, 12:05 PM   #10
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,372

Original Poster
Rep: Reputation: 51
Quote:
Originally Posted by permaroot View Post
So are you saying the only way it DOESNT load the unwanted page is if you only open FF with the INRAM profile, with no other FF profile windows having been loaded since boot?
First, i want the page. I just do not want some parts of it - from a specific domain.

If you meant "no other FF profile windows" of *that* profile, INRAM, that is what i tried to say. But i repeated the test with these commands or steps:

1. all INRAM windows are closed
2. cd /dev/shm/.inram
3. rm -fr * .* > /dev/zero # /dev/zero just to be silent
4. copied the unwanted script URL
5. clicked in the firefox icon with the command i said before
6. doubleclicked in the inram profile to open it
7. opened a private window and asked it to load the copied URL -> worked
8. closed the private window, opened the copied URL in the normal window -> also worked, of course

So, i do not know how to explain what happened in #7 > step 1. It happened differently now. Two times - i repeat the steps i just described here.

So, the conclusion for this thread is "we have to explicitly put all subdomains in /etc/hosts" ? Something with a wildcard is the first basic idea that i am surprised if it does not exist.
 
Old 08-18-2019, 12:13 PM   #11
permaroot
Member
 
Registered: Aug 2019
Location: Arden, NC
Distribution: Arch Linux
Posts: 129

Rep: Reputation: 48
Quote:
Originally Posted by dedec0 View Post
First, i want the page. I just do not want some parts of it - from a specific domain.

So, the conclusion for this thread is "we have to explicitly put all subdomains in /etc/hosts" ? Something with a wildcard is the first basic idea that i am surprised if it does not exist.
Ah I see. I too thought maybe globbing could work.

If your goal is adblocking have you thought of just running a pi hole?
 
1 members found this post helpful.
Old 08-18-2019, 12:19 PM   #12
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,372

Original Poster
Rep: Reputation: 51
As root, i edited the /etc/hosts file, adding ' page2.gsa.com' to the line where gsa.com is. After saving the file, i "cachelessly" (although i am still not sure that my "cacheless firefox bypasses its DNS cache) reloaded the page with some unwanted contents. It was clean.

Unless any of you want to say something else of what i imagined, i may soon consider this thread as closed.

Thanks to everyone
 
Old 08-18-2019, 12:24 PM   #13
dedec0
Senior Member
 
Registered: May 2007
Posts: 1,372

Original Poster
Rep: Reputation: 51
Quote:
Originally Posted by permaroot View Post
Ah I see. I too thought maybe globbing could work.

If your goal is adblocking have you thought of just running a pi hole?
Pi hole? Mmm... much interesting! I never heard about them. But with that solution, i am satisfied, for now.
(-:o

Last edited by dedec0; 08-18-2019 at 02:34 PM.
 
Old 08-18-2019, 12:48 PM   #14
permaroot
Member
 
Registered: Aug 2019
Location: Arden, NC
Distribution: Arch Linux
Posts: 129

Rep: Reputation: 48
Quote:
Originally Posted by dedec0 View Post
Pi hole? Mmm... much interesting! I never heard about them. But with that solution, i am satisfied, for now.
(-
It is used as a dns caching/ad blocking server. You can run it on many versions of Linux, usually on a raspberry pi. It provides network wide adblocking.

http://https://pi-hole.net/
 
1 members found this post helpful.
Old 08-19-2019, 02:50 PM   #15
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,974

Rep: Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623Reputation: 3623
Are you using a proxy?

Simple ping to http://badsite should fail if hosts is correct. dig or nslookup to badsite will fail too.

Use 0.0.0.0 instead of localhost.

I use a good hosts file on almost all systems.
 
1 members found this post helpful.
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
I edited /etc/fstab to practice rescue operation, but I am Stuck it is not mounting AnApproach2DigestLinux Linux - Newbie 2 12-06-2012 10:52 PM
NFS write problem, UID correct, rw in /etc/exports -but still cannot write polgard Linux - Networking 2 09-19-2008 03:14 AM
Verify /etc/hosts usage and round robin in /etc/hosts MikeyCarter Linux - Software 1 06-24-2008 12:20 PM
/etc/hosts, /etc/resolve.conf and /etc/host.conf config probs below_average Linux - Networking 1 12-08-2004 10:07 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 05:12 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration