I have three domains registered for the static IP we have from our ISP: 'family.org', 'hobby.org', 'hobby.com'. I'm setting up two virtual hosts for each of those domains: 'www.family.org' and 'test.family.org', etc. The 'www' sites are hosted on one server ('Prod') while the 'test' sites are on another (my desktop system). All the virtual hosts are listening on port 8080. Nginx is in the mix, too, not only as the Nginx service but as the internal DNS service. The network's firewall is routing all port 80 traffic to the Nginx system where is it is forwarded to the 'Prod'/'Test' servers via port 8080. The attached drawing shows what I'm trying to configure. At the moment, external connections to the 'prod' websites are working. (Ultimately, whatever solution gets implemented needs to make sure the 'test' sites are not accessible from the Internet.)

The internal network is currently set up as 'family.net'. I have no resistance to changing that to something else, say, 'home.net' if it helps make things less ambiguous and eases debugging, maintenance, etc. There aren't that many hosts that would need to be modified and rebooted.

My aim is for users to be able to access the test sites using, for example, 'http://test/family.org' and the 'prod' sites by issuing 'http://www.family.org' without having to append ':8080'. My understanding from reading a few articles in the internet is that Apache virtual hosts should be listed in the DNS zone as CNAME entries/aliases to the host that they reside on. That sort of makes sense but that, I think, assumes that the web servers are listening on port 80. My thought was to point DNS entries to the Nginx server so that my desktop browser port 80 connections would look like they came from the internet/firewall. So... I added the CNAME records to be associated with the Nginx server (port 80 in --> port 8080 out). Well that's not working.

Some test results using 'lynx' to try and access the sites internally:

(Man... debugging was so much easier when there weren't so many bozos intercepting bad web requests and popping up domain registration ads.) All of those look like the current DNS settings that are intended to handle these virtual hosts are badly broken.

Looking at the 'host' results... they're totally weird. From the DNS server, I can't resolve host names for anything. From the firewall, I get strange things like:
But I can ping 'www.family.org' and the other virtual hosts since, in the end, the IP address is correct.

From all other hosts on the internal network, 'host hostname' results cannot be obtained (NXDOMAIN). Reverse lookups work just fine. Likely a missing or misplaced '@' or '.'. At one point, trying to do lookups for the 'www' virtual hosts returned the static IP on the firewall. Commenting out the CNAME records for the virtual hosts fixes all the 'host' command problems. (Though, obviously, no lookups for the virtual hosts is possible.)

Q: Just what is the correct way to add virtual hosts to the DNS files?

One other thing I've just tried is to ensure that my desktop system has 'files dns ...' as the order in '/etc/nsswitch' and adding the virtual host names as aliases to a record for the Nginx server in /etc/hosts. I can reach all of the 'test' virtual hosts but I'm not able to reach any of the 'www' virtual hosts---if a connection is made to Apache at all, it's to the default virtual host's page. I'll double check the virtual host configuration on the 'prod' Apache server. 'nmap' reports that 8080 is listening and `apachectl -S' show that servers are all listening on '8080'. I'll try using 'curl' to push some requests into that Apache system to see what I get back.

Note: I'm posting this with a warning of sorts that I'll likely be unable to respond with any additional information someone might request as I'll be away from these systems for a couple of days. (There should be internet access where I'm going to be, so I can respond to questions, but posting any files from the systems won't be possible.)

Thanks in advance for any hints, suggestions, etc.

Attached Thumbnails

 

Packet routing doesn't look at names or ports, just IP address. If there is a direct route from test to prod, then the packet won't go through the proxy. Check routing first with traceroute.

Once routing is correct, look at name resolution. Its kind of confusing to mix hosts files and DNS. If DNS should work then get rid of the hosts entries. Use dig to test. If that still isn't working then post your zone files.

I haven't used nginx as a DNS server, but only as a reverse proxy. I've been using bind9 to provide DNS service: it is capable of a "split view" configuration, and responds differently to requests from external internet vs requests from local network.

When I add a virtual host, at my registrar, I point it toward my DNS server's public IP. Then, on the DNS server, I add a zone file in /etc/bind/internal, and another in /etc/bind/external. Now if I am already connected to my LAN and make a DNS query, the server returns a local ip; and if I'm connected to some random hotspot, dns query is answered with external ip.

Your desktop is just another backend http server. Have nginx forward requests for test.family.org to your desktop, and requests for www.family.org to Prod.

And I agree with smallpond: since the bind9 server can answer internal DNS queries, I don't need to maintain entries in the hosts file for every device on a LAN.

I don't use CNAME records that often: only when I want there to be a "synonym" for the host name... like if I had a domain for my wife: christine.site.tld and I wanted it also accessible from christy.site.tld and from chris.site.tld, since she goes by all three names (her family still calls her Christy, but she goes by Chris)... One of these labels would get the A record, and the others would have CNAME records.

The A records just point to the nginx server, which then forwards the request to the appropriate backend depending on the name. The CNAME records just point to A records.

Yep. That's one reason Nginx came into the picture: 80->8080 translation. (The other being getting ready to start the process of setting up with CertBot and moving sites from 'http' to 'https'. Recommendations for leisure reading related to that process are welcomed.)

Without CNAMES record in place and aliases in /etc/hosts/, traceroute went to the default route and directly out onto the internet, hitting a couple of different places in two continents (whoa... Brazil?), before quitting. Re-enabled the CNAME records... and [I]we're working... internally... mostly.

Accessing the sites from outside the firewall is working once I tweaked the CNAME records for the domains. (To add 'www.domainname' back in as a CNAME to the 'domainname'.

The mostly working part? If accessing a page on a site internally that contains a link to another of the virtual hosts, following those links fails. Those links work when accessing the page externally. Nginx doesn't appear to be seeing the packet from the 'prod' web server when following the link and I get 'cant' find' errors in the browser. (Testing was done by adding a rule in the Nginx system's iptables INPUT chain to log the packets received from the 'prod' server when clicking on the links that fail.) I find it odd that packets that are being received from the 'prod' server are from port '8080'. I changed everything to port 80... that didn't do anything. Same problem on the internal sites. It's looking like there's something 'odd' about the configuration on the 'prod' Apache side. 'prod' cannot reach other 'prod' sites. 'test' can reach other 'test' sites but not their 'prod' counterparts. I haven't tweaked the DNS to move the CNAME records from the Nginx system to their respective Apache servers. Yet.

Not sure if it makes any difference but the Apaches are slightly different versions: test = 2.4.33, prod = 2.4.41. According to 'apachectl -V', they've been compiled with the same options. I'm tarring up the configs for both servers and taking them with me to see if any changes crept into one site's configs that might be causing the difference in behavior. I can't imagine a point release would introduce a significant change. I see if I can track down release notes for the different versions (.34 -> .41)


I wasn't using both at the same time. Things are messy enough.

Not going to be able to address this any further until the weekend (travel). More updates then.

Thanks for the feedback...

What I referred to as the 'Nginx server' isn't trying to do anything DNS-related with Nginx. (Nginx can do a lot but I can't imagine how that could even be done.) It's also running the Bind9 service for the LAN. I'll have to look into the 'split view' configurations. My first thought is that my efforts should probably be concentrated there.

That's a different configuration than I'm using. DNS is (currently) all for the internal network. Externally, DNS is handled by Domain/DNS provider. The internal DNS forwards to them as needed.

That's what it's doing.

As I mentioned above, they weren't being used simultaneously: I commented out the CNAMEs when testing with the hosts entries and vice versa. Currently, the /etc/hosts entries have been removed entirely.

I have (essentially):
I'd have to go back and track down the link to the article that suggested using that for virtual hosts. When I get back, I'll move those CNAME records to the appropriate Apache servers and test again.

Thanks for the feedback. I appreciate the pointers.

(Hmm... not sure why I chose '123.123' for those last two octets; should have been '13.2'.)

After beating this configuration to death and getting nowhere -- I thought i'd ironed all the kinks until I realized that while I could ping everything from anywhere I still had trouble issuing 'host' commands for much of the network without getting 'NXDOMAIN' responses, dig was not coming up with 'ANSWER' sections, etc. -- I reconfigured the DNS zone files. Now, instead of using the CNAME records that pointed back to the Nginx server (as shown above), I created zone files for each of the domains used for the virtual hosts -- 'db.family.org', 'db.hobby.org', and 'db.hobby.com' -- and the necessary 'stanzas' in 'named.conf.local' to load them. These are, essentially, slightly modified versions of the 'db.family.net' zone file with all the 'A' records removed except:
All three new zone files have these exact same records. They all still point back to the Nginx server but no CNAMEs are used.

I expect that, should I decide that each of those sites needs a forum, I could merely modify each new zone file to contain:
and I'd be all set. (I'll try this for, um, 'fun' and verify.)

I have no idea why the use of CNAMEs was causing such a problem. I still haven't re-located the article that suggested their use. I'm wondering if it was rather dated and relying on something that more current software no longer supports.

Update: Found it. It was a five-year-old StackOverflow post. Probably out-of-date for today's software.


For now, things are looking up. (Just a little Bind 9 humor there.)