I am trying to configure (3) DNS servers to allow lookups between multiple subnets/locations.

dns01 = master located in datacenter1 CentOS 6.3 using named
dns02 = slave located in datacenter1 CentOS 6.3 using named
dns03 = stand alone located in datacenter2 CentOS 6.3 using named

dns01 & dns02 are located on the 10.1.10.0/24 subnet
dns03 is located on the 10.2.10.0/24 subnet

both datacenters are connected to one another via a vpn tunnel

All routing between datacenters works great and I can ping and ssh etc. between all hosts in each datacenter. DNS lookups for each datacenter work fine as well, it's only when I try to go between datacenters that I cannot resolve host names. Hosts in datacenter1 cannot find hostnames in datacenter2 and vice versa.

Both datacenters use the same domain name of let's say business.net

How do I configure DNS (named) to allow host lookups between both locations/subnets?????

I can post my named.conf and forward and reverse zones for each server if needed.

Yes please post...


Few things can be wrong...

1) You need to "allow query" in the named.conf file possibly
2) You need to open ports 53 tcp/udp on the hosts firewall
3) You need to be listening on the IP address and not the loopback (again in the named.conf file)

-C

Here are my configs:
dns01 /etc/named.conf

//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//

options {
listen-on port 53 { 127.0.0.1; 10.1.10.201; };
listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
allow-query { localhost; 10.0.0.0/8; };
allow-transfer { localhost; 10.1.10.202; };
forwarders {8.8.8.8; 8.8.4.4; };
recursion yes;

dnssec-enable yes;
dnssec-validation yes;
dnssec-lookaside auto;

/* Path to ISC DLV key */
bindkeys-file "/etc/named.iscdlv.key";

managed-keys-directory "/var/named/dynamic";
};

logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};

zone "." IN {
type hint;
file "named.ca";
};
zone"business.net" IN {
type master;
file "forward.business";
allow-transfer {10.1.10.202; };
allow-update { none; };
};
zone"10.1.10.in-addr.arpa" IN {
type master;
file "reverse.business";
allow-update { none; };
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

dns01nyc /etc/named.conf

//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//

options {
listen-on port 53 { 127.0.0.1; 10.2.10.201; };
listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
allow-query { localhost; 10.0.0.0/8; };
allow-transfer { localhost; 10.2.10.1; };
forwarders {8.8.8.8; 8.8.4.4; };
recursion yes;

dnssec-enable yes;
dnssec-validation yes;
dnssec-lookaside auto;

/* Path to ISC DLV key */
bindkeys-file "/etc/named.iscdlv.key";

managed-keys-directory "/var/named/dynamic";
};

logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};

zone "." IN {
type hint;
file "named.ca";
};

zone"business.net" IN {
type master;
file "forward.business";
allow-transfer {10.2.10.1; };
allow-update { none; };
};
zone"10.2.10.in-addr.arpa" IN {
type master;
file "reverse.business";
allow-update { none; };
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

forward and reverse for dns01

$TTL 86400
@ IN SOA dns01.business.net. root.business.net. (
20140117 ;Serial
3600 ;Refresh
1800 ;Retry
604800 ;Expire
86400 ;Minimum TTL
)
@ IN NS dns01.business.net.
@ IN NS dns02.business.net.
;10.1.10.0/24-IPAddr
@ IN A 10.1.10.1
@ IN A 10.1.10.2
@ IN A 10.1.10.11
@ IN A 10.1.10.12
@ IN A 10.1.10.201
@ IN A 10.1.10.202
;10.1.10.0/24-hostnames
firewall IN A 10.1.10.1
switch01 IN A 10.1.10.2
esxi01 IN A 10.1.10.11
esxi02 IN A 10.1.10.12
dns01 IN A 10.1.10.201
dns02 IN A 10.1.10.202


$TTL 86400
@ IN SOA dns01.business.net. root.business.net. (
20140117 ;Serial
3600 ;Refresh
1800 ;Retry
604800 ;Expire
86400 ;Minimum TTL
)
@ IN NS dns01.business.net.
@ IN NS dns02.business.net.
@ IN PTR business.net.
;10.1.10.0/24-IPAddr-ARecord
firewall IN A 10.1.10.1
switch01 IN A 10.1.10.2
esxi01 IN A 10.1.10.11
esxi02 IN A 10.1.10.12
dns01 IN A 10.1.10.201
dns02 IN A 10.1.10.202
;10.1.10.0/24-IPAddr-PTRRecord
firewall IN PTR 10.1.10.1
switch01 IN PTR 10.1.10.2
esxi01 IN PTR 10.1.10.11
esxi02 IN PTR 10.1.10.12
dns01 IN PTR 10.1.10.201
dns02 IN PTR 10.1.10.202
;IPAddr-Hostnames
1 IN PTR firewall.business.net.
2 IN PTR switch01.business.net.
11 IN PTR esxi01.business.net.
12 IN PTR esxi02.business.net.
201 IN PTR dns01.business.net.
202 IN PTR dns02.business.net.

forward and reverse for dns01nyc
$TTL 86400
@ IN SOA dns01nyc.business.net. root.business.net. (
20140101 ;Serial
3600 ;Refresh
1800 ;Retry
604800 ;Expire
86400 ;Minimum TTL
)
@ IN NS dns01nyc.business.net.
;10.2.10.0/24-IPAddr
@ IN A 10.2.10.1
@ IN A 10.2.10.11
@ IN A 10.2.10.12
@ IN A 10.2.10.201
;10.2.10.0/24-hostnames
firewallnyc IN A 10.2.10.1
esxi01nyc IN A 10.2.10.11
esxi02nyc IN A 10.2.10.12
dns01nyc IN A 10.2.10.201

$TTL 86400
@ IN SOA dns01nyc.business.net. root.business.net. (
20131011 ;Serial
3600 ;Refresh
1800 ;Retry
604800 ;Expire
86400 ;Minimum TTL
)
@ IN NS dns01nyc.business.net.
@ IN PTR business.net.
;10.2.10.0/24-IPAddr-ARecord
firewallnyc IN A 10.2.10.1
esxi01nyc IN A 10.2.10.11
esxi02nyc IN A 10.2.10.12
dns01nyc IN A 10.2.10.201
;10.2.10.0/24-IPAddr-PTRRecord
firewallnyc IN PTR 10.2.10.1
esxi01nyc IN PTR 10.2.10.11
esxi02nyc IN PTR 10.2.10.12
dns01nyc IN PTR 10.2.10.201

;IPAddr-Hostnames
1 IN PTR firewallnyc.business.net.
11 IN PTR esxi01nyc.business.net.
12 IN PTR esxi02nyc.business.net.
201 IN PTR dns01nyc.business.net.

1) I have allow query for the entire 10.0.0.0/8 subnet so I don't *think* that's it but I could be wrong
2) hmm I have this in iptables: which should allow dns (port 53)through the firewall
# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p udp -m state --state NEW --dport 53 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport 53 -j ACCEPT
3)I have listening enabled on both loopback and ip address see named.conf

This all looks correct (from first blush)

Can you query from one dc to the other?

Do this for both datacenters..

I assume you are routable (via internal IPs) between datacenters

--C

[root@dns02 ~]# dig @10.2.10.201 business.net ns

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @10.2.10.201 business.net ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10938
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;business.net. IN NS

;; ANSWER SECTION:
business.net. 86400 IN NS dns01nyc.business.net.

;; ADDITIONAL SECTION:
dns01nyc.business.net. 86400 IN A 10.2.10.201

;; Query time: 67 msec
;; SERVER: 10.2.10.201#53(10.2.10.201)
;; WHEN: Wed Feb 26 13:42:37 2014
;; MSG SIZE rcvd: 69

[root@dns01nyc ~]# dig @10.1.10.201 business.net ns

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> @10.1.10.201 business.net ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62965
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;business.net. IN NS

;; ANSWER SECTION:
business.net. 86400 IN NS dns02.business.net.
business.net. 86400 IN NS dns01.business.net.

;; ADDITIONAL SECTION:
dns01.business.net. 86400 IN A 10.1.10.201
dns02.business.net. 86400 IN A 10.1.10.202

;; Query time: 67 msec
;; SERVER: 10.1.10.201#53(10.1.10.201)
;; WHEN: Wed Feb 26 07:50:57 2014
;; MSG SIZE rcvd: 102

[root@dns01nyc ~]#

Here's the results when I try to ping each dns server

[root@dns01nyc ~]# ping dns01.business.net
ping: unknown host dns01.business.net


[root@dns02 ~]# ping dns01nyc.business.net
ping: unknown host dns01nyc.business.net
[root@dns02 ~]#

Routing works because I can ping, ssh, and scp etc.

Obviously I am replacing the identity of my real domain name with business.net

So DNS is working for neither location, actually? I would think that if you had established that ''named'' was working in the "master" location, you should be able to ping dns01 from dns01 by name, but it looks like you cannot.

Where are you setting the DNS servers for these boxes? is the router passing that information through, or a gateway server, or is it locally defined per machine?

Yes what notKlaatu said...post the contents of your /etc/resolv.conf file

--C

Locally defined

[root@dns02 ~]# cat /etc/resolv.conf
# Generated by NetworkManager
search business.net
nameserver 10.1.10.201 <== this is dns01 located in datacenter1
nameserver 10.2.10.201 <== this is dns01nyc located in datacenter2
nameserver 8.8.8.8


[root@dns01nyc ~]# cat /etc/resolv.conf
# Generated by NetworkManager
search business.net
nameserver 10.2.10.201 <== this is dns01nyc ip address located in datacenter2
nameserver 10.1.10.201 <== this is dns01 ip address located in datacenter1
nameserver 8.8.8.8

A quick recap:

dns01 & dns02 are on the 10.1.10.0/24 subnet
all hosts on 10.1.10.0/24 can reach each other via dns just fine

dns01nyc is on 10.2.10.0/24 subnet
all hosts on 10.2.10.0/24 can reach each other via dns just fine


The two datacenters are linked via VPN tunnel and all traffic is permitted on the tunnel

hosts on 10.1.10.0/24 cannot ping hosts via dns name on 10.2.10.0/24 subnet and vice versa

both locations use business.net dns suffix

This is how iptables looks on all (3) DNS servers


[root@dns01 ~]# cat /etc/sysconfig/iptables
# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -p udp -m state --state NEW --dport 53 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport 53 -j ACCEPT
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A INPUT -p udp -i eth0 --sport 53 -j ACCEPT
-A OUTPUT -p udp -o eth0 --dport 53 -j ACCEPT
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT

nslookup works but ping does not

[root@dns01 ~]# nslookup firewallnyc.business.net 10.2.10.201
Server: 10.2.10.201
Address: 10.2.10.201#53

Name: firewallnyc.business.net
Address: 10.2.10.1

[root@dns01 ~]# ping firewallnyc.business.net
ping: unknown host firewallnyc.business.net
[root@dns01 ~]#


as you can guess I need hosts on each subnet to reach each other via dnsname. I can ping them via IP just fine