Greetings all,

For our network, we have 2 resolving name servers like this for instance:

* Recursive NS - 10.100.1.1
* Recursive NS - 10.100.1.2

We have a few hundred Linux machines with these two IP's in resolv.conf (or worse yet, some machines might only have one of the IP's above.)

So obviously regardless of the state of the two machines, both IP's must always be listening with DNS. How can we make the other automatically take over in the event of a failure?

I've been trying to adopt heartbeat to take on this task. It seems to monitor two severs via serial cable, and has backend scripts to manage starting/stopping the daemon. It seems to have scripts for Oracle, Apache, MySQL; but no script for NameD.

Has anyone had experience in a similar high-availability failover with DNS resolution? We're familiar with BIND and would prefer not to use DJBDNS.

I'd like to hear experiences people have had with heartbeat or other solutions. It really seems simple; but there's been no practical application for this out in the wild I guess. It seems a bit dumb to employ a $4000+ load balancer for the job when something simpler could do.

I'm missing something here. Why do cluster (heartbeat) at all?

BIND has master/slave setup and resolv.conf if it has both servers will make the lookup go to the second server automatically if it can't reach the first server. The solution here is to make sure all your machines have both name servers in resolv.conf.

I planned to employ this solution because of an antiquated environment that was poorly administrated prior to me in our organization.

Linux has automatic failover in resolv.conf but about 300 machines in the network are possibly lacking the extra nameserver. Also the timeout period for the primary nameserver is incredibly high; and lookups will go incredibly slow until it queries the secondary server. This will slow down mail, ssh connections, and wreck other havok until the primary nameserver comes back up.

A high-availability solution avoids this by ensuring that both the IP's are always available.

I'd suggest it is better to setup "standard" methodology and fix the 300+ servers (which could be done via s simple shell script as we did here when changing nameserver IPs a few months back).

Also resolv.conf does have methods for shortening the timeout period between when it tries the first server and the second.

From resolv.conf man page on RHEL5:

timeout:n
sets the amount of time the resolver will wait for a response from a remote name server before retrying the query via a different name server. Measured in seconds, the default is RES_TIMEOUT (currently 5, see <resolv.h>).

Note that your "failover" in heartbeat is going to take a finite amount of time so if you don't adjust resolv.conf you'll have a situation in event of failure that some lookups get no answer at all. Many apps won't retry and will simply abort if that occurs.

I haven't played with heartbeat (though I've done much with other forms of clustering) so I'll leave it to someone else to answer it the way you asked. You may not need a module for BIND though - just keep it up and running on both nodes and failover only the IP address.