I need NFS to use a VIP

prowla · 12-06-2013, 11:54 AM

Hi.

I have an Oracle Linux 6.1 (clone of Red Hat) server on which I have an NFS 4 share configured.

The server's IP is 10.10.10.43 on eth0, and I have an IP alias/VIP 10.10.10.103 on eth0:1, which is on the same subnet.

Code:

10.10.10.43 ux061t  # nodename
10.10.10.103 nfs-b  # VIP for NFS

I want to have it so that clients can only attach to NFS when talking to 10.10.10.103, but can't quite get it to work and could use a bit of help.

My /etc/exports is.

Code:

/app/share *(ro,sync,no_subtree_check,fsid=0)
/app/share/nfs-b 10.10.10.*/23(rw,sync,no_subtree_check,root_squash,fsid=1)

I've got NFS set to only use NFS 4 and to run over the usual specific ports (by uncommenting the appropriate lines in /etc/sysconfig/nfs).

Code:

LOCKD_TCPPORT=32803
LOCKD_UDPPORT=32769
RPCNFSDARGS="-N 2 -N 3"
MOUNTD_PORT=892
STATD_PORT=662

I've configured iptables to allow clients to connect from my subnet and have attempted to restrict the interface used by specifying -i and -d options (NB. iptables doesn't support specifying "-i eth0:1", which would have been great).

Code:

-A INPUT -m state --state NEW -p udp -m udp --dport 111 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p tcp -m tcp --dport 111 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p udp -m udp --dport 662 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p tcp -m tcp --dport 662 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p udp -m udp --dport 875 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p tcp -m tcp --dport 875 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p udp -m udp --dport 892 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p tcp -m tcp --dport 892 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p udp -m udp --dport 2049 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p tcp -m tcp --dport 2049 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p udp -m udp --dport 32769 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT  
-A INPUT -m state --state NEW -p tcp -m tcp --dport 32803 -i eth0 -d 10.10.10.103/23 -s 10.10.10.0/23 -j ACCEPT

I can mount filesystems on a client, either manually, via fstab entries, or via automounter).

But the issue I have is that clients can mount from either the server's IP or the VIP.

Code:

nfs-b:/nfs-b   20G  3.7G   15G  20% /tmp/good
ux061t:/nfs-b  20G  3.7G   15G  20% /tmp/bad

I want to make it so only the VIP can be used.

The background is that we need to be able to move the share to another host, either manually or under cluster control, and whilst we could attempt to impose a discipline where the clients must specify the VIP, I want to be absolutely certain that only the VIP is used and that attempted mounts using the underlying servername or IP address fail.

It would be great to get some pointers on this - thanks!

Sydney · 12-08-2013, 03:53 AM

I would look at setting up a route via the route command to do this, but have never tried.

prowla · 12-09-2013, 10:19 AM

Hmmm - I hadn't thought of using routing.

TBH, I'm not sure that would stop clients mounting by servername/IP as opposed to the VIP.

berndbausch · 12-09-2013, 07:03 PM

Quote:

Originally Posted by prowla

Hmmm - I hadn't thought of using routing.

TBH, I'm not sure that would stop clients mounting by servername/IP as opposed to the VIP.

You use iptables to allow clients to connect. It should not be difficult to create iptables rules to prevent them from connecting either.

prowla · 12-10-2013, 01:26 AM

Quote:

Originally Posted by berndbausch

You use iptables to allow clients to connect. It should not be difficult to create iptables rules to prevent them from connecting either.

It's be great if you could suggest how?

The iptables rules I have are above, but they don't seem to block it...

zhjim · 12-10-2013, 03:15 AM

The extract of the iptables rules ACCEPT nfs traffic to the VIP. So you would need to deny the others somewhere. Either through policy or explitly. Take your rules flip the dest ip and -j to REJECT. See if that goes. So you would need to allow the regular traffic to the VIP so.

berndbausch · 12-10-2013, 04:02 AM

Quote:

Originally Posted by prowla

It's be great if you could suggest how?

The iptables rules I have are above, but they don't seem to block it...

Without any iptables rule, everything is allowed. One common strategy is to reject any connection by default, then allow only those addresses you want to access the ports you want. E.g.

Code:

iptables -P INPUT REJECT # This sets the INPUT policy: If no rule matches, reject the connection.
                         # If you want to be rude, you can also DROP instead of REJECT it
iptables -A INPUT -s 127.0.0.0/8 -j ACCEPT # accept any connection from localhost
<your NFS rules>

Or you can write rules that block more surgically:

Code:

-A INPUT -m state --state NEW -p udp -m udp --dport <one of your NFS ports> -i eth0 -d 10.10.10.43 -s 10.10.10.0/23 -j REJECT

By the way, in your rules you have "-d 10.10.10.103/23". I am not sure what that does, but it may be equivalent to 10.10.10.0/23. In any case, you don't need the "/23" part.