I've got a weird NFS server problem that I can't figure out.
I've had a long-time running NFS server on a Debian Stable 3.1 system. Within the last few weeks, I noticed that clients were hanging when trying to access the NFS shared files on my file server. Also, the startup of NFS is suddenly very slow, and I frequently get RPC Timed Out error messages when mounting clients.
Here are the symptoms:
1. "Mount server:/exportedfile /localfile" usually works, but takes 15+ secs to connect, and often (20% of the time) fails with an "RPC Timeout" failure. Previously, they connected almost immediately. The problem is worse if I use a script to mount several shares in quick succession.
2. Clients frequently hang trying to access previously-mounted NFS shares.
3. Start up of NFS-Kernel server hangs for 60+ secs during the "exporting files" step.
Here is data I know:
1. The system is a stable Debian and has not been messed with (other than Debian apt-get upgrades). It has been running for almost a year without any trouble.
2. rpc.mounted and portmap are running and don't put any errors in the logs.
3. The problems appears on multiple clients. I'm pretty sure it's a server issue.
4. rpcinfo reports the server is running fine
5. All the directories listed in "exports" seem OK. I can access those files locally on the server without any trouble.
6. /etc/hosts.allow has an entry of "ALL: 192.168.1.0/24(rw,async,no_root_squash)", so I don't think that's the problem
7. All other programs running on this server (music, ntp, etc) seem fine.
8. The kernel is a stock Debian 22.214.171.124-386
9. Network traffic is low. Nothing "weird" is showing up in ethereal scans.
10. I have plenty of disk space and plenty of free memory.
Here is a clue I think might be significant:
1. running "exportfs -r -v" lists the files in the "/etc/exports" file quickly, then pauses for a very long time before it loads the previously mounted exports into the kernel. This step is very quick, however, if I remove all the entries from /var/lib/nfs/rmtab.
This problem has me totally confused. I don't know where else to look for troubleshooting clues. Anybody have any suggestions?