Hi everyone,

I've got a weird NFS problem with incorrect file sizes being reported by NFS clients. I’m seeing it on both Red Hat 8.0 and SUSE SLES 9.0. I researched the SUSE problem extensively in early Jan, but found nothing.

Thanks in advance for you help!

Jeff Thayer
-----------------------------------------------------------
Red Hat 8.0 problem:

Here's what I see from node4, the NFS client:

-bash-2.05b$ ls -l file1 file2 file3 file4
-rw-rw-r-- 1 user1 group1 2824510414 Dec 25 15:31 file1
-rw-rw-r-- 1 user1 group1 2824510414 Dec 25 21:42 file2
-rw-rw-r-- 1 user1 group1 3849501956 Dec 25 22:40 file3
-rw-rw-r-- 1 user1 group1 3427997248 Dec 25 23:39 file4

#uname -a
Linux 2.4.18-14-MSA #1 SMP Tue Apr 15 10:14:02 PDT 2003 i686 i686 i386 GNU/Linux

# cat /etc/*rel*
Red Hat Linux release 8.0 (Psyche)

#grep node1 /etc/fstab
node1:/work/node1 /work/node1 nfs hard,rsize=8192,wsize=8192,timeo=14,intr
-----------------------------------------------------------
And from node1, the NFS server:

-bash-2.05b$ ls -l file1 file2 file3 file4
-rw-rw-r-- 1 user1 group1 127378561998 Dec 25 15:31 file1
-rw-rw-r-- 1 user1 group1 127378561998 Dec 25 21:42 file2
-rw-rw-r-- 1 user1 group1 8144469252 Dec 25 22:40 file3
-rw-rw-r-- 1 user1 group1 3427997248 Dec 25 23:39 file4

#name -a
Linux 2.4.18-14-MSA #1 SMP Tue Apr 15 10:14:02 PDT 2003 i686 i686 i386 GNU/Linux

#cat /etc/*rel*
Red Hat Linux release 8.0 (Psyche)

# more exports
/work/node1 10.0.2.0/255.255.255.0(rw,sync,no_root_squash,insecure)

Red Hat 8.0 NOTES:
Red Hat 8.0 is using NFS over udp.

The default Red Hat 8.0 nfs version value is version 2, according to man fstab, but according to “Red Hat Linux 8.0: The Official Red Hat Linux Reference Guide” :

Red Hat Linux supports both NFSv2 and NFSv3, and uses NFSv3 by default when connecting with a server that supports it.

I cannot figure out how to verify what version is actually doing the mounts.
---------------------------------------------------------------
---------------------------------------------------------------
SUSE SLES 9.0 NFS Problem:

node6: NFS server, exporting the files:

$ls -l file*
-rw-rw-r-- 1 user group1 6957057768 Dec 12 18:30 file1
-rw-rw-r-- 1 user group1 3148553333 Jan 7 00:45 file2
-rw-rw-r-- 1 user group1 10813920555 Dec 12 18:03 file3
-rw-rw-r-- 1 user group1 11975371659 Jan 6 17:25 file3.st

$ cat /etc/*rel*
SUSE LINUX Enterprise Server 9 (x86_64)
VERSION = 9
PATCHLEVEL = 2

$ uname -a
Linux node6 2.6.5-7.201-smp #1 SMP Thu Aug 25 06:20:45 UTC 2005 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/exports
/work/node6 10.1.4.0/255.255.255.0(rw,sync,no_root_squash)
/work/node6 10.2.6.0/255.255.255.0(rw,sync,no_root_squash)
--------------------------------------------------------------
node5: NFS client with incorrect file sizes:
$ ls -l file*
-rw-rw-r-- 1 user group1 2662090472 Dec 12 18:30 file1
-rw-rw-r-- 1 user group1 3148553333 Jan 7 00:45 file2
-rw-rw-r-- 1 user group1 2223985963 Dec 12 18:03 file3
-rw-rw-r-- 1 user group1 3385437067 Jan 6 17:25 file3.st

$ grep node6 /etc/fstab
node6:/work/node6 /work/node6 nfs hard,rsize=8192,wsize=8192,timeo=14,intr

$ cat /etc/*rel*
SUSE LINUX Enterprise Server 9 (x86_64)
VERSION = 9
PATCHLEVEL = 2

$ uname -a
Linux node5 2.6.5-7.201-smp #1 SMP Thu Aug 25 06:20:45 UTC 2005 x86_64 x86_64 x86_64 GNU/Linux
----------------------------------------------------------------------------------
node8: another, very similar NFS client with CORRECT file sizes:

$ ls -l file*
-rw-rw-r-- 1 user group1 6957057768 Dec 12 18:30 file1
-rw-rw-r-- 1 user group1 3148553333 Jan 7 00:45 file2
-rw-rw-r-- 1 user group1 10813920555 Dec 12 18:03 file3
-rw-rw-r-- 1 user group1 11975371659 Jan 6 17:25 file3.st

$ cat /etc/*rel*
SUSE LINUX Enterprise Server 9 (x86_64)
VERSION = 9
PATCHLEVEL = 2

$ uname -a
Linux node8 2.6.13-15.7-smp #1 SMP Tue Nov 29 14:32:29 UTC 2005 x86_64 x86_64 x86_64 GNU/Linux

$ grep node6 /etc/fstab
node6:/work/node6 /work/node6 nfs hard,rsize=8192,wsize=8192,timeo=14,intr 0 0

SUSE NOTES:
The default value for NFS on SUSE 9.0 is version 3, but I don’t know how to confirm the actual version that is doing the mounting. Rpcinfo –p lists both udp and tcp, but I don’t know how to confirm which protocol the clients are using. Tcpdump seemed to show some nodes were using udp and others were using tcp.

Here’s what Novell has to say about SLES 9.0 NFS clients:

Increasing the default value of 4096 bytes for the parameters rsize and wsize to 8192 or even 16384 can improve the performance. By default, NFS version 3 (NFSv3) is used with TCP as default network protocol. Both can be changed; see man 5 nfs for details.

All your problem are with the file sizes exceeding 2G .. 2G is the limit of the signed numbers in 32 bits and a there are a lot of linux programmes/tools that don't handle individual file sizes greather than 2G.
firefox still has the problem (at least last I checked it)
One way to potentially fix the problem is to "upgrade" all your distro's to the newest one and hope this has been taken care of in the packages you are using, barring that I think there is little else you can do.

Widdows 2K has a quasi fix for it .. I don't know WinXP.