NFS error while running tcpdump.
 

Hello,

I have configured NFS Server 1 IBM Web Server. The IBM Web Server can upload all data onto NFS Server. Now, today i was having slow response on IBM Web Server & by measuring the NFS, i found below error while running "tcpdump" command.

tcpdump -n -i eth1 | grep 2049

18:36:37.237451 IP 10.100.19.241.2049 > 10.100.19.88.1758143293: reply ok 1448 read [|nfs]
18:36:37.237476 IP 10.100.19.241.2049 > 10.100.19.88.539981409: reply ERR 1448
18:36:37.237481 IP 10.100.19.241.2049 > 10.100.19.88.796287348: reply ERR 1448
18:36:37.237488 IP 10.100.19.241.2049 > 10.100.19.88.1986098295: reply ERR 1448
18:36:37.237566 IP 10.100.19.241.2049 > 10.100.19.88.539762736: reply ERR 1448
18:36:37.237595 IP 10.100.19.241.2049 > 10.100.19.88.740307248: reply ERR 1448
18:36:37.237599 IP 10.100.19.241.2049 > 10.100.19.88.808333616: reply ERR 208
18:36:37.237792 IP 10.100.19.241.2049 > 10.100.19.88.1758143294: reply ok 1448 read [|nfs]
18:36:37.237810 IP 10.100.19.241.2049 > 10.100.19.88.1700950889: reply ERR 1448
18:36:37.237827 IP 10.100.19.241.2049 > 10.100.19.88.539762735: reply ERR 1332
18:36:37.237866 IP 10.100.19.241.2049 > 10.100.19.88.825830968: reply ERR 1448
18:36:37.237947 IP 10.100.19.241.2049 > 10.100.19.88.808333625: reply ERR 1200
18:36:37.237967 IP 10.100.19.241.2049 > 10.100.19.88.775435808: reply ERR 1448
18:36:37.237973 IP 10.100.19.241.2049 > 10.100.19.88.959330358: reply ERR 1448
18:36:37.237980 IP 10.100.19.241.2049 > 10.100.19.88.908078112: reply ERR 1448
18:36:37.237987 IP 10.100.19.241.2049 > 10.100.19.88.540030496: reply ERR 1448
18:36:37.238041 IP 10.100.19.241.2049 > 10.100.19.88.825897271: reply ERR 1448
18:36:37.238048 IP 10.100.19.241.2049 > 10.100.19.88.540094778: reply ERR 1448
18:36:37.238055 IP 10.100.19.241.2049 > 10.100.19.88.825237553: reply ERR 1448
18:36:37.238109 IP 10.100.19.241.2049 > 10.100.19.88.808529952: reply ERR 1448
18:36:37.238120 IP 10.100.19.88.1758143297 > 10.100.19.241.2049: 116 read [|nfs]
18:36:37.238129 IP 10.100.19.241.2049 > 10.100.19.88.842019120: reply ERR 704
18:36:37.238142 IP 10.100.19.241.2049 > 10.100.19.88.740306806: reply ERR 1448
18:36:37.238158 IP 10.100.19.241.2049 > 10.100.19.88.892805164: reply ERR 1200
18:36:37.238175 IP 10.100.19.241.2049 > 10.100.19.88.1758143288: reply ok 148 setattr [|nfs]
18:36:37.238256 IP 10.100.19.241.2049 > 10.100.19.88.1758143296: reply ok 1448
18:36:37.238263 IP 10.100.19.241.2049 > 10.100.19.88.1869440302: reply ERR 1448
16133 packets captured
23339 packets received by filter
7100 packets dropped by kernel

10.100.18.241 is the IP of NFS Server & 10.100.19.88 IP belongs to IBM Web Server.

More details of NFS on both Linux & AIX is given below.

My AIX NFS Version is :-

"mount" Command on AIX

/dev/hd4 / jfs2 Nov 21 16:00 rw,log=/dev/hd8
/dev/hd2 /usr jfs2 Nov 21 16:00 rw,log=/dev/hd8
/dev/hd9var /var jfs2 Nov 21 16:00 rw,log=/dev/hd8
/dev/hd3 /tmp jfs2 Nov 21 16:00 rw,log=/dev/hd8
/dev/fwdump /var/adm/ras/platform jfs2 Nov 21 16:01 rw,log=/dev/hd8
/dev/hd1 /home jfs2 Nov 21 16:01 rw,log=/dev/hd8
/proc /proc procfs Nov 21 16:01 rw
/dev/hd10opt /opt jfs2 Nov 21 16:01 rw,log=/dev/hd8
/dev/backuplv /backup jfs2 Nov 21 16:01 rw,log=/dev/loglv00
/dev/oraclelv /oracle jfs2 Nov 21 16:01 rw,log=/dev/loglv00
10.100.19.240 /web_upload/ /web_upload nfs3 Feb 12 11:27

Also "nfsstat" command in AIX giving same output.

Version 3: (365616579 calls)
null getattr setattr lookup access readlink read
0 0% 6284111 1% 193382 0% 1318563 0% 730659 0% 0 0% 353731942 96%
write create mkdir symlink mknod remove rmdir
1511215 0% 185493 0% 26974 0% 0 0% 0 0% 102644 0% 84 0%
rename link readdir readdir+ fsstat fsinfo pathconf
132247 0% 31 0% 26915 0% 88015 0% 201331 0% 30 0% 0 0%
commit
1082943 0%

The LINUX (CentOS 5.3)NFS Server is having below version of NFS.

rpm -qa | grep nfs
system-config-nfs-1.3.23-1.el5
nfs-utils-lib-1.0.8-7.2.z2
nfs-utils-1.0.9-33.el5

rpm -qa | grep portmap
portmap-4.0-65.2.2.1

Kindly let me know if anyone know the solution or provide me possible solution.