LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 06-28-2015, 01:46 AM   #1
mdooligan
Member
 
Registered: Feb 2009
Location: Vancouver BC
Distribution: Mandrake10/ArchHackery/Gentoo
Posts: 179

Rep: Reputation: 22
LAN data movement *very* slow one direction...


... as in 45Kbits/sec slow.

I have simple domestic LAN with 3 nodes. 2 of the PCs are basically 10 year old clones that work together fine and always have.

The 3rd one is new desktop just acquired a few weeks ago. Let's call them OldA, OldB, and Newbox. OldA is my main PC and the leader of the group. If I do tests between the 3 of them with iperf, I get ~94Mbits/sec, 112MB in 10sec, all 6 ways. Looks satisfactory, for me anyway.

But...

I mount an NFS share from each on each, so we can freely copy data between any of them to any other, we have 12 ways to copy data:

OldB: copy <-> OldA from OldB, both ways = ~94Mbits/sec
OldB: copy <-> Newbox from OldB, both ways = ~94Mbits/sec
OldA: copy <-> OldB from OldA, both ways: ~94Mbits/sec
Newbox: copy <-> OldB from Newbox, both ways = ~94Mbits/sec

... OldB is fine, so we can ignore it.

Newbox: copy <-> OldA from Newbox, both ways = ~94Mbits/sec
OldA: copy FROM Newbox: ~94Mbits/sec
OldA: copy TO Newbox: ~54Kbits/sec <-- !!!

I swapped ports and cords and cleaned up the snakepit. The problem persists. Pushes from OldA to Newbox suck. I have to Pull from Newbox. This has been consistent, between about 10 different distros I've tried over the past few weeks.

Newbox is a Dell OptiPlex 755 desktop with onboard NIC:
Code:
00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02)
        Subsystem: Dell OptiPlex 755
        Kernel driver in use: e1000e
        Kernel modules: e1000e
When I finally settled on a distro, I got to polishing things up. This MB NIC gives me problems: I cannot load e1000e from initrd or the box thinks it's CAT5 is unplugged and I have to unplug/plug to get it to go, but, this is random, not at every boot. If I load the module later during boot, it's fine. Please note I upgraded kernel module e1000e to the most recent version.

After googling up and down over the past week, I tried mii-tool:

OldB:
Code:
[root@OldB ~]$ sudo mii-tool -v eth0
eth0: negotiated 100baseTx-FD, link ok
  product info: vendor 00:00:20, model 32 rev 1
  basic mode:   autonegotiation enabled
  basic status: autonegotiation complete, link ok
  capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
  advertising:  100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
  link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
OldA:
Code:
[root@OldA ~]$ sudo mii-tool -v eth0
eth0: negotiated 100baseTx-FD, link ok
  product info: vendor 00:00:20, model 32 rev 1
  basic mode:   autonegotiation enabled
  basic status: autonegotiation complete, link ok
  capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
  advertising:  100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
  link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
Newbox:
Code:
[root@Newbox ~]# mii-tool -v eth0
eth0: negotiated 100baseTx-FD flow-control, link ok
  product info: vendor 00:aa:00, model 57 rev 0
  basic mode:   autonegotiation enabled
  basic status: autonegotiation complete, link ok
  capabilities: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
  advertising:  1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
  link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
I'm getting into land of deep magic here. What's that missing flow-control mean? How do I fix it? There seems to be some suggestion OldA and Newbox are not auto-negotiating correctly or one is set at half-duplex instead of full duplex.

Is there an ethtool hack or something I can do here? Some driver option I've never heard of?

Peace and Cheer.
 
Old 06-28-2015, 03:21 AM   #2
ferrari
LQ Guru
 
Registered: Sep 2003
Location: Auckland, NZ
Distribution: openSUSE Leap
Posts: 5,869

Rep: Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152
Well, the iperf tests would indicate no problems. Did you do TCP and UDP tests? Assuming that these machines connected via a common switch?

This seems to be more about NFS tuning. Which version of NFS are you using? Are you using TCP or UDP? Maybe you should share your mount parameters here (for all three machines)

Some reading:
http://www.slashroot.in/how-do-linux...d-optimization
http://www.cyberciti.biz/faq/linux-u...t-performance/
 
1 members found this post helpful.
Old 06-29-2015, 04:45 AM   #3
mdooligan
Member
 
Registered: Feb 2009
Location: Vancouver BC
Distribution: Mandrake10/ArchHackery/Gentoo
Posts: 179

Original Poster
Rep: Reputation: 22
Awesome, Thank you for the reply.
Quote:
Originally Posted by ferrari View Post
Well, the iperf tests would indicate no problems. Did you do TCP and UDP tests? Assuming that these machines connected via a common switch?

This seems to be more about NFS tuning. Which version of NFS are you using? Are you using TCP or UDP? Maybe you should share your mount parameters here (for all three machines)

Some reading:
http://www.slashroot.in/how-do-linux...d-optimization
http://www.cyberciti.biz/faq/linux-u...t-performance/
Yes, they are all connected through a common ISP provided switch/router.

TCP and UDP tests? No. Network always just sorta works, I've never tuned or tested anything before this. I didn't even know iperf and mii-tool existed before. How is this done? I'm very curious, time to learn something.

Here's some infos for all 3 machines:

OldA fstab:
Code:
# OldB has sporadic MB issue, but otherwise works great, therefore 'soft' mount.
OldB:/mnt/hda10 /mnt/nfs/OldB/hda10 nfs noauto,users,soft,timeo=60 0 0
Newbox:/mnt/sdb1 /mnt/nfs/Newbox/sdb1 nfs noauto,sync,users,exec 0 0
OldB fstab:
Code:
OldA:/mnt/sdc1 /mnt/nfs/OldA/sdc1 nfs user,dev,rsize=8192,wsize=8192,noauto,suid,exec,soft 0 0
Newbox:/mnt/sdb1 /mnt/nfs/Newbox/sdb1 nfs noauto,sync,users,exec 0 0
Newbox fstab:
Code:
OldA:/mnt/hdc1 /mnt/nfs/OldA/hdc1 nfs noauto,sync,users,exec 0 0
OldB:/mnt/hda10 /mnt/nfs/OldB/hda10 nfs noauto,sync,users,exec 0 0
The problem is between OldA and Newbox. Something perhaps to note is that Newbox will not mount the other PCs shares unless they are explicitly exported to it, eg:
/etc/exports:

Code:
/mnt/sdc1 192.168.1.66(rw,insecure,sync,no_root_squash)
instead of

Code:
/mnt/sdc1 192.168.1.*(rw,insecure,sync,no_root_squash)
which works for the other boxes. I obviously have no security concerns at all.

Those are the same exports parameters I use on all 3 boxes.

OldA & OldB versions:

Code:
kmountd nfs-utils 1.0.6 (rpc.mountd)
mount: mount-2.12
kernel: 2.6.6 #6 SMP Tue Nov 15 11:15:23 PST 2005 i686
Newbox:
Code:
rpc.mountd version 1.3.2
mount from util-linux 2.26.2 (libmount 2.26.0: assert, debug)
kernel: 4.0.5-1-ARCH #1 SMP PREEMPT Sat Jun 6 18:52:28 CEST 2015 i686 GNU/Linux
Thank you for the links. I'm reading them right now.

OldA /proc/mounts:
Code:
Newbox:/mnt/sdb1 /mnt/nfs/Newbox/sdb1 nfs 
rw,sync,nosuid,nodev,v3,rsize=32768,wsize=32768,
hard,udp,lock,addr=Newbox 0 0
Newbox /proc/mounts:
Code:
OldA:/mnt/sda1 /mnt/nfs/OldA/sda1 nfs
rw,sync,nosuid,nodev,relatime,vers=3,rsize=32768,wsize=32768,
namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,
mountaddr=192.168.1.64,mountvers=3,mountport=922,mountproto=udp,
local_lock=none,addr=192.168.1.64 0 0
OldA ifconfig:
Code:
eth0      Link encap:Ethernet  HWaddr 00:C0:9F:93:64:48  
          inet addr:192.168.1.64  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::2c0:9fff:fe93:6448/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:12391085 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15966391 errors:0 dropped:2 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:532801195 (508.1 Mb)  TX bytes:926370216 (883.4 Mb)

Server rpc stats:
calls      badcalls   badauth    badclnt    xdrcall
1142043    6          6          0          0       
Client rpc stats:
calls      retrans    authrefrsh
12252      17         0       Interrupt:19 Base address:0x2000
Looks like we're losing packets. RFI one way?

Newbox ip -s addr/link show eth0:
Code:
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:c0:9f:93:64:48 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.64/24 brd 192.168.1.255 scope global eth0
    inet6 fe80::2c0:9fff:fe93:6448/64 scope link 
       valid_lft forever preferred_lft forever
    RX: bytes  packets  errors  dropped overrun mcast   
    532867158  12391431 0       0       0       12939  
    TX: bytes  packets  errors  dropped carrier collsns 
    927056481  15967010 0       2       0       0 

Server rpc stats:
calls      badcalls   badfmt     badauth    badclnt
978        0          0          0          0       
Client rpc stats:
calls      retrans    authrefrsh
15         0          15
Looks like 8 nfs threads running on each machine.

OK. I just read about 'sync' and 'async' options for mount. I changed OldA's fstab to mount Newbox async. The copy to Newbox speed leaps to proper rate: 9-10 MB/s.

After further investigation with mount options, I have discovered that OldA and OldB have been connected (udp,async), not (udp,sync) as I had assumed all this time. /proc/mounts doesn't report sync or async on those 2 boxes.

I've switched everything to (tcp,async) in all the fstabs and speed is 9-10 MB/s all 12 ways. There's the answer, I guess.
 
Old 06-29-2015, 06:01 AM   #4
ferrari
LQ Guru
 
Registered: Sep 2003
Location: Auckland, NZ
Distribution: openSUSE Leap
Posts: 5,869

Rep: Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152Reputation: 1152
Quote:
OK. I just read about 'sync' and 'async' options for mount. I changed OldA's fstab to mount Newbox async. The copy to Newbox speed leaps to proper rate: 9-10 MB/s.

After further investigation with mount options, I have discovered that OldA and OldB have been connected (udp,async), not (udp,sync) as I had assumed all this time. /proc/mounts doesn't report sync or async on those 2 boxes.

I've switched everything to (tcp,async) in all the fstabs and speed is 9-10 MB/s all 12 ways. There's the answer, I guess.
Glad to read of your progress with the NFS performance.
 
Old 06-30-2015, 03:03 AM   #5
mdooligan
Member
 
Registered: Feb 2009
Location: Vancouver BC
Distribution: Mandrake10/ArchHackery/Gentoo
Posts: 179

Original Poster
Rep: Reputation: 22
Yes, thanks again.

I was looking in the wrong place. Sometimes a fresh outside perspective works wonders. I'll tag thread as solved.

Peace and cheer.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Iperf test - slow in one direction trafikpolisen Linux - Networking 5 01-10-2015 11:57 PM
in need of direction for encrypted LAN + video playback lleb Linux - Server 2 06-19-2014 02:20 PM
fluxbox slow movement of windows help please?? michaelinux Linux - Software 3 06-04-2010 02:57 PM
slow cursor movement after new install rblampain Debian 5 12-14-2008 11:02 PM
Slow movement in Wine lord_cedrich Linux - General 1 09-18-2006 05:06 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 05:25 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration