Odd LAN transfer speeds varying on platform initiation...
 

I have stumbled across something that is frustrating the heck out of me. Hopefully someone here can help. My network is as follows:

- 100 Mbps switch
- Cat 5e cabling

PC1:
- Linux Mint 16 64-bit Cinnamon
- Intel mobo w/ integrated 1Gbps ethernet
- Intel Core 2 Duo @ 2.3 Ghz
- 2 GB Ram
- 500 GB SATA2 HDD AHCI mode
- EXT4 partitions
- "data" folder shared via usershare

PC2:
- Windows 7 Home Premium 64-bit
- AMD Phenom II quad core @ 3.2 Ghz
- AMD motherboard with integrated Realtek 1Gbps ethernet
- 6 GB of RAM
- 2 TB SATA2 HDD AHCI Mode
- NTFS partitions
- "media" folder shared via windows file sharing with password protection (homegroup disabled)

Scenario 1:
I sit at PC1 which acts as a very basic data server for my home network. When I initiate transfers while sitting at this machine using NEMO it goes as follows:
- Copy a file from PC1 into "media" folder on PC2 (size doesn't matter). Sending speed is about 4.4 MB/s (40 Mbps or so).
- Copy a file from PC2 "media" folder to PC1 (size doesn't matter). Receiving speed is about 5.5 MB/s (50 Mbps or so).
- Using Firefox, downloading large files from PC2's simple HTTP server, transferring 1 file goes around 3.3 MB/s. Transferring 5 large files maxes out the connection at 11.2 MB/s.

Scenario 2:
I set at PC2 which is basically a client on my home network most of the time. When I initiate transfers while sitting at this machine using Windows Explorer it goes as follows:
- Copy a file from PC2 into "data" folder on PC1 (size doesn't matter). Sending speed is 11.2 MB/s (99.7 Mbps or so).
- Copy a file from PC1 "data" folder to PC2 (size doesnt matter). Receiving speed is 11.2 MB/s (99.7 Mbps or so).

Any transfer initiated from PC2 (win7) operates at the proper speed regardless of direction of transfer. Any transfer initiated from PC1 (Mint) goes MUCH slower regardless of direction of transfer. Does this make sense to anyone? Is there a way to get Mint to transfer as fast? I don't understand what would cause these difference.

PC1:
- Intel Core 2 Duo @ 2.3 Ghz

PC2:
- AMD Phenom II quad core @ 3.2 Ghz

The main reason is CPU power.

Install iperf, which runs RAM to RAM. This cuts out all the disk subsystem / file system variables and just tests the network.

Compare the results you get with your file transfers should help narrow things down. Both processors should have enought grunt to push 1G back to back. OK about 960Mbps is more like it, but you get my drift.

The CPUs never break 15% utilization during transfers. And as I already illustrated, the Core 2 Duo has no problems sending or receiving at 99.7 Mbps if I initiate the transfer at the windows box. this isn't a hardware restriction.

I have done this before but I forgot what speeds I got. I can only hit 100 Mbps due to my network switch limitations. The 100 Mbps should be well below any HDD or system hardware speed bottlenecks.

I'll run one test with iperf server on the linux box and one with the server running on the windows box and post the results.

Running iperf -s on Linux box and iperf -c on Win7 machine: 54.5 Mbps

Running iperf -s on Win7 and iperf -c on Linux: 71.9 Mbps

I noticed there are two different default TCP window sizes depending on which machine is running as server. Is that normal?

Also, when running iperf -c in linux, the -m option worked and reported 1500 MTU but running the -m option in win7 could not determine MTU.

Those numbers are low.

TCP window varies dependant on OS, distro and sometimes version. As long as your machines are locally connected with sub 1mS latency then the lowest window size is adequate, although low window size will very rapidly throttle throughput if latency starts to rise. What ping time do you get? Is it the same for large pings? ~1500bytes?

So you know the issue isn't disk related. Could be a switch issue. If you can find a crossover cable, try iperf back-to-back. If its still low its likely a NIC or driver issue. If its better then something switch related. Duplex and speed settings? Dodgy cabling? Is your switch reporting errors?

The speeds are low on iperf but I still get 99.7 Mbps on transfers initiated by the windows machine (both directions).

The ping is 3-6ms. Each machine is connected to the same switch. total length of cable is probably 20ft. 5' on one and 15' on the other. Cat5e cable for both.

I haven't done a direct connect, but when I temporarily connected them to a 1Gbps switch, the transfer speeds were around 600 Mbps which seemed "ok."

I don't think there are any hardware issues. The issue is why the Windows transfers (both directions) are faster than Linux. I think it has to do something with the TCP packets but I'm not sure how to prove that. Does windows determine the TCP settings when it initiates a transfer and vice versa? Or maybe it is Linux internal traffic control not willing to saturate the connection.

I don't know how to do a large ping.

I have wiped everything and installed XFCE version of Mint. Exact same problems :(

TCP has a set of default settings that it applies to new outbound TCP sessions, unless the application requests something different. It also has a "range" defined that controls how far it can go in negotiating a new inbound connection.

Did you try iperf with UDP? any difference?

The fact that windows is ok with the same hardware setup, but iperf/Linux is slow even with an identical hardware setup would tend to point to the linux drivers as being the source of the issue.

Do you have different nics you can try?

Some of the issues I've seen are related to the offloading functions. TCP Checksum, Segmentation and other processing of TCP can be handed to the NIC card driver/hardware in order to free up the CPU from this overhead.

You use ethtool to show and set these offload settings, unfortunately its a trial end error process if something is misbehaving.

This is quite a good post on the subject, google will find you a lot more.

Thanks for your help baldy but a few questions.

1) Would it be a driver issue since it does transfer at 99.7 Mbps even under Linux (it just happens when working at the win7 box)? I would think it could never hit those speeds if it was a NIC/driver issue.

2) How do I run the UDP test in iperf?

I do think it has something to do with the TCP stack. Maybe windows doesn't like Linux's TCP settings and that slows it down?

I just finished attempting a UDP iperf test. I was running iperf -s -u and iperf -c -u and it came back with 1.05 Mbps. I must have not done something right on that one.

I have also noticed that running iperf -s on either machine, whatever is restricting file transfers initiated in Linux is also restricting iperf to those same speeds (~6 MB/s win7->linux & ~5 MB/s linux->win7).

I'm wondering if maybe win7 uses something other than TCP for file sharing transfers.

When transferring files from linux to windows via NEMO you are using gvfs-smb as a samba client. When transferring files from windows to linux you are using the samba server. The client and server are separate processes.

A fairly quick search showed there were problems with gvfs-smb being slow or freezing. For some additional data you can manually mount the windows share (command is below) as a quick test to determine if gvfs is the problem. The samba client uses mount.cifs which is not the same thing as gvfs-smb.

mount -t cifs //windows/share /media/windows_mountpoint (create a mount point of your own choosing)

P.S I believe that Thunar also uses gvfs which could be the same reason why it is slow.

I appears to be a bug.
https://bugzilla.gnome.org/show_bug.cgi?id=532951

Thank you so much! I'll investigate this ASAP. Last night I was reading and it began to seem that it was something like you explained. It seemed Windows 7 is using something else to manage file transfers than what Linux is using. I thought it might be either a netbios or SMB/Cifs issue but hadn't figured it out yet. It makes sense because that is the biggest difference. When using a linux machine, Linux has to mount the windows shared folder but it does not when windows controls the transfer.

TCP setting on Windows and Linux could cause the issue. Following link tell you how to tuning TCP setting on Linux. You should pay attention on congestion avoidance algorithm and timestamp.
http://thesimplecomputer.info/advent...-tuning-page2/