Summary: scp'ing files take forever unless network traffic is created from a different terminal and then the scp transfer speeds up.

Yes it is that crazy.



Explained in more detail:

I have three computers sitting on a lan.
1. redhat 7.2 mainly used as a file server
2. redhat 9 mainly used as my desktop
3. redhat 9 soon to replace my ageing 7.2 machine (computer1)

These are connected through cat5e cabling and run by a linksys router (as my NAT).

I ssh to computer3 from computer2 in two different terminals. I begin to scp files from one of the terminals, transfering the files from computer1 to itself (computer3). The transfer is painfully slow, maybe 50-100k per second.

Then (get this).

I just type in the other terminal (just hit on the keyboard) and bam, my scp transfer speeds fly up to 4-5 megs a second. After about 1-3 minutes the transfer speed falls back to 50-100k until I type some characters in the other terminal and then the scp goes back to 4-5 megs a second.

I know this sounds odd. I did not believe it myself but I have tcplogs to prove it.

High transfer speed http://www.patnjay.org/stuff/good.tcp

Low transfer speed http://www.patnjay.org/stuff/bad.tcp
(sorry I can't post a url, I don't have 5 post yet, whatever)

(view the tcpdump logs by "/usr/sbin/tcpdump -r bad.tcp")

An excerpt from the good and bad logs are as follows:


High Speed

20:02:17.177988 guardian.ssh > 192.168.1.104.32792: P 1448:1944(496) ack 1 win 16800 <nop,nop,timestamp 8694634 9597117> (DF) [tos 0x8]
20:02:17.178337 192.168.1.104.32792 > guardian.ssh: . ack 1944 win 31856 <nop,nop,timestamp 9597117 8694634> (DF) [tos 0x8]
20:02:17.180256 guardian.ssh > 192.168.1.104.32792: . 1944:3392(1448) ack 1 win 16800 <nop,nop,timestamp 8694635 9597117> (DF) [tos 0x8]
20:02:17.180384 guardian.ssh > 192.168.1.104.32792: . 3392:4840(1448) ack 1 win 16800 <nop,nop,timestamp 8694635 9597117> (DF) [tos 0x8]
20:02:17.180395 192.168.1.104.32792 > guardian.ssh: . ack 4840 win 34752 <nop,nop,timestamp 9597117 8694635> (DF) [tos 0x8]
20:02:17.180505 guardian.ssh > 192.168.1.104.32792: . 4840:6288(1448) ack 1 win 16800 <nop,nop,timestamp 8694635 9597117> (DF) [tos 0x8]
20:02:17.180631 guardian.ssh > 192.168.1.104.32792: . 6288:7736(1448) ack 1 win 16800 <nop,nop,timestamp 8694635 9597117> (DF) [tos 0x8]
20:02:17.180640 192.168.1.104.32792 > guardian.ssh: . ack 7736 win 34752 <nop,nop,timestamp 9597117 8694635> (DF) [tos 0x8]


Low Speed

22:21:04.273065 guardian.ssh > 192.168.1.104.32792: . 1:1449(1448) ack 80 win 16800 <nop,nop,timestamp 9527405 10429826> (DF) [tos 0x8]
22:21:04.273096 192.168.1.104.32792 > guardian.ssh: . ack 1449 win 34752 <nop,nop,timestamp 10429827 9527405> (DF) [tos 0x8]
22:21:04.273184 guardian.ssh > 192.168.1.104.32792: . 1449:2897(1448) ack 80 win 16800 <nop,nop,timestamp 9527405 10429826> (DF) [tos 0x8]
22:21:04.273195 192.168.1.104.32792 > guardian.ssh: . ack 2897 win 34752 <nop,nop,timestamp 10429827 9527405> (DF) [tos 0x8]
22:21:04.273291 guardian.ssh > 192.168.1.104.32792: P 2897:4137(1240) ack 80 win 16800 <nop,nop,timestamp 9527405 10429826> (DF) [tos 0x8]
22:21:04.273303 192.168.1.104.32792 > guardian.ssh: . ack 4137 win 34752 <nop,nop,timestamp 10429827 9527405> (DF) [tos 0x8]
22:21:04.273949 guardian.ssh > 192.168.1.104.32792: . 4137:5585(1448) ack 80 win 16800 <nop,nop,timestamp 9527405 10429827> (DF) [tos 0x8]




I can tell that the ack is a lot higher in slower transfer speeds. I assume this the time the the destination server took to acknowelge the packet sent to it. But that is as far as I can go.


If anyone can help me with this. That would be great. I'm stumped.

did you ever figure this out?

this problem is driving me nuts...and wasting tons of my time when using ssh...was gonna look for another ssh client, but seeing this post makes me wonder if this is something more endemic to the system than program...

That is really bizarre. You're a little bit off about the ACKs. That is just acknowledging the TCP seqence, so it's irrelevant. It almost sounds like one of your boxes is using QoS rules or something like that and is deprioritizing traffic to IPs that don't have active connections.

QoS and traffic shaping are way over my head so I won't even begin to speculate on a fix.

I did find out the solution for this problem. I'm sorry I did not post it back to this user list, my error.

Anyhow it was because my nic cards where not interfacing properly. My new rh 9.0 computer's (referred in my earlier post as 'computer 3') mother board is an nforce using the driver, nvnet.

It seems that this driver screws up being left to its own devices to do speed (10 or 100 kbit/s) negotiations. This might be due to the linksys router, I don't know. So I had to set it. I stuck this line in my modules.conf

options nvnet speed=2 optimization=1

speed=2 : Sets it at 100kbit/s
optimization=1: Does some other crap, I think it uses more cpu for some thing or another. Its in nvidia's doc file.


I hope this Helps

jason^