Slow File Transfer Speed
 

Hello, I have what seems to be a relatively simple problem, but I cannot exactly pin point it. In summary, my friend recently gave me his old (very old) computer which I turned into a Linux web/file server. Everything is up an running beautifully, and I have verified the local ethernet network interface is running at 100Mb/s:
On the client side, when I run ttcp to test the bandwidth I get similar results:
(note that the values are in bytes per second, and when computed, bits per second is in the range of 95-100, which is very acceptable).

Aditionally, my server is very low key, meaning it essentially only gets hit when I issue a request (essentially no other users), so incoming/outgoing traffic is essentially nil. I have verified this by issuing a tcpdump when there was no requests being issued, and I saw that there was very little network traffic (a few arp/dns/dhcp/etc. packet every once in a while).

My problem arises when I try to transfer large amounts of data from one host to another. When I do so, it takes a long time 2-3 minutes to transfer ~250MB, which should only take about 20 seconds, a rate of about 16b/s. This occurs regadless of the mechanism which I am transfering the files, eg. I've used ftp, samba, and scp to perform the transfer with the same results. My first question is whether anyone has any idea why these transfers are so slow.

Since I bought this machine second hand, I know very little about the host. I'm not sure if the motherboard has a DMA controller or the CPU speed of my processor. I know that the machine only has 392MB (128 + 256 modules) of memory, which could be better, as well as an IDE hard drive interface which also could be improved. I know that all these factors have an effect on my issue, I was wondering if anyone knew a way how to collect more information/statistics about where the bottleneck is occuring when the data transfer is taking place.

Finally, a related question. I know my cpu can be classified in the x86 category (once again I'm not sure which model it exactly is), and I was wondering, if I upgraded my CPU and motherboard to solve this problem, if my existing Linux installation will work? I'm assuming that so long as I stay with the same instruction set, eg. upgrade to one of the recent celerons, pentiums, or amd processor, my system won't break, but I would like to make sure before I spend the money and perform the upgrade.

Anyways I appreciate everyone reading this long post :-p and any ideas/solutions that you might have. Thanks!!!

You would need to transfer at 6 gigabits per second to send a 256MB file on the network in 20 seconds. Plus the theoritical limit of a half duplex ethernet connection is about 37% due to the overhead in sending packets. ( Although I think when I read this it was about 10-baseT and not 100-baseT )

A 100 megabits per second includes header information as well as gaps. I read somewhere that the theoretical limit is around 37% for an ethernet connection but this was for 10 Base-T. Dividing by 8 bits per byte, 100 Mbs = 12.5 megabytes per second. This will be reduced by the overhead of packet headers so it will be about 275 megabytes per minute.

jschiwal;
I appreciate your response. I have a couple of questions pertaining to your response. First, I know nothing about the difference between half and full duplex internet. According to ethtool, eth0 support 100Mb/s with both, with my current configuration set to Full duplex, so I was wondering if I would get a performance increase by setting it to half-duplex. I also find it suprising that you can only use 37Mb/s to send data over a 100Mb/s ethernet link. This doesnt seem like an efficient standard if 63% of available bandwidth is used to transmit control information and packet headers. Just not sure if I interpreted your point clearly enough. Lastly, even if you can only transmit 275MB in one minute, it takes several minutes on my LAN to transfer a 250MB file, even though every host (including my switch) has 10/100 interfaces. I could be wrong though, and am interested in seeing feedback pertaining to normal LAN transfer times on similar networks. As always, I appreciate all the help.

Full duplex is a feature of switches (vs hubs) where you can have traffic transfer both ways at once, however most protocols are one way where you send something, wait for a reply and send more. Using a switch instead of a hub will eliminate collisions for most traffic because the switch gives you a dedicated line between two ports. If you have full duplex working, it may be better for times when you have two separate programs working on different ports. Then you might have an advantage with full-duplex. You may run into a problem if you are trying to replicate files to a number of hosts at the same time. Then you should use half-duplex. We had to set the switches for racks of adtec duets (mpeg2 video inserters) to 10 Base-T half for replication to function reliably. However, a spot might go to 24 units at once. The ones on our local network are 100-BaseT so a file first goes to a single unit, and then replicates to up to 24 other units at the same time. So that is like having effectively 2400 Base-T ( which doesn't exist ).

I was wrong on the 37% figure. It is a myth based on the simplified models that were first published in an ACM paper by Bob Metcalfe and David Boggs. The authors did caution that the models were simplified but the 36.8% utilization figure was repeated in different articles, one of which I must have read. I guess it is an urban myth, like the one about NiCad batteries having memory, which I was even told about at work. ( Only an early test model of NiCad had the memory effect, but the myth persisted ). So I don't know what the limit is, but remember, that a frame gets wrapped each time is goes down a level of the OS protocol. I thought I read it in Ethernet the Definitive Guide by Charles E. Spurgeon, but found this information when I pulled it off my bookshelf. Also the limit was the utilization limit and not a speed limit, which is what I remembered. Maybe the NiCads batteries don't have a faulty memory effect but I guess my own brain does.

One time at work I had to transfer a bunch of video commercials that I had copied from my laptop to our new server. Each spot was around 27MB in size and I was connected directly through a cross-over cable. I was getting about 10-30 megabytes a second ( A spot between 1-3 seconds ) which I thought was an incredible speed. ( And was probably close to the maximum transfer rate of my laptops harddrive ). The sending protocol we used when the network was 10 base-T took about 45 sec to 2 minutes per spot.