LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 07-06-2007, 02:19 PM   #1
g_k
LQ Newbie
 
Registered: Jul 2007
Posts: 3

Rep: Reputation: 0
Unhappy gigabit ethernet really slow


Hi!

The performance of my Gigabit Ethernet is really bad. I have 2 hosts with up-to-date gentoo installations, three kinds of gigabit ethernet cards (Marvell Yukon - skge, Realtek - r8169, Intel - e1000) a gigabit switch (3com) and 2 different cables (patch and crossover, both Cat6). I tried all possible combinations between the cards with and without the switch, however, when I scp a file from one host to another I get not more than ~19MB/s. When using 100MBit cards I get 11MB/s!

I'm really stuck now...

Any help greatly appreciated!

Thx, georg

P.S.:

The current setup is "host1(e1000) <-> switch <-> host2(skge)":
ethtool output on host1:
Code:
Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: umbg
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: yes
ethtool output on host2:
Code:
Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x00000037 (55)
        Link detected: yes
 
Old 07-06-2007, 02:33 PM   #2
t1n0m3n
Member
 
Registered: Nov 2004
Distribution: Kubuntu 7.10
Posts: 69

Rep: Reputation: 15
Does ifconfig show errors?

Are you sure that you are not bottle-necking somewhere else.... Like the write capacity of the hard drive?

My home network could be gig, but since I have a laptop it is worthless (for what I would use it for). I can't go much beyond 100Mbps transfers anyway. My laptop can't write to HD that fast.

Last edited by t1n0m3n; 07-06-2007 at 02:38 PM.
 
Old 07-08-2007, 06:27 AM   #3
g_k
LQ Newbie
 
Registered: Jul 2007
Posts: 3

Original Poster
Rep: Reputation: 0
host1:
Code:
$ dd if=/dev/zero of=testfile bs=16k count=196608
196608+0 Datensätze ein
196608+0 Datensätze aus
3221225472 Bytes (3,2 GB) kopiert, 58,2955 s, 55,3 MB/s

ifconfig:
  inet Adresse:192.168.1.1  Bcast:192.168.1.255  Maske:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  RX packets:2277332 errors:0 dropped:0 overruns:0 frame:0
  TX packets:5445228 errors:0 dropped:0 overruns:0 carrier:0
  Kollisionen:0 Sendewarteschlangenlänge:1000
  RX bytes:165588568 (157.9 Mb)  TX bytes:3559215059 (3394.3 Mb)
  Basisadresse:0xa000 Speicher:da800000-da820000
host2:
Code:
$ dd if=/dev/zero of=testfile bs=16k count=196608
196608+0 records in
196608+0 records out
3221225472 bytes (3.2 GB) copied, 71.6973 s, 44.9 MB/s

ifconfig:
  inet addr:192.168.1.5  Bcast:192.168.1.255  Mask:255.255.255.0
  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  RX packets:5443044 errors:0 dropped:0 overruns:0 frame:0
  TX packets:2275232 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:3558945484 (3394.0 Mb)  TX bytes:163996442 (156.3 Mb)
  Interrupt:17
Where else could be a bottle neck?

Thx, georg
 
Old 07-08-2007, 08:33 PM   #4
t1n0m3n
Member
 
Registered: Nov 2004
Distribution: Kubuntu 7.10
Posts: 69

Rep: Reputation: 15
What am I looking at here?
Those throughput times look like read *not write* times to me.
Your write to hard drive throughput should be much slower. (Unless these are RAID setups of course)

When transferring files, I take it as a general rule of thumb that I am limited by the write speed of the hard drive of the machine that I am transferring to.

I have much better throughput when transferring files *from* my laptop than *to* my laptop.

You could also be running into CPU issues as well. Although my money is on Hard Drive is your bottle neck.

*EDIT* BTW, I have 2 raptors in Raid 0 on my home machine, so it is a lot more obvious to me when I transfer files back and forth. Although I have yet to check if my home machine's write speed is actually faster than my laptop's read speed.

Last edited by t1n0m3n; 07-08-2007 at 08:38 PM.
 
Old 07-08-2007, 08:43 PM   #5
t1n0m3n
Member
 
Registered: Nov 2004
Distribution: Kubuntu 7.10
Posts: 69

Rep: Reputation: 15
Some other thoughts....

If you were to mount part of your ram as a drive, I wonder what the result would be. (Transferring from ram drive to ram drive that is)

Also, if you were to disclose what your HD setups on both machines were, that may shed some light on this.
 
Old 07-09-2007, 12:14 PM   #6
g_k
LQ Newbie
 
Registered: Jul 2007
Posts: 3

Original Poster
Rep: Reputation: 0
Quote:
What am I looking at here?
Those throughput times look like read *not write* times to me.
I really just copy-pasted from console, so they are write times.
Host1 has a standard IDE hard drive I bought last year, and on host2 there is a software raid5 with 3 disks.

However, suddenly it struck me that scp cannot be a good benchmark because it encrypts the data and maybe also adds some other overhead. So I did some tests with dd on the mounted NFS share:
read: ~50MB/s
write:~15MB/s

I still don't understand the bad write performance (I double-checked those local 45MB/s), however, I can live with this situation now...

Thx for your help! Regards, georg
 
Old 07-09-2007, 06:09 PM   #7
thorn168
Member
 
Registered: Oct 2004
Location: USA
Distribution: Vector Linux 5.1 Std., Vector Linux 5.8 Std., Win2k, XP, OS X (10.4 & 10.5)
Posts: 344

Rep: Reputation: 42
How long are the cables? Have they been properly terminated? Why are you using a crossover cable? (what is its purpose in the network scheme?)

As for hardware...how fast is your PCI bus? I would guess that the PCI bus is the source of the bottleneck.

Network-->NIC-->PCI bus-->CPU-->HD

If the network and the Nic are fast and the CPU and the HD are fast, the only thing that could slow them all down is the PCI bus.
 
Old 07-09-2007, 07:17 PM   #8
michaelnel
LQ Newbie
 
Registered: Apr 2005
Location: San Francisco, California
Posts: 12

Rep: Reputation: 0
Did you try jumbo frames? The nics and the switch would have to both be set for jumbo (mtu=9000)
 
Old 07-13-2007, 03:57 PM   #9
t1n0m3n
Member
 
Registered: Nov 2004
Distribution: Kubuntu 7.10
Posts: 69

Rep: Reputation: 15
When I said "Those throughput times look like read *not write* times to me." I meant that the numbers themselves look like read times.
It also seemed odd that your host1's single drive is writing faster than your raid array. Could be to the newer drive technology.

So a couple of assumptions:

1. Transfers from host1 to host2 should be faster due to host2 being able to write to its raid drive at a faster rate.
2. Transfers from host2 to host1 should bottleneck on the 15MB/s write speed of host1's standard IDE HD.

A comparison of these two assumption scenarios plus an IPerf benchmark (in both directions) might be interesting.

The only other thing I can think of is a bad NIC. I have seen a NIC give asymmetric throughput results before.

thorn168: You are assuming that these NICs are actually on a PCI bus.
g_k: what kind of NICs are in these machines and if they are built-in, what are the makes and models of the motherboards?

Last edited by t1n0m3n; 07-13-2007 at 04:32 PM.
 
Old 04-07-2010, 05:49 PM   #10
Abida54
LQ Newbie
 
Registered: Dec 2009
Posts: 8

Rep: Reputation: 0
According to my experience Ethernet throughput is highly dependent on the size of the packets sent. maybe the files you are copying are small. I had the same problem a while ago.

here is the link that shows how the throughput is dependent on the size of the packets:

http://support.novell.com/techcenter...a19960103.html
 
Old 04-07-2010, 08:15 PM   #11
jefro
Guru
 
Registered: Mar 2008
Posts: 11,590

Rep: Reputation: 1419Reputation: 1419Reputation: 1419Reputation: 1419Reputation: 1419Reputation: 1419Reputation: 1419Reputation: 1419Reputation: 1419Reputation: 1419
Might peek here too.

http://www.enterpriseitplanet.com/ne...le.php/3497796

Almost the same.

http://www.enterprisenetworkingplane...0950_3485486_2

Last edited by jefro; 04-07-2010 at 08:18 PM.
 
  


Reply

Tags
gigabit, jumbo


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Slow gigabit ethernet reggie Linux - Networking 3 05-20-2007 08:06 AM
Gigabit ethernet detection shankara Linux - Hardware 1 04-30-2007 02:20 AM
Broadcom Gigabit Ethernet Slow with Suse 9.3 kuhazor Linux - Hardware 4 07-17-2005 11:02 AM
SuSE 9.3 Gigabit Ethernet fulch92a Suse/Novell 4 04-27-2005 04:16 PM
gigabit ethernet quercusalba Linux - Hardware 2 12-29-2004 05:09 PM


All times are GMT -5. The time now is 01:55 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration