LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 06-03-2007, 08:31 AM   #46
Bobymc
Member
 
Registered: Apr 2006
Location: INDONESIA
Distribution: SLAX,Damn S.L,Suse,Mandrake,Rd HAT62,72,73,90, Mandriva2k6, FEdora, SUNmicrosys.
Posts: 269

Rep: Reputation: 30
Question Peak or Real?


Quote:
Originally Posted by chrisphillips
Rubbish. Cat5e can saturate a gigE connection. Using jumbo frames (9000 MTU) I can get 989 Mbps user data throughputs with TCP. The 11 Mbps is ethernet/IP/TCP headers.


Chris
Would you be more specific and give details?
Like hardware spec/config/net distance etc......
That will prove your state.
How about IpV6 have you try that?
 
Old 06-23-2007, 05:19 AM   #47
chrisphillips
LQ Newbie
 
Registered: Jan 2007
Location: Sydney
Distribution: Debian & Ubuntu
Posts: 6

Rep: Reputation: 0
Quote:
Originally Posted by Bobymc
Would you be more specific and give details?
Like hardware spec/config/net distance etc......
That will prove your state.
How about IpV6 have you try that?
??? I have nothing to "prove". I routinly transfer data between sites 500km on a dedicated 1 Gbps network. Any decent (intel pro/1000, nvidia etc) NICs can easily satutate the network.

The tricky bit is transfering data internationally Australia to Europe has a 330msec RTT. That makes TCP transfers difficult. But I can get > 700 Mbps with UDP.

Never tried IPv6 no need yet.

The bottom line is any modern mobo and a half decent NIC (not realtek they are crap!) can sustain 1 Gbos.
 
Old 06-23-2007, 07:28 PM   #48
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Original Poster
Rep: Reputation: 51
Quote:
Originally Posted by chrisphillips
??? I have nothing to "prove". I routinly transfer data between sites 500km on a dedicated 1 Gbps network. Any decent (intel pro/1000, nvidia etc) NICs can easily satutate the network.

The tricky bit is transfering data internationally Australia to Europe has a 330msec RTT. That makes TCP transfers difficult. But I can get > 700 Mbps with UDP.

Never tried IPv6 no need yet.

The bottom line is any modern mobo and a half decent NIC (not realtek they are crap!) can sustain 1 Gbos.
Because I'm still having a problem duplicating your results, I hookup up two beefy file servers with GigE Intel NICs using a crossover cable - computer to computer, no switches, maybe 3 meters of cable, MTU of 9000 (actually 9014) on both machines. I still only get 3-400Mb/s.

If you would be so kind, could you post your timing data? i.e. the result of "time dd if=/dev/zero of=/nfs/share/foo bs=1M count=1000" - that's assuming, of course, you're using NFS. If not, what are you using for transfers? could you post some test results showing the speeds you're getting?

Thanks.
 
Old 06-23-2007, 11:29 PM   #49
chrisphillips
LQ Newbie
 
Registered: Jan 2007
Location: Sydney
Distribution: Debian & Ubuntu
Posts: 6

Rep: Reputation: 0
Quote:
Originally Posted by BrianK

If you would be so kind, could you post your timing data? i.e. the result of "time dd if=/dev/zero of=/nfs/share/foo bs=1M count=1000" - that's assuming, of course, you're using NFS. If not, what are you using for transfers? could you post some test results showing the speeds you're getting?

Thanks.
Oh - your complaining that you cannot get decent performance on nfs over gigabit links. Well that is another question entirely!

I am taking about ethernet TCP speeds - this is memory to memory. If you are using nfs then the speeds could be terrible. Have you tested the raw speed of the device you are writing to? A single disk could max out at 300-400 Mbps plus the substantial nfs overhead.

I use all my own software for network (and disk timing). Network transfer of data is my job to I cannot reply on tools such as "dd".

Using my own software which reads from a file, sends down a TCP socket and does the opposite on a second PC I can sustain about 600 Mbps (disk-to-disk). This is limited by the RAID array write speed (remember write speeds are usually a lot slower than read speed).

CPU usage is < 20%. This is a dual opteron (242?) system with a Tyan mobo. It is 2 years old.

You may want to check stuff like TCP offload. The tcp segmetation offload on the pci express intel pro/1000 desktop seems totally broken.

Cheers
Chris
 
Old 06-25-2007, 01:08 PM   #50
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Original Poster
Rep: Reputation: 51
Quote:
Originally Posted by chrisphillips
I use all my own software for network (and disk timing)...
Ah ha! Now we're on to something. It sounds like there is not an available protocol for sharing files at such high speeds - nfs, samba, ftp, etc. etc. etc. actually do have some sort of limit of around 3-400 Mb/s (what everyone here has been seeing). Does that sound accurate? Is your code open source? If so, where is it, I'd love to check it out.

For the record, yes, I've tested write speed on the receiving device to be around 70MB/s (that's bytes not bits). I've also tested using nfs, ftp, smb, and scp (and whatever windows-windows computers use for transfers) - all with pretty similar results, in several different facilities.
 
Old 06-25-2007, 04:12 PM   #51
Snowbat
Member
 
Registered: Jun 2005
Location: q3dm7
Distribution: Mandriva 2010.0 x86_64
Posts: 338

Rep: Reputation: 31
FTP has no such limit - here's 88 MB/s over GigE. Since the server processor was maxed out during the transfer, I'd expect more if my server had a faster processor or less interrupt-hungry NIC or was configured for jumbo frames. The key to speed here is that the file was served from a ramdisk and saved to /dev/null. The server and client hard drives can handle only about half that speed and would ~half the transfer rate if used as a source or destination.

# wget -O /dev/null --ftp-user={REDACTED} --ftp-password={REDACTED} ftp://192.168.1.50/ramdisktest
--16:57:50-- ftp://192.168.1.50/ramdisktest
=> `/dev/null'
Connecting to 192.168.1.50:21... connected.
Logging in as {REDACTED} ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD not needed.
==> PASV ... done. ==> RETR ramdisktest ... done.
Length: 512,000,000 (488M) (unauthoritative)

100%[===========================================>] 512,000,000 88.81M/s ETA 00:00

16:57:56 (88.14 MB/s) - `/dev/null' saved [512000000]


Furthermore, here is the same test done locally on the server showing just shy of 200 MB/s over FTP via the loopback interface:
# wget -O /dev/null --ftp-user={REDACTED} --ftp-password={REDACTED} ftp://192.168.1.50/ramdisktest
--17:28:40-- ftp://192.168.1.50/ramdisktest
=> `/dev/null'
Connecting to 192.168.1.50:21... connected.
Logging in as {REDACTED} ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD not needed.
==> PASV ... done. ==> RETR ramdisktest ... done.
Length: 512,000,000 (488M) (unauthoritative)

100%[===========================================>] 512,000,000 198.34M/s

17:28:42 (198.29 MB/s) - `/dev/null' saved [512000000]

Last edited by Snowbat; 06-25-2007 at 04:13 PM.
 
Old 06-26-2007, 01:11 AM   #52
Snowbat
Member
 
Registered: Jun 2005
Location: q3dm7
Distribution: Mandriva 2010.0 x86_64
Posts: 338

Rep: Reputation: 31
I've had a brief look at serving from the ramdisk over GigE with nfs, smb, and scp too...

nfs: Unfortunately tmpfs (used to create my ramdisk) is one of the local filesystems known not to work with Linux NFS servers. The tmpfs filesystem was not visible over nfs - the directory it was mounted on showed instead.

smb: Much slower - best I've seen was about 9.5 MB/s. Not CPU-bound either as server and client both showed ~90% idle during transfer. This was surprising. Maybe there are samba tweaks to speed this up.

scp: Also slow at 13.6MB/s but server processor was maxed out with ssh consuming 99% so I'd expect more with a faster processor.
 
Old 06-26-2007, 01:24 PM   #53
lord-fu
Member
 
Registered: Apr 2005
Location: Ohio
Distribution: Slackware && freeBSD
Posts: 676

Rep: Reputation: 30
Quote:
Thanks for the suggestions.
Actually I would like to second the thanks for information, this has been a question of mine as well, I have a setup similar to the OP's and this info has helped me figure out a few new things. Muchos Gracias!

Last edited by lord-fu; 06-26-2007 at 01:25 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Mandrake 10 Internet very slow (<1kb/sec) while windows got 50k/sec SafeTechs Mandriva 13 09-01-2006 04:07 PM
Fedora 4: Asus A7n8x Can't get Marvell GigE to light up lsgko Linux - Networking 1 08-24-2005 06:14 AM
hdparm 64MB in 19.68 sec=3.25 MB/sec illtbagu Linux - General 11 06-26-2003 07:03 PM
GigE with some of the cheaper cards mastahnke Linux - Networking 0 04-16-2003 07:57 PM
which distr do I use for a 300MB Laptop tuyojrc Linux - Distributions 6 04-24-2002 05:29 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 01:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration