File transfer rates?

fur · 02-25-2005, 06:25 PM

The encryption that scp uses is also going to lower the transfer speeds. Maybe you could try with ftp? You could also try replacing ethernet cables/cards if you have extras to try to rule out a problem with them.

Matir · 02-25-2005, 08:34 PM

Indeed. Though, is there anything, other than replacing the motherboard (or adding a PCI IDE controller) I can do to improve performance?

In anticipation of questions:

Code:

fserv root # hdparm /dev/hdb

/dev/hdb:
 multcount    = 16 (on)
 IO_support   =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    = 256 (on)
 geometry     = 24792/255/63, sectors = 203928109056, start = 0

fserv root # hdparm -i /dev/hdb

/dev/hdb:

 Model=Maxtor 6B200P0, FwRev=BAH41B70, SerialNo=B413ME6H
 Config={ Fixed }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=57
 BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=16
 CurCHS=17475/15/63, CurSects=16513875, LBA=yes, LBAsects=66055248
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 *udma2 udma3 udma4 udma5 udma6
 AdvancedPM=yes: disabled (255) WriteCache=enabled
 Drive conforms to: (null):

fserv root # lspci | grep IDE
0000:00:14.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)

I'm not posting the hdparm -I output for now, unless anyone wants it. It's long and really doesn't show THAT much more, as far as I can tell.

uberNUT69 · 02-25-2005, 10:34 PM

I can see two significant items in your hdparm output:
- lack of 32bit IO support
- the drive is only doing dma2 instead of 5/6

Do you have some other slow device on the primary port?
I'm under the impression that this is bad practice, and pretty sure that you can't 'fix' it with software.

here's output of mine for reference: (I have an identical drive on hdb too)
spitfire:~# hdparm /dev/hda

/dev/hda:
multcount = 0 (off)
IO_support = 1 (32-bit)
unmaskirq = 1 (on)
using_dma = 1 (on)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 256 (on)
geometry = 65535/16/63, sectors = 60022480896, start = 0
spitfire:~# hdparm -i /dev/hda

/dev/hda:

Model=ST360021A, FwRev=3.19, SerialNo=3HR1J14S
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=2048kB, MaxMultSect=16, MultSect=off
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=117231408
IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=no WriteCache=enabled
Drive conforms to: device does not report version:

* signifies the current active mode

spitfire:~# lspci | grep IDE
0000:00:11.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06)

Matir · 02-25-2005, 10:43 PM

I think the big problem on that might actually be the chipset. The box is just a Pentium II. I seem to recall reading somewhere that 32 bit transfer mode could be dangerous. In fact, ALL my boxes have 16 bit transfer mode. I'll read up on that for now.

uberNUT69 · 02-25-2005, 11:05 PM

The dma transfer mode is more of an issue I think.
Why is it only only mode 2?

Matir · 02-26-2005, 11:06 AM

I believe the IDE chipset does not support anything higher than mode 2. Is there a way I can test higher modes with no risk to data? (The hdparm man page indicates that setting the wrong mode may be damaging to data)

uberNUT69 · 02-26-2005, 07:06 PM

If the auto setting in bios is giving you dma2 then that will probably have to do.
Might pay to have a look at bios though and double check that dma2 is the fastest mode.
I wouldn't recommend trying a fiddle with hdparm.
However, before you get completely side-tracked from your original question, have you tried a transfer from, say hda to hdb and tested the rate? Got anything on your secondary port? Try transfers from that as well.
Have you shown rates for non-scp transfers yet?
Have both the systems got enough ram? ... ie. perhaps swapping is affecting transfers.

hmmm ... time for morning coffee

jschiwal · 02-28-2005, 07:45 PM

If the ide chipset on your computer is limited to dma 2 then you could install an ultra ata 133 ide controller board. That is assuming that the drives aren't to old for the higher modes. A new controller card can be had for under 20 bucks.

Matir · 02-28-2005, 09:04 PM

Well, the /dev/hdb (/home) drive is brand spanking new as of about a week ago. It's ATA/133. I have to wonder how much it's worth though... I'll look into it.

gomichan · 03-15-2005, 03:27 AM

I agree with fur.
scp can consume a significant amount of CPU overhead while it
tries to encrypt the data intended to be transferred.

Before you go ahead with spending money on a hardware investment,
I recommend that you experiment with ftp and rcp to see what kinds
of transfer rates you achieve.

Sincerely,

baldy3105 · 03-18-2005, 11:43 AM

1.7 MB/S is VERY slow. Even a hardrive in PIO mode could more than cope with that. This sounds to me like a duplex mismatch. You need to check for error son your switch ports and nic's if anything is reporting collisions then something is not Full duplex. Remember one end as Full and the other as auto DOES NOT WORK! You must have full/full, auto/auto, auto/half or half/half.

The 37% figure is from a study done that showed that if you have a lot of devices contending for ethernet there are so many collisions and retransmissions that only 30% of whats on the wire is real throughput, retransmissions are counted as overhead.

Pete