ext3 usb hd - slow transfer rates (ubuntu/manual mount)

chochem · 06-02-2008, 05:14 PM

I have an external usb harddisk that's formatted as ext3. I have a setup in which it is manually mounted (have some bad HAL experiences) with a simple 'sudo mount -o noatime -o nodiratime -L [label] [mount point]' command. I have been having some problems with slow transfer rates (more specifically the apttern was usually fast speeds up until the first 60-80 Mb, then snail speed)

On a completely fresh install I tested it and transfer rates in both directions was excellent with a 700 Mb avi file. I then rebooted and now all transfers are horribly slow from the get go. I've check dmesg but all it say when I mount is:

Code:

[ 9073.234644] EXT3 FS on sdb1, internal journal
[ 9073.234663] EXT3-fs: mounted filesystem with ordered data mode.

And this the dmesg output, when I connect the drive:

Code:

[ 3741.647806] usb 3-3: new high speed USB device using ehci_hcd and address 3
[ 3742.962939] usb 3-3: configuration #1 chosen from 1 choice
[ 3743.070705] scsi3 : SCSI emulation for USB Mass Storage devices
[ 3743.079315] usb-storage: device found at 3
[ 3743.079324] usb-storage: waiting for device to settle before scanning
[ 3749.034861] usb-storage: device scan complete
[ 1323.009910] scsi 3:0:0:0: Direct-Access     WD       5000AAV External 1.65 PQ: 0 ANSI: 4
[ 2250.774835] sd 3:0:0:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[ 3750.266722] sd 3:0:0:0: [sdb] Write Protect is off
[ 3750.266740] sd 3:0:0:0: [sdb] Mode Sense: 21 00 00 00
[ 3750.266748] sd 3:0:0:0: [sdb] Assuming drive cache: write through
[ 2251.801340] sd 3:0:0:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[ 3752.046261] sd 3:0:0:0: [sdb] Write Protect is off
[ 3752.046275] sd 3:0:0:0: [sdb] Mode Sense: 21 00 00 00
[ 3752.046283] sd 3:0:0:0: [sdb] Assuming drive cache: write through
[ 3752.046297]  sdb: sdb1
[ 3752.575950] sd 3:0:0:0: [sdb] Attached SCSI disk
[ 3752.576058] sd 3:0:0:0: Attached scsi generic sg2 type 0

I don't know what "Assuming drive cache: write through" or "mode sense" mean but apart from that, I can't see anything particularly terrifying or anything which screams that something is wrong. I've tried mounting it as ext2 and with no options but that didn't seem to change much. My only idea is that the connection is being treated as USB 1 rather than 2 but why that should be I could not say (all my ports are usb2 capable).

EDIT: I should just add that I get much better speeds connecting to the drive in Windows (through ext2ifs for windows), so I don't think the drive as such is to blame.

strick1226 · 06-02-2008, 06:25 PM

Have you tried running hdparm speed tests to see what it reports? I have a feeling you're right, in that it sounds as if it's running at usb 1.0/1.1 speed instead of 2.0...

more info on hdparm from the gentoo wiki:

http://gentoo-wiki.com/HOWTO_Use_hdp...nce#hdparm_-tT

A USB 1.0 connection will transfer a max of ~1.5 MegaBytes/s (MB/s), whereas USB 2.0 provides up to 60 MB/s--a bit faster than most IDE hard drives in a single-drive config allow (outside of small bursts, anyway)... The "buffered disk read" results should provide the most realistic benchmark, I think...

Hope this helps.

smbell100 · 06-02-2008, 07:35 PM

I have had similar problems using a Samsung 320Gb hard drive. I use it on 7 different systems, using it to run my own operating system on 2 of them.

On the 2 work machines I use, no problems. On a 4200X2 no problems. On a 900Mhz Duron with USB1, no problems. On the others (various processors and motherboards), it sometimes only connects at USB1 speeds, and sometimes loses the external drive completely. When this happens I have found that the only way to get the drive reconnected is to shut everything down, switch the power off, wait a few seconds and then switch the power on again. A reboot doesn't work as the BIOS either hangs when trying to find the drive, or fails to see it at all.

I am using a mix of Slackware 12.0 and Mandriva 2008.0 Powerpack. My assumption is that either some motherboards aren;t entirely compatible with the drive, or that the USB drivers for a particular chipset are a bit flaky. I haven't had time to investigate more thoroughly yet. I just work around it, using a network and a machine that is reliable with the drive.

I have been looking for an answer, and have found these two links which look interesting;

https://bugs.launchpad.net/ubuntu/+s....20/+bug/61235

http://taint.org/2006/12/13/191554a.html

Hope this helps

chochem · 06-06-2008, 12:22 PM

Hi, thanks for the replies, guys. Sorry for the delay in follow up - been kinda busy.

strick: Here's hdparm -Tt /dev/sdb1

Code:

/dev/sdb:
 Timing cached reads:     2 MB in  5.72 seconds = 358.19 kB/sec
 Timing buffered disk reads:    2 MB in  5.62 seconds = 364.23 kB/sec

Pathetic.... Truth be told, this is from a fat32 partition (I formatted the drive in a fit of frustration) but based on recollection, I'd say it's exactly the same as before. I've also dug out an former laptop drive, put it in a usb-connect-box-thingy and there's no real difference - theyøre both mind numbingly slow and neither seems to complain to dmesg. Browsing the drive with my file manager (emelfm2) will occasionally cause it to stop responding for long periods at a time.

smbell100: Thanks for the links but both cases seem different from mine - my drive doesn't disconnect as such (I think that's an issue with the dynamic mounting that HAL si for - or have I misunderstood something? - it certainly reminds me of the problems I had with HAL - would get into this annoying loop of mounting and dismounting my Sansa). It's just got slow transfer speeds (and when mounting/dismounting, the 'drive questioning' is pretty damn slow too).

EDIT: One vague idea... As I wrote in my first post, I believe I did a test on my fresh install and it looked good. Then somehow it all went bad. Now the only major things I've done with the system since install is upgrading and I believe there was a kernel upgrade between plain 8.04 and now - could this be the cause of it? Any input is valued - reinstalling is so tedious, especially if it's for nothing.

smbell100 · 06-06-2008, 05:41 PM

That is different to mine - mine disconnects so thoroughly that a reboot fails to find the disk.

Sorry I can't help more

chochem · 06-08-2008, 01:44 AM

Got it. Tried booting the old kernels without any change, so I went ahead and reinstalled. The difference is unmistakable...

Code:

/dev/sdb1:
 Timing cached reads:   652 MB in  2.00 seconds = 325.95 MB/sec
 Timing buffered disk reads:   58 MB in  3.07 seconds =  18.90 MB/sec

I have locked down the kernel to 2.6.24-16 but I don't really think that's the issue. The culprit appeared to be the 'noapic' boot option that is commonly recommended as a quick 'fix' for the 'MP-BIOS bug: 8254 timer not connected to IO-APIC' error message. That removed from the grub menu.lst, everything has behaved since then (knock on wood...)

EDIT: It seems like this just brought on the next issue, namely that at seemingly random points, speed would drop from lightning fast to a crawl and make the shell/file manager unresponsive. Making a wholly uneducated guess, I came to the conclusion that it might be an interrupt thing and went looking for another boot parameter. Seems like 'pci=routeirq' does the job but I'm not gonna sell this skin just yet...

strick1226 · 06-09-2008, 07:22 AM

chochem,

Congrats on finding a solution!
If I had to take a guess, disabling APIC equaled less IRQ's, so the computer was having to share the limited number of them--probably resulting in less-than-ideal transfer rates, depending on whether or not the USB controller was on a particularly heavily-shared IRQ.

I've not had to use the noapic option myself, nor the pci=routeirq ... but definitely will take note of them.

Again, congrats on the (double) fix. Hope it stays stable for you!