I am having problems copying large files (>400MB) from an
ext3 linux (Red Hat) filesystem onto a USB device (Maxtor external
hard drive) formatted with a vfat filesystem.
I am using the cp command to do the
copy. Smaller files copy as expected. The USB device is
mounted read/write with uid and gid set to a specific user.
I have been using an 888MB file as a test file; when I try to
cp this file onto the USB device, the command locks up and
does not return (I've waited > 2 hours); in fact, the entire
machine locks up, and I am not able to establish another login
session, not even as root from the console. Other existing
sessions also freeze.

The var/log/messages files doesn't provide any helpful messages.
I have tried copying the same 888MB file within the same linux ext3
filesystem, and it works as expected (I think this tells me
that cp is OK, and that the ulimits are OK).

I tried cp-ing the file as root, but it said the USB mounted
filesystem was read-only, presumably because it was mounted with
the gid and uid options of a specific user. After this error message,
I tried another small file as the non-root user, and this also
failed with a "filesystem read-only" message. I unmounted, and
remounted to make the vfat filesystems read-write once again.

The vfat filesystem I am trying to write to is 30 GB in size, with
approx 22 GB free space. I then tried using sftp; I sftp'ed to the same
machine from a working directory on the USB device, and issued a "get"
command to get the 888MB file. It worked, but was excruciatingly slow;
sftp reports transfer stats while copying; the intitial rate was between
10 - 20 MB/sec, but it began to slow down around the 400MB mark, and
became progressively slower; by completion, it was down to 700kb/sec.
It took about 30 mins for the 888MB file to complete.

Can anyone suggest what to try next? The USB device needs to be vfat because it is being used to transfer files to windows. Apologies for the long post, and thanks to any responders.

Can you post the output of
lsmod
and
lspci
please?

Also, is the USB interface supposed
to be 1.1 or 2?

As for /var/log ... you may want to re-compile
the kernel (which version are you using, btw?)
with USB mass storage verbose debug on ...


Cheers,
Tink

Here is lsmod output:
Module Size Used by Not tainted
soundcore 7044 0 (autoclean)
nls_iso8859-1 3516 2 (autoclean)
nls_cp437 5148 2 (autoclean)
vfat 13196 2 (autoclean)
fat 40184 0 (autoclean) [vfat]
usb-storage 75296 2
ide-cd 35776 0 (autoclean)
cdrom 34144 0 (autoclean) [ide-cd]
lp 9220 0 (autoclean)
parport 39072 0 (autoclean) [lp]
autofs 13780 0 (autoclean) (unused)
e1000 72320 1
floppy 59056 0 (autoclean)
microcode 5248 0 (autoclean)
keybdev 2976 0 (unused)
mousedev 5688 1
hid 22404 0 (unused)
input 6208 0 [keybdev mousedev hid]
usb-ohci 23688 0 (unused)
usbcore 83168 1 [usb-storage hid usb-ohci]
ext3 95784 11
jbd 56856 11 [ext3]
ips 45572 13
sd_mod 13744 30
scsi_mod 116904 3 [usb-storage ips sd_mod]

Here is lspci output:
00:00.0 Host bridge: ServerWorks CNB20-HE Host Bridge (rev 33)
00:00.1 Host bridge: ServerWorks CNB20-HE Host Bridge
00:00.2 Host bridge: ServerWorks CNB20-HE Host Bridge
00:06.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:0f.0 Host bridge: ServerWorks CSB5 South Bridge (rev 93)
00:0f.1 IDE interface: ServerWorks CSB5 IDE Controller (rev 93)
00:0f.2 USB Controller: ServerWorks OSB4/CSB5 OHCI USB Controller (rev 05)
00:0f.3 ISA bridge: ServerWorks GCLE Host Bridge
00:10.0 Host bridge: ServerWorks: Unknown device 0101 (rev 05)
00:10.2 Host bridge: ServerWorks: Unknown device 0101 (rev 05)
00:11.0 Host bridge: ServerWorks: Unknown device 0101 (rev 05)
00:11.2 Host bridge: ServerWorks: Unknown device 0101 (rev 05)
06:08.0 Ethernet controller: Intel Corp. 82546EB Gigabit Ethernet Controller (Copper) (rev 01)
06:08.1 Ethernet controller: Intel Corp. 82546EB Gigabit Ethernet Controller (Copper) (rev 01)
08:02.0 RAID bus controller: IBM ServeRAID Controller

The kernel version is 2.4.21-4; this is Red Hat Enterprise Linux 3.0 ES.
The USB interface is supposed to be 2.

Thanks for your help. After thinking about this a while, this seems typical of a filled buffer of some sort; that is, the buffer fills up after a certain size, and thereafter, the copy becomes excruciatingly slow. Just a thought.

Wrong thought, though ;)

usb-ohci is wrong, if the chipset is
USB 2 you need to load ehci-hcd.

Otherwise you're shovelling bits @
150KB/s which may seem somewhat
slow with a file that size :}


Cheers,
Tink

But I am seeing copy speeds around 20MB/sec with smaller files. Even with the large file (888M), I see a rate up to 20MB/sec for the first 300M or so; then the rate drops progressively. If the wrong module/driver is in use, wouldn't the rate be slow in all situations?

Maybe that's just while it's using RAM to
temporarily buffer? How much RAM does
the box have you said? :) And which tool
are you using to measure the speed, anyway?


The only reliable method would be
time cp foo bar

Anything else is guesstimations, and can heavily
vary with system load, memory utilisation and how
the program "thinks" it should measure ;}


Cheers,
Tink

This machine has 2.5G ram and 4G swap. I've been monitoring the transfer rate using rsync with the stats/progress flags; this reports the rate every few seconds. Unlike cp, rsync does eventually finish the copy, but the rate steadily decreases. I also monitored output from "top" in another session while the rsync is running; top reports no swap is used, and free memory only decreases a bit. It does show that %iowait increases to 99% about the same time the transfer rate begins to decline. Do you know if the USB device has some sort of write buffer? If so, how can it be changed?

Sorry for the delay ...

What does free say about the buffers?
I have to say that even with USB 2.0 20 MB/s
would be pretty darn good :}

Did you try the kernel recompile-thing?
USB Mass-storage verbose debug?

Or are there any indications in syslog,
messages or debug?


Cheers,
Tink

We think the problem is solved. We installed a D-Link DU-520 card in the machine, rebooted, and it copied a 888MB test file in around 3 mins. We noticed it added a couple of lines to modules.conf, the relevant one being: alias usb-controller2 ehci-hcd.

Told you it was a driver issue, using USB 1.1 rather
than 2.0 ;)


Cheers,
Tink