Performance of fwrite() on usb mass storage device?
 

Hi,
I am trying to write to usb flash drive using a C program. I have just created some files in USB device and doing my tasks using fopen,fread and fwrite on those files. I was just wondering what is the standard data rate that can be achieved using fwrite() on a file in usb. Secondly what is better way of transferring data to USB flash drive using C Program? Obviously I am a newbie :)

Regards
AB

It really depends on what you are doing.

A USB storage device is treated no differently than any other storage device... Disks are disks.

It usually is fastest to use the largest buffer...

But it sucks for random block I/O, which is much better in small sizes.

If all you are doing is copying a file to a USB flash drive, how about just using cp?

Thanks for instant reply
 

thanks jpollard,
My code reads data in the form a kernel module in the form of 4KB chunks and these chunks are written to files in USB. Expected data rate is 32MB/s and I am using Usb3.0 flash drive. I was wondering whether fwrite() can serve the purpose?

Regards

Why not just use cp?

4KB chunks may not be the best size - 8/16/32k may be better, but that depends on the filesystem and the device...

Whether you use fwrite/write or anything else, is up to you. No single one is better.

Thanks,

My implementation is i.e. there are 4 files in USB disk.
file0.txt
file1.txt
file2.txt
file3.txt

each time data from kernel module arrives in form of 4KB it has to be appended to a file (in round robin pattern). So I guess cp might not work as it will replace the file with new file. 'cp' takes directory locations (files) as parameter where I have only buffers in my program that will be copied to files.Also each time 'cp' is called fopen and fclose will also be called that might slow down overall operation. So I think I have only choice of fwrite() as per my understanding (which might be questionable :D ).

The speed of fwrite is irrelevent.

Concatenating modules doesn't exactly make any sense - they can't be used that way.

The problem of speed depends on the device. As I understand it, the flash is divided up into rather large blocks - to add a single 4K block to a file requires reading up to 64k, replacing the one 4K update, then writing the 64K somewhere else (wear leveling). So 4K could be rather bad on the device.

And that is much slower than a cp would be, and a lot less overhead (and wear and tear).

But you can use whatever you want. The system will handle it.

Data rates on flash media vary greatly and it depends on the type of write your attempting to do. Since these are 4k blocks your probably doing non sequential writes to the device. In that case then writes will be rather slower in magnitude to reads. USB flash drives are probably the worst for performance in this area considering they are usually a low cost solution. However there is variance even in them with regards to quality of flash memory used.

Like jpollard said writing data to the device in 4k blocks is probably a bad idea because of the general wear and tear on the device ( why putting a swap file on flash is also not good ).

You could however instead of writing to a file could create a buffer for the data until it reaches a particular size and then dump to the disk when full or whatnot. Several ways that could be implemented.