LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices


Reply
  Search this Thread
Old 02-07-2019, 06:11 PM   #1
latgarf
LQ Newbie
 
Registered: Feb 2019
Posts: 1

Rep: Reputation: Disabled
USB flash speed slow: 20MB/s actual vs 250MB/s advert, USB 3.1 Gen 1, 512GB Kingston HXS3


I have a 512GB Kingston HyperX Savage USB flash drive. It's USB 3.1 Gen 1 and its advertised write speed is 250 MB/s. I use it for backup once a week.

The problem is the slow average write speeds. For instance, when I copy my 80 GB /images folder (containing 20k files of size 1kB to 30MB) onto this empty flash drive using command
Code:
$ rsync -ahv --info=progress2 /source-dir /destination-dir
I observe these average write speeds:
Code:
100 MB at 230 MB/s
200 MB at 210 MB/s
300 MB at 190 MB/s
  1 GB at 170 MB/s
  2 GB at 160 MB/s
  3 GB at  65 MB/s
  4 GB at  40 MB/s
  5 GB at  33 MB/s
 10 GB at  24 MB/s
 20 GB at  21 MB/s
 30 GB at  20 MB/s
 60 GB at  20 MB/s
I've tried formatting the drive with various filesystems: fat, ext4, f2fs - similar behavior. I have 16GB of RAM.

I wonder about the nature of this speed decrease and if there are any ideas how I could try to improve it?

Last edited by latgarf; 02-11-2019 at 01:59 PM.
 
Old 02-14-2019, 05:37 AM   #2
marius162
LQ Newbie
 
Registered: Apr 2015
Posts: 13

Rep: Reputation: Disabled
Not sure if it's related, but I've been struggling with a problem around my method of backing up my system ssd which reminds me of what you are describing. I'm using cinammon as a live usb maintenance and rescue system, storing the 240 GB sandisk 3d ultra ssd with the operating systems to a usb3 4tb mechanical harddrive with dd. However, this did not work overly reliable since sometimes, storing the 240 gb would take around an acceptable 40 minutes and at other times, the process would begin with a transfer rate of approximately 220 MB/s but then fall rapidly in a manner similarly to that which you described. At first, I thought it might be due to the backup drive being full but it turned that that's not it. My final solution was to buy a 384 gb sandisk which I now use as the destination for the backup. This way, it only takes 15 minutes for the complete process although the phenomon surfaces here from time to time, too; removing, reinstalling and reformatting (with ext4) the only partition on the 384 gb makes it go away, at least for once.

Last edited by marius162; 02-14-2019 at 05:42 AM.
 
Old 02-14-2019, 07:19 AM   #3
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: macOS, Slackware
Posts: 803

Rep: Reputation: 281Reputation: 281Reputation: 281
I'm not really surprised here. For a given total fixed size, it's well known that numerous small files take a lot longer to be copied than large files.
I would be interested in knowing your write speed for a large file transfer so we can compare...

Last edited by l0f4r0; 02-14-2019 at 07:20 AM.
 
Old 02-14-2019, 02:18 PM   #4
Brains
Member
 
Registered: Apr 2009
Distribution: Debian testing
Posts: 955

Rep: Reputation: 240Reputation: 240Reputation: 240
I can write much faster to my USB2 Sandisk Cruiser blades than my USB3 Sandisk Cruizer Glides, both in a USB3 port.

There are many factors that come into play, CPU cache sizes, CPU interface bandwidth, RAM type, amount of available RAM, mobo bus speeds, amount of I/O requests etc. Advertised speeds are usually determined on optimal hardware configuration, tons of quality RAM, with minimal I/O requests. And they don't mention the size and quantity of the files that achieved those speeds.

But I'm just taking a guess here, typically when transferring from one drive to another, data is not simply read from one and sent down the bus to the other, it will be written to a buffer/cache in some form of RAM. Since reading is faster than writing, the buffer can get filled quicker with larger files, writing to the other drive is not the CPUs only job, they still have to process all other I/O requests at the same time, writing to the USB likely has a lower "nice" factor than many other I/O requests.

The way I see it, when I transfer my Debian package cache to USB for updating my offline Debian, which currently has thousands of packages, KDE will report the copying has finished, but I can't remove the USB for another 10 minutes or so till the task of writing those files to the USB is completed. Since I have 16GB of RAM, I assume that when it reported the copying finished, it finished writing to the buffer, not the drive.

So...the speeds posted in post #1, is that the speed related to writing to the buffer or the drive? My guess is that's the speed related to writing to the buffer.

So for a 100MB size you got near advertised speed, perhaps not same quality of hardware. But when the cache is maxed, you're getting around 20MB/s, that's a good indicator of the speed associated with writing to the drive with your configuration and amount of current I/O requests. The manufacturer may have only written one 3.5MB mp3 file and the computer reported 250MB/s speed, then they clicked on the icon in the taskbar to safely remove the drive, by this time the file was written and it reported the drive can safely be removed. They advertised "up to", not consistent. Did they feed you a line of BS?

When I went to purchase SD cards for my Nikon camera which has smaller cache than comparable Conons, in order to record high quality video, I wanted fast memory. Since I don't trust salesmen, I looked at benchmark comparison sites to determine which brand best suited my needs and price range.

Last edited by Brains; 02-14-2019 at 03:46 PM.
 
Old 02-14-2019, 04:06 PM   #5
zeebra
Member
 
Registered: Dec 2011
Distribution: Mageia, Slackware, Maemo
Posts: 524
Blog Entries: 2

Rep: Reputation: 108Reputation: 108
not sure exactly, but it could have something to do with NAND flash and the way it writes. It has to empty cells before it can write to them, so if something is filling the space it needs to write to, it needs to clear that first, then write. NAND flash is particularly slow at this.

What you could theoretically do to test this is to write zeroes to your disk, then format it with EXT4, and then write a large batch of files to the disk.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
3rd gen CPU in a 2nd Gen Laptop Motherboard benifits are? BW-userx Linux - Hardware 4 05-18-2016 01:56 PM
[SOLVED] SanDisk X300 2.5" 512GB SATA III Internal Solid State Drive (SSD) BW-userx Linux - Hardware 4 04-11-2016 07:56 PM
LXer: Encryption busted on NIST-certified Kingston, SanDisk and Verbatim USB flash dr LXer Syndicated Linux News 0 01-08-2010 02:50 AM
Cant install USB Flash Drive Kingston DTI/1 hemanth_ap Linux - Hardware 3 05-03-2006 02:01 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware

All times are GMT -5. The time now is 01:46 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration