LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
  Search this Thread
Old 01-13-2019, 02:12 PM   #1
chinmoy
LQ Newbie
 
Registered: Jan 2019
Posts: 3

Rep: Reputation: Disabled
FAT32 R/W performance too low for big File transfer


i have mounted FAT32 formatted USB storage to etx4 FS and executed large file copy operation . Observed that FAT32 R/W performance to ext4 partition is very much slower than same ext4 partitions .

I've tried various combinations of with mount with ASYNC option and increased the FAT cluster but it didn't help though.

Debugged the v4.4.84 kernel and looks like following funcs. are taking more time .

Code:
fat_write_end(){ 
   ...
      block_write_end(){
        ... 
        grab_page_cache_write_begin(){
            ...

        }   //+96.923us

    }   //+168.089us

 }  //+189.778us
Could you please suggest how to improve the performance.

Help is greatly appreciated!
 
Old 01-13-2019, 02:23 PM   #2
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: MID-SOUTH USA
Distribution: Slackware 14.2 current / Linux Mint
Posts: 7,603

Rep: Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533
FAT32 hum 32 -- hum, how old is FAT32 and when was the last time they updated that method of file storage? just ball parking it here, but that is an old DOS format that was something to behold when it went from FAT to FAT32. I am sure there are write and read limitations on it as all formats. I'd think ext4 being a standard format in Linux it would be kept updated, even in its performance not only for how it stores data, but how it reads and writes it to a partition.

Whereas FAT32 is just something that is still in the mix because it is hard to just get rid of it, for whatever the reasons. You might even be a perfect example because you're still using FAT32 for whatever your reasons.

I am sure there are tech heads in here that can give you a much better techy details on this.

Last edited by BW-userx; 01-13-2019 at 02:26 PM.
 
Old 01-13-2019, 03:16 PM   #3
ehartman
Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 299

Rep: Reputation: 132Reputation: 132
Quote:
Originally Posted by BW-userx View Post
FAT32 hum 32 -- hum, how old is FAT32 and when was the last time they updated that method of file storage?
Fat32 was introduced in one of the update releases for Windows-95 and before Windows-98 came, about 1996, so more then 20 years ago.
Not too long ago a further extension/update was released by M$, called exFAT
 
Old 01-13-2019, 04:17 PM   #4
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: MID-SOUTH USA
Distribution: Slackware 14.2 current / Linux Mint
Posts: 7,603

Rep: Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533Reputation: 1533
Quote:
Originally Posted by ehartman View Post
Fat32 was introduced in one of the update releases for Windows-95 and before Windows-98 came, about 1996, so more then 20 years ago.
Not too long ago a further extension/update was released by M$, called exFAT
see I knew a techy would show up.


me just back yard logic on this, FAT32 could only read and write x-amont of data on one time, no matter if it is being written to some type of format that supersedes it. Basic Bottle necking is going to occur.

https://support.microsoft.com/en-us/...-fat-and-exfat

performance
https://www.flexense.com/fat32_exfat...omparison.html

then one needs to take into consideration through put mediums, data being transferred on the same hard drive as apposed to one hard drive to the other, along with the means used to connect the two.

same drive all that needs to be done it a quick rewrite in the file listings to where it is "now located" within the file system, between two drives actual moving of the data has to occur and be written to where it is located on the file listings in the other drive, and removed from the original drive, which takes more time.

Last edited by BW-userx; 01-13-2019 at 04:28 PM.
 
Old 01-14-2019, 01:52 AM   #5
vivekvikky
LQ Newbie
 
Registered: May 2018
Posts: 1

Rep: Reputation: Disabled
Even i have similar issue when i have executed iozone to benchmark different file systems some time back. I have run iozone test which took around 10 mins to complete whole set of iozone operations on ext4 . where took approximately ~2 hours to complete whole iozone set (read/write/ rewrite / reread/ fread/ fwrite ...) on FAT32 . Both operations were performed on same disk with different partitions.
 
Old 01-14-2019, 02:38 AM   #6
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 10,394
Blog Entries: 7

Rep: Reputation: 2567Reputation: 2567Reputation: 2567Reputation: 2567Reputation: 2567Reputation: 2567Reputation: 2567Reputation: 2567Reputation: 2567Reputation: 2567Reputation: 2567
maybe the bottleneck is the USB connection, not the filesystem?

chinmoy, how are you connecting the external storage? usb 2 or 3 or, god beware, 1.1?

also it stands to reason that even usb 3 transfer speeds will always be slower than internal copying on the same filesystem.
 
Old 01-14-2019, 07:09 AM   #7
chinmoy
LQ Newbie
 
Registered: Jan 2019
Posts: 3

Original Poster
Rep: Reputation: Disabled
Hi ondoho,
I have tried with USB 2.0 only .
 
Old 01-14-2019, 07:55 AM   #8
wpeckham
Senior Member
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, Fedora, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, Vsido, tinycore, Q4OS
Posts: 2,775

Rep: Reputation: 1148Reputation: 1148Reputation: 1148Reputation: 1148Reputation: 1148Reputation: 1148Reputation: 1148Reputation: 1148Reputation: 1148
Quote:
Originally Posted by chinmoy View Post
Hi ondoho,
I have tried with USB 2.0 only .
Well USB2 is much slower than the internal drive interface speeds. USB3 is slower than many, but is far closer and faster than a couple older storage interfaces (ie. much better if it is an option).

While FAT32 is slower than EXT4, many inexpensive USB devices do not react at the speed of the USB interface. I know this because I tend to use mostly the most inexpensive USB devices and run into it a LOT.

If you have a second identical USB device formatted to EXT4 you might want to perform some quick data copy tests to verify that the file system is actually involved in this case. The results would be specific to your exact hardware combination, but would help you eliminate or confirm one factor in this equation.

If you perform such a test, please share your results. I would really like to see what obtains.
 
Old 01-14-2019, 08:19 AM   #9
chinmoy
LQ Newbie
 
Registered: Jan 2019
Posts: 3

Original Poster
Rep: Reputation: Disabled
Yeah .I have already tired and formatted with EXT4 fs and seen the better performance than EXT4 .
Even i have tried with Approach 3 configuration of https://lonesysadmin.net/2013/12/22/...m-dirty_ratio/ link and found the strange behavior.
Executed two test cases :
Case 1 : copied a 256 MB file with bs of 64K
Case 2 : Ran iozone whole test suit

Observation :
Case 1 is giving better result but IOZONE performance (Case 2) is worst than default page cache configuration .
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: ArchBang Best Arch based distro for old or low-end hardware with high performance and low resource utilization LXer Syndicated Linux News 0 10-25-2016 01:30 PM
LXer: My Nerd Life: Too Loud, Too Funny, Too Smart, Too Fat LXer Syndicated Linux News 0 01-24-2014 06:21 AM
Ideas for a low cost, low power, low noise web server/torrent box alecjtaylor Linux - Hardware 13 01-10-2012 06:26 AM
Formating FAT32, but linux created FAT32 is not like win created FAT32?? Difference? boki15 Linux - Newbie 19 07-21-2010 05:41 AM
LXer: Big Blue's big green server transfer LXer Syndicated Linux News 0 08-02-2007 06:46 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel

All times are GMT -5. The time now is 05:53 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration