Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I have been having this problem for quite some time.
Scenario. . .
Ubuntu 21. Has happened to any version of Linux I use.
Thumbdrive = 512Gig (or 128G. 64gig)
ExFat installed on Linux with no problems.
At first the Drive seems ok when I copy. But if I do a
huge copy of files, (let's say 10 gig or so). It falsely
tells me the files are copied, space is "possibly" taken, but no files
are copied.
I have noticed that the Directories I created DO Exist, but the
files I copied into them (anywhere between 100meg to 500meg) do
not exist at all. I have wasted hours of my time for nothing.
I cannot dismount and remount in hopes to recover the files,it doesn't work.
I go to a windows computer and it says thumbdrive corrupt. So I fix it, and
the files do not exist.
This seems to only happen in the ExFat formatted thumb drives.
Is there a fix for this problem? IT is such a problem, that I can almost
guarantee a corrupt thumbdrive if I use one in Exfat.
need to know how did you do that exactly and how did you check exactly. What I think is:
you execute the copy to usb, but did not wait for completion. You need to [safely] eject and/or umount your usb stick, cannot be removed without that (that's why you get this error).
Sometimes there is a false message that a copy has completed when the files are only in the cache, not yet persisted to disk.
If you're in a terminal, the sync command can be used to ensure the cache is written - run it in a terminal and wait for it to come back (for 10GB of data it may take a while).
This is what a GUI "safely remove" (or similarly named option) should also do, and why you should always use such options before removing a storage device (whether on Windows or Ubuntu or any other OS).
Ok, so you've now revealed a whole bunch of new information that should have been in the first post!
There's still important details missing though...
* "Ubuntu 21" could refer to either 21.04 or 21.10 - which one?
* The kernel version you're running is relevant to what sort of exFAT support you have - what version of Linux is it?
* Does "ExFat installed on Linux" mean you manually installed the old non-kernel drivers that your OS probably doesn't need?
* You've not said if the devices/ports are USB 2 or USB 3
* Did they come pre-formatted as exFAT or did you format them that way (if so, how?)
* What precisely are those "5 different brands"?
You can run "inxi -SMxxz --usb" to provide some of that - post the results inside "[code]..[/code]" tags.
It should be a technical problem but it is not. It is a political problem.
Both the politics and the data loss are in the nature of (Ex)FAT. It was designed for 5.25" floppies and is most unsuitable for anything of modern size as the design basically guarantees that it will lose your data. Try a different format if you can. The catch is if you can, since M$ has multiple software patents encumbering ExFAT in the US it has financial incentive to leverage its position to exclude all other formats, no matter how much more appropriate they may be for storage media.
Maybe that information is not much help beyond providing context for the problem.
One thing that I can be sure of. . .to a degree. . .
If I copy just one directory of files at a time, the success rate is extremely high.
That I remember, I don't think it has failed.
If I copy multiple directories at the same time with many files,
the failure rate is extremely high. I would say almost 100 percent.
Both the politics and the data loss are in the nature of (Ex)FAT. It was designed for 5.25" floppies and is most unsuitable for anything of modern size as the design basically guarantees that it will lose your data.
?????
exFAT was released in 2006 AFAIK designed for embedded systems like cameras with memory cards to overcome the 4GB FAT32 limitation.
I agree it might be due to a conflict if both fuse and the kernel module are loaded at the same time.
Kind of odd that exfat fails on a major distro like that. I'd be more inclined to believe that the usb controller timing is at fault. Are you using an add on usb board?
Otherwise if you can try this on a different computer it might prove the usb drive or computer at fault.
My guess is that it isn't exfat filesystem. That doesn't mean that all the parts going to and from it are good.
One thing that I can be sure of. . .to a degree. . .
If I copy just one directory of files at a time, the success rate is extremely high.
That I remember, I don't think it has failed.
If I copy multiple directories at the same time with many files,
the failure rate is extremely high. I would say almost 100 percent.
you can still try to use sync, fsck and similar tools to check that device. And also you can try another filesystem if that works at all on that usb stick.
Kind of odd that exfat fails on a major distro like that. I'd be more inclined to believe that the usb controller timing is at fault. Are you using an add on usb board?
Otherwise if you can try this on a different computer it might prove the usb drive or computer at fault.
My guess is that it isn't exfat filesystem. That doesn't mean that all the parts going to and from it are good.
Every distrobution I have ever ran that accepts large thumbdrives has had this issue. . .on any computer.
I can read from the thumb drive without anyproblem with the ExFat drivers installed. but when writing to
the thumbdrive. . .there is no guarantee. . .
I am not sure, but I have noticed that if I have multiple files writing to the
thumb drive at the same time, it seems to fail on me.
It shows everything is copying, but when all said and done, everything I did is completely
lost.
I stick the drive into a windows unit, and everytime it says "Thnumdrive has errors on it, do you want to
scan and fix".
Then i get Found.00 Found.01. etc. and one of them is usually close to the amount
of data I copied. (Just guessing here on the amount).
Obviously the latter. Switch to NTFS if you want compatibility or a linux filesystem if you do not. I've only needed exFAT support a few times and never had a problem.
Is your aim to solve this issue, or do you only want to complain?
I would love to solve the problem. . .
I can't find answers on the internet.
I have quite a few larger USB drives and External Hard drives
that I am afraid to even plug in.
It is frustrating when you plug these things in, and lose your data
while trying to write to it.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.