Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Yes sure I did. I did this operation multiple times and the files get corrupted as soon as they are copied into the ext4 drive without even removing the pendrive. If I use FAT32 it all works fine. However not all files get corrupted (mp.3 and mp.4 istantly corrupted, Pictures in jpg seem to not get corrupted, Office documents are fine and finally PDF sometimes get corrupted, sometimes not.)
This seems to be ignored by others, so let me highlight it here.
If your data is instantly corrupted on copy, without the USB device being unmounted, physically removed, or in any way interrupted, then it is something about the USB device itself or the driver for it that is causing the corruption.
It could be the specific combination of driver and device, but that would be somewhat unusual.
I would ask about the kind of corruption, but that may not matter much at this point. I am interested in how you detect the corruption and how it presents to you. It is information you might want to add to this thread for consideration.
What are you using to make the copy? I would like to see the command line, if any. Perhaps something that verifies the copy would work better. (Rsync comes to mind.) Was this USB device prepared with any kind of encryption in mind?
See if a different device shows the same symptoms. If not, then try different file systems on THAT device.
As much as I would love to dig into the root cause, your purposes will be better served by finding a workaround.
This seems to be ignored by others, so let me highlight it here.
I agree it's important, but I feel a few people have asked what they have done, and how they've detected these corruptions. Unfortunately there's no details about that.
I've had at least one brand of USB3 flash drive that simply would not work correctly on ext4. I assume it was the controller/caching as the next 20 brands worked correctly.
If the drive you have fails on ext4 then consider a new brand or use supported filesystem.
I've had at least one brand of USB3 flash drive that simply would not work correctly on ext4.
Could you post the brand/model of the USB flash drive? I searched for usb flash drives that do not work with ext4 and it seems to be an actual issue. I found this thread that describes a similiar issue as OP: https://superuser.com/questions/1317...format-as-ext4
This one seems to be a driver issue and in both cases the problem is seems to be limited to ext4 format.
@OP please post brand/model of your USB and the Distro you are using.
I am sure some posts at LQ have had this same issue that was resolved by changing the drive brand.
There was never any tests that I did to try to resolve the problem. Too much trouble but I am pretty sure it was limited to one big name brand. It could be that the motherboard or any part of the system could have caused it. All I know is I could not use it as a bootable linux distro on ext4. xfs worked and I think I tried btrfs that failed but it was a while back.
This seems to be ignored by others, so let me highlight it here.
If your data is instantly corrupted on copy, without the USB device being unmounted, physically removed, or in any way interrupted, then it is something about the USB device itself or the driver for it that is causing the corruption.
It could be the specific combination of driver and device, but that would be somewhat unusual.
I would ask about the kind of corruption, but that may not matter much at this point. I am interested in how you detect the corruption and how it presents to you. It is information you might want to add to this thread for consideration.
What are you using to make the copy? I would like to see the command line, if any. Perhaps something that verifies the copy would work better. (Rsync comes to mind.) Was this USB device prepared with any kind of encryption in mind?
See if a different device shows the same symptoms. If not, then try different file systems on THAT device.
As much as I would love to dig into the root cause, your purposes will be better served by finding a workaround.
Thank you to have highlighted the points. When I try to open the corrupted files it tells me that it is an unrecognized file format or it tells straight that the file is corrupted. The unrecognized file format thing is bu#$%^^&t because if I download the same file with the same format and open it in the hard drive it works fine. It is not a problem of a USB in particular it happens with all my pendrives if formatted to ext4. I also did backups with linux mint backup tool and when I remove the pendrive (unmounting it and oblviously formatted to ext4) and then instert it the files are just gone or maybe there are some left for unknown reasons. As I said before if I use FAT32 on the same devices it all works perfectly without any kind of issues or data loss.
I've had at least one brand of USB3 flash drive that simply would not work correctly on ext4. I assume it was the controller/caching as the next 20 brands worked correctly.
If the drive you have fails on ext4 then consider a new brand or use supported filesystem.
It doesn't fail in just one drive it fails on all my pendrives that are made by different companies they are not all the same.
I am sure some posts at LQ have had this same issue that was resolved by changing the drive brand.
There was never any tests that I did to try to resolve the problem. Too much trouble but I am pretty sure it was limited to one big name brand. It could be that the motherboard or any part of the system could have caused it. All I know is I could not use it as a bootable linux distro on ext4. xfs worked and I think I tried btrfs that failed but it was a while back.
The pendrives are not of the same brand they are made by different companies.
All I got is I've never run into it and used many a USB thumbstick exactly as I say.
Has anyone ever torn some apart? We did a few and amazingly they constituted of a micro SD card coupled with connections to attach it to the USB connector.
Sort of amazed us.
Kingston, Sansa, Memorex. Those were some brands I used. Whatever our parts buyer got when he ordered a bunch. We made some prototype products with USB sticks included to store the data, provide a mountable file system outside the product, and be available to be removed if the manufacturer so was inclined. Couple hundred prototypes, never had data problems, with the exception that I had to work a trick to ensure files would be available for offloading right after a field activity was completed.
TMI:
To fix I stored all data into a temp file, upon activity completion I forked a child to rename the temp to the final record, then exited the child. Upon successful termination signal from the child all file handles had to be resolved for that child, that made the file available and not corrupt.
Unless the drive itself is faulty it's most likely the fault of your choice of file systems. You should perform a chkdisk/scandisk from within windows to check for and correct errors.
The drive has likely been corrupted either by trying to copy a file over 4 GB in size, or by filenames that contain characters that are not permitted on a windows file system.
I can only say that one name brand did I ever have any issue. If multiple name brand usb flash drives fail on ext4 then I'd have to assume some other issue. Ext4 has been used by too many people on usb flash to claim it is a problem. (I'd think)
All filesystems get some updates every so often. Use latest version on modern distro if possible.
I guess rtmistler suggestion of tear down or other test could reveal same chips causing this issue.
OK, bottom line, if you have a device where EXT4 does not work, stop using EXT4 on that device and use what WORKS. We have not nailed down a cause, but it is likely to be the driver or software on your machine. (Else it would not affect all USB flash devices.)
FAT systems are to be avoided on all semi permanent storage due to the shortcomings of that format. Flash is not semi permanent storage, it is strictly temporary and untrustworthy. For flash use fat is just fine. Use the EXFAT to get expanded file and system sizes appropriate for modern capacity flash. You may want to run tests to see if other file systems suitable for USB FLASH use perform well and are suitable for your use on your devices on that system. I would be interested in running such tests myself, but I cannot replicate your issue.
Because flash is more prone to failure than other media, if you use it for backups make sure that you have multiple copies of your backups so that WHEN (not if) one fails you do not lose that data. This is good practice for ANY backup plan no matter what the backup media, but a bit more critical for flash media.
Since I have no use for Windows, I've always re-formated by ext2. I chose that because the fact that it doesn't maintain a journal means that it imposes less wear on the drive; I also mount with the parameters noatime,nodiratime.
USB flash drives will eventually fail. My rule when there's a problem is firstly to use fsck; if it happens again, reformat; third time, chuck it.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.