LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices


Reply
  Search this Thread
Old 06-30-2020, 06:10 PM   #1
refaehei
LQ Newbie
 
Registered: Jun 2020
Posts: 6

Rep: Reputation: Disabled
Mounting a USB disk: FAT32 ok, ext4 not...


Hello,

Apologies if the answer is somewhere in the site, but I couldn't find it. I bought a 2TB Toshiba USB drive for my laptop's backups, and I am having some problems mounting it (I run Debian 10 with kernel 4.19.0-9-amd64).

First, I was unhappy with the writing speed, compared to other disks I have. Especially because after Ctrl-C a rsync local copy, it took several minutes for the drive to finish the copy and allow me to eject. I have an older drive that does not do that at all. I tried changing two system values, something called vm...bytes... as per some website, but no improvement.

Next, I decided to format it, and that's when things got weird for me. I went for ext4 (I will only use it in modern Linux computers). But upon plugging again, although it was automounted and opened in Dolphin, no writes were allowed, only as root.

I tried a couple of solutions for automount. Since it's external, based on things I read I left fstab for last, and I started by following https://unix.stackexchange.com/a/560380 to set up unit files (whatever they are). To make it short, it didn't work. In any case I've reformatted the drive a few times so that UUID is not valid any more, those files I set up won't interfere.

I also saw a suggestion on udev rules (?) but I couldn't find anything clear enough for me.

I plugged an older drive that I have, and since it works as it should, I checked it to see that it is ext3. So I tried formatting the new one (with GParted) but it took so very long (after a few hours, less than halfway) that I stopped it. After that I put it as FAT32 and it did work like a charm. Then back to ext4, back to only root.

Finally I tried fstab via the current UUID. For the mount options I tried "defaults" only, later "user" only, and still no writing for non-root. After that, FAT32 once more (all good), ext4 again (not good). On top of that, maybe in relation to having GParted open sometimes, for unmounting I am asked for the root password sometimes.

A lot of this is quite new for me (all I did was read and follow suggestions) so I am really stumped. If it turns out that FAT32 is not going to be significantly worse in capabilities or speed, I may well go with it. But I would really appreciate some pointers to the solution of the mystery, because I am very curious about it.

Thanks a lot!
 
Old 06-30-2020, 06:28 PM   #2
sp331yi
Member
 
Registered: Mar 2020
Location: NM
Distribution: antiX 19.2 | Slacko pup | Miyo
Posts: 104

Rep: Reputation: Disabled
Some will tell you to fdisk the USB stick to rid it of any proprietary bs the manufacturer may have added (SanDisk does this, IME) but
Check out these two links, for starters:

You'll probably want to partition 2TB, won't you, OP?

https://www.howtogeek.com/106873/how...ions-on-linux/

https://tecadmin.net/format-usb-in-linux/
 
1 members found this post helpful.
Old 06-30-2020, 07:14 PM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 18,991

Rep: Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283
The OP seems well versed in the use of gparted to partition/format.

I don't run Debian, but that's a pretty old kernel. If I had to guess I'd reckon it's a USB3 drive, or maybe connector. Try re-plugging the drive in and then run this from a terminal - post the output; copy-and-paste should work.
Code:
dmesg | tail -n 20
BTW, comments such as the following are pointless - we can't help if we don't know (exactly) what you did, and any messages if applicable. Save and post it all.
Quote:
I tried changing two system values, something called vm...bytes... as per some website, but no improvement

Last edited by syg00; 06-30-2020 at 07:18 PM. Reason: s /plugg/re-plugg/
 
Old 07-01-2020, 04:57 AM   #4
fatmac
Senior Member
 
Registered: Sep 2011
Location: Upper Hale, Surrey/Hants Border, UK
Posts: 3,679

Rep: Reputation: Disabled
I don't 'automount' any external drives, & have normal write times when I mount them manually.

If you don't want to have to use sudo or root to write files to it, you must own the mount point & have read/write permission on the directory.
 
Old 07-01-2020, 09:33 AM   #5
refaehei
LQ Newbie
 
Registered: Jun 2020
Posts: 6

Original Poster
Rep: Reputation: Disabled
Thanks for all your answers. Here is the output of dmesg.
Code:
[40102.303512]  sda: sda1
[40102.306194] sd 2:0:0:0: [sda] Attached SCSI disk
[40149.265410] sd 2:0:0:0: [sda] Synchronizing SCSI cache
[40149.454332] usb 2-2: USB disconnect, device number 19
[40168.521652] usb 2-2: new SuperSpeed Gen 1 USB device number 20 using xhci_hcd
[40168.543610] usb 2-2: New USB device found, idVendor=0480, idProduct=0900, bcdDevice= 3.15
[40168.543614] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[40168.543615] usb 2-2: Product: External USB 3.0
[40168.543616] usb 2-2: Manufacturer: TOSHIBA
[40168.543618] usb 2-2: SerialNumber: 20191211006036F
[40168.544851] usb-storage 2-2:1.0: USB Mass Storage device detected
[40168.545148] scsi host2: usb-storage 2-2:1.0
[40171.685242] scsi 2:0:0:0: Direct-Access     TOSHIBA  External USB 3.0 5438 PQ: 0 ANSI: 6
[40171.686371] sd 2:0:0:0: Attached scsi generic sg0 type 0
[40171.688006] sd 2:0:0:0: [sda] 3907029164 512-byte logical blocks: (2.00 TB/1.82 TiB)
[40171.688389] sd 2:0:0:0: [sda] Write Protect is off
[40171.688392] sd 2:0:0:0: [sda] Mode Sense: 23 00 00 00
[40171.688747] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[40171.710972]  sda: sda1
[40171.714795] sd 2:0:0:0: [sda] Attached SCSI disk
I found what I did with the dirty byte stuff I mentioned. I created a .conf file in /etc/sysctl.d with this:
Code:
vm.dirty_background_bytes = 16777216
vm.dirty_bytes = 50331648
(I rebooted a couple of times, so it must be running like that now)

BTW the kernel is the version I got yesterday (running Debian 10, did an upgrade yesterday to see if it helped)

Quote:
If you don't want to have to use sudo or root to write files to it, you must own the mount point & have read/write permission on the directory.
Ok, I am willing to mount this particular disk manually. But I haven't been able to... I created inside /media/username a folder TOSHIBA (and chown-ed it, permissions are drwxr-xr-x). I formatted the drive to ext4 again. When I plug it I am asked if I want to open it with the file browser (Dolphin); the system creates a new folder TOSHIBA1 (doh!). I then did in the terminal:
Code:
su
umount /dev/sda1
mount /dev/sda1 /media/sevillad/TOSHIBA/
I received no answer to the last command, and "mount" did NOT show me the mounted disk in the list. In Dolphin, the icon showed as not mounted, and when I clicked on it, it was mounted, root-only, in TOSHIBA1 as before. It's like what is said in the comment to https://askubuntu.com/a/769559 .

Thanks again
 
Old 07-01-2020, 11:04 AM   #6
DavidMcCann
LQ Veteran
 
Registered: Jul 2006
Location: London
Distribution: PCLinuxOS, Xubuntu
Posts: 5,579

Rep: Reputation: 1994Reputation: 1994Reputation: 1994Reputation: 1994Reputation: 1994Reputation: 1994Reputation: 1994Reputation: 1994Reputation: 1994Reputation: 1994Reputation: 1994
I must admit your account left me a bit lost with all that systemd stuff, so I'll just tell you what I do and you can adapt it if you feel it's applicable.

My backup USBs are formatted to ext4 and they automount to /media/usb. Naturally I created that directory and made myself the owner. Its use is set by an fstab entry
Code:
/dev/sdc1 /media/usb auto noauto,noatime,users 0 0
I use "auto" rather than "ext4" just in case I have to mount any other USB stick. In the parameters, "noauto" ensures that booting is not delayed by the computer trying to find a device that's not plugged in and "users" ensures that I don't need to be root to mount.
 
1 members found this post helpful.
Old 07-01-2020, 01:50 PM   #7
refaehei
LQ Newbie
 
Registered: Jun 2020
Posts: 6

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by DavidMcCann View Post
My backup USBs are formatted to ext4 and they automount to /media/usb. Naturally I created that directory and made myself the owner. Its use is set by an fstab entry
Code:
/dev/sdc1 /media/usb auto noauto,noatime,users 0 0
Thanks for the info. Unfortunately it didn't work. I got the UUID and put in fstab:
Code:
UUID=8723153a-c7fe-4415-a568-56d4ac67273a  /media/username/TOSHIBA auto noauto,users,exec 0 0
and when the system notifies me of it and I say "Open with File Manager" I get an error popup "Malformed URL". As of now, the TOSHIBA folder exists and is mine (name and group). The "media" folder belongs to root, but that is not an impediment for other things to be mounted under it routinely.

When I plug my old USB disk everything is as desired (no fstab entry or anything else required, from the first time just "Open with File Manager" in the menu that pops up). The folder (named after the UUID) is created automatically, owned by me. I run "mount" after mounted and got:
Code:
/dev/sda1 on /media/username/c3d45a39-8f32-4a12-9079-a067540bccbe type ext3 (rw,nosuid,nodev,relatime,uhelper=udisks2)
I can't begin to understand how is all this not happening for the new disk without any fiddling on my side, especially when it does work if formatted as FAT32.

Another weird thing that I noticed: when formatted as ext4 a "lost+found" folder is created, and GParted reports that 30GB are already in use (right after formatting, GParted says 1.9GB). That's a lot!
 
Old 07-02-2020, 03:58 AM   #8
refaehei
LQ Newbie
 
Registered: Jun 2020
Posts: 6

Original Poster
Rep: Reputation: Disabled
I did one more test: another Linux computer (Mint, I don't remember the version). I tried the old and the new disk without any previous setup and got the same as in my computer: the old disk (ext3) works as expected, and the new one (ext4) is not writable for the regular user. The output of "mount" shows the same mounting options in both cases.
 
Old 07-02-2020, 05:40 AM   #9
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 18,991

Rep: Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283Reputation: 3283
This ...
Quote:
Originally Posted by refaehei View Post
Another weird thing that I noticed: when formatted as ext4 a "lost+found" folder is created, and GParted reports that 30GB are already in use (right after formatting, GParted says 1.9GB). That's a lot!
... and ...
Quote:
I tried the old and the new disk without any previous setup and got the same as in my computer: the old disk (ext3) works as expected, and the new one (ext4) is not writable for the regular user.
says to me the disk is broken, or at least isn't performing as Linux expects - shingle disks have caused problems when first released for example. That dmesg suggests it may be disconnecting and reconnecting constantly, but the output is too short - can you try "-n 40" and post it again ?.

The lost+found is the filesystem trying to fix itself after an error (say the device disappearing before re-appearing again). If it keeps growing, it's never a good sign.

None of the above explains why FAT works tho' - maybe not much checking going on as it's pretty basic.

I'd also delete the sysctl file you created - those numbers are way out of order unless you have a seriously large amount of RAM.
 
Old 07-02-2020, 06:25 AM   #10
refaehei
LQ Newbie
 
Registered: Jun 2020
Posts: 6

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by syg00 View Post
This ...... and ... says to me the disk is broken, or at least isn't performing as Linux expects - shingle disks have caused problems when first released for example. That dmesg suggests it may be disconnecting and reconnecting constantly, but the output is too short - can you try "-n 40" and post it again ?.
Here is all I got, with comments.
Code:
plugging (the first line is the unplug of a headset to free the USB port)
[96830.692042] usb 1-2: USB disconnect, device number 12
[96846.876044] usb 2-2: new SuperSpeed Gen 1 USB device number 29 using xhci_hcd
[96846.898871] usb 2-2: New USB device found, idVendor=0480, idProduct=0900, bcdDevice= 3.15
[96846.898874] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[96846.898875] usb 2-2: Product: External USB 3.0
[96846.898877] usb 2-2: Manufacturer: TOSHIBA
[96846.898878] usb 2-2: SerialNumber: 20191211006036F
[96846.900229] usb-storage 2-2:1.0: USB Mass Storage device detected
[96846.900415] scsi host2: usb-storage 2-2:1.0
[96850.001680] scsi 2:0:0:0: Direct-Access     TOSHIBA  External USB 3.0 5438 PQ: 0 ANSI: 6
[96850.002310] sd 2:0:0:0: Attached scsi generic sg0 type 0
[96850.003981] sd 2:0:0:0: [sda] 3907029164 512-byte logical blocks: (2.00 TB/1.82 TiB)
[96850.004503] sd 2:0:0:0: [sda] Write Protect is off
[96850.004508] sd 2:0:0:0: [sda] Mode Sense: 23 00 00 00
[96850.004932] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[96850.027589]  sda: sda1
[96850.030672] sd 2:0:0:0: [sda] Attached SCSI disk

after GUI mount attempt (bad url msg due to fstab problem, but mounted anyway)
[96846.876044] usb 2-2: new SuperSpeed Gen 1 USB device number 29 using xhci_hcd
[96846.898871] usb 2-2: New USB device found, idVendor=0480, idProduct=0900, bcdDevice= 3.15
[96846.898874] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[96846.898875] usb 2-2: Product: External USB 3.0
[96846.898877] usb 2-2: Manufacturer: TOSHIBA
[96846.898878] usb 2-2: SerialNumber: 20191211006036F
[96846.900229] usb-storage 2-2:1.0: USB Mass Storage device detected
[96846.900415] scsi host2: usb-storage 2-2:1.0
[96850.001680] scsi 2:0:0:0: Direct-Access     TOSHIBA  External USB 3.0 5438 PQ: 0 ANSI: 6
[96850.002310] sd 2:0:0:0: Attached scsi generic sg0 type 0
[96850.003981] sd 2:0:0:0: [sda] 3907029164 512-byte logical blocks: (2.00 TB/1.82 TiB)
[96850.004503] sd 2:0:0:0: [sda] Write Protect is off
[96850.004508] sd 2:0:0:0: [sda] Mode Sense: 23 00 00 00
[96850.004932] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[96850.027589]  sda: sda1
[96850.030672] sd 2:0:0:0: [sda] Attached SCSI disk
[96931.422053] EXT4-fs (sda1): recovery complete
[96931.796878] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[96967.371249] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[96968.704613] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[96974.683885] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[96976.123530] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[96981.159492] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[96982.397727] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[97102.501649] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)

after umount and unplug
[97372.471878] usb 2-2: USB disconnect, device number 29
[97372.473350] sd 2:0:0:0: [sda] Synchronizing SCSI cache
[97372.473386] sd 2:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK

another plug after removing the offending fstab line
[97478.093110] usb 2-2: new SuperSpeed Gen 1 USB device number 30 using xhci_hcd
[97478.115814] usb 2-2: New USB device found, idVendor=0480, idProduct=0900, bcdDevice= 3.15
[97478.115819] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[97478.115823] usb 2-2: Product: External USB 3.0
[97478.115825] usb 2-2: Manufacturer: TOSHIBA
[97478.115828] usb 2-2: SerialNumber: 20191211006036F
[97478.117523] usb-storage 2-2:1.0: USB Mass Storage device detected
[97478.117965] scsi host2: usb-storage 2-2:1.0
[97481.230338] scsi 2:0:0:0: Direct-Access     TOSHIBA  External USB 3.0 5438 PQ: 0 ANSI: 6
[97481.231089] sd 2:0:0:0: Attached scsi generic sg0 type 0
[97481.232732] sd 2:0:0:0: [sda] 3907029164 512-byte logical blocks: (2.00 TB/1.82 TiB)
[97481.233172] sd 2:0:0:0: [sda] Write Protect is off
[97481.233178] sd 2:0:0:0: [sda] Mode Sense: 23 00 00 00
[97481.233532] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[97481.534025]  sda: sda1
[97481.537141] sd 2:0:0:0: [sda] Attached SCSI disk

after successful GUI mount
[97510.631121] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)

after umount and unplug
[97598.207714] sd 2:0:0:0: [sda] Synchronizing SCSI cache
[97598.404938] usb 2-2: USB disconnect, device number 30
Quote:
The lost+found is the filesystem trying to fix itself after an error (say the device disappearing before re-appearing again). If it keeps growing, it's never a good sign.
I checked more carefully and the folder is just a few kb. But GParted is saying that 30GB are in use. It seems too much for whatever partition information there is, if that's what the program is reporting about.

Quote:
I'd also delete the sysctl file you created - those numbers are way out of order unless you have a seriously large amount of RAM.
Done.

I will format it as ext3 (it may take many hours) and see, since my old trusty disk gives 0 problems.

Thanks a lot for the company!
 
Old 07-02-2020, 08:45 AM   #11
michaelk
Moderator
 
Registered: Aug 2002
Posts: 20,228

Rep: Reputation: 3460Reputation: 3460Reputation: 3460Reputation: 3460Reputation: 3460Reputation: 3460Reputation: 3460Reputation: 3460Reputation: 3460Reputation: 3460Reputation: 3460
The lost+found directory is created when the file system is created and is where fsck puts "lost" stuff it finds during a file system check.

Drive manufactures typically list their drive sizes using base 10 versus 16 (gigabytes versus gibibytes) so your 2 TB disk is actually only 1.86 TB.

That 1.86 TB is just an empty container. A filesystem contains lots of metadata i.e all the stuff necessary to be able to find your files etc which is about 2% and probably the 30GB you are referring to in post #8. This is all normal.

Once the ext4 partition is mounted you can easily change its permissions using chmod to be accessible by your regular user. I suspect you changed the permissions on the old ext3 drive.

Since a vfat permissions are not really linux compatible the desktop automatically mounts it as R/W for that user.
 
Old 07-02-2020, 10:38 AM   #12
fatmac
Senior Member
 
Registered: Sep 2011
Location: Upper Hale, Surrey/Hants Border, UK
Posts: 3,679

Rep: Reputation: Disabled
Linux sets aside 5% of disk space for recovery purposes, but I have never needed it, so I use tune2fs to reduce my / down to just 1%, & my /home to 0%, & that will give you back a fair bit of usable disk space.

Usage:
Code:
sudo tune2fs -m 1 /dev/sda1
 
1 members found this post helpful.
Old 07-02-2020, 11:56 AM   #13
refaehei
LQ Newbie
 
Registered: Jun 2020
Posts: 6

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by michaelk View Post
That 1.86 TB is just an empty container. A filesystem contains lots of metadata i.e all the stuff necessary to be able to find your files etc which is about 2% and probably the 30GB you are referring to in post #8. This is all normal.
That's a great relief. I guess I'm not used to big drives and the big numbers they produce.

Quote:
Once the ext4 partition is mounted you can easily change its permissions using chmod to be accessible by your regular user. I suspect you changed the permissions on the old ext3 drive.
I have changed (chown) the folder, and it works! It may have been the case with the ext3 drive, altough I document these things and I didn't find anything when searched the other day. But it makes sense.

Well, back to backups... I plugged and unplugged a few times to make sure it works fine. Then I started my backup, which was as slow as when I did it for the first time (no apparent improvement from fat32 to ext4). And after a while it gave an I/O error. Since some part of KDE crashed with it, I was no longer able to mount it from the device notifier, so I did it from the terminal; and then the backup (local rsync) became FAST! A 600+ MB file that took more than 3 minutes to copy originally was done in 11 seconds. But after a while, at around 37k files copied, it is much slower (a 21 MB file in more than two minutes) and even things like "ls" and "more TEXTFILE" take a few seconds to be resolved. Still as unusable for weekly backups as when I opened the drive...

Any ideas on the speed issue? So far all this weirdness was explainable without blaming the hardware, and I expect it to be the case as well, but I have no idea where to start, and I may not have gained anything by moving out of fat32. Thanks a lot!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Can't create disk image on FAT32 formatted USB using Mac Disk Utility name_Christian Linux - Newbie 9 05-24-2018 08:06 AM
How to change file system from ext4 journeying to ext4 writeback ? 5883 Linux - Newbie 6 03-10-2014 08:04 AM
modprobe: Module ext4 not found. no dependencies for kernel module 'ext4' found TheIndependentAquarius Linux - Newbie 6 01-25-2012 05:07 AM
Is it safe to format USB flash to ext4 or ext4? joham34 Linux - Newbie 2 01-08-2011 11:58 AM
Formating FAT32, but linux created FAT32 is not like win created FAT32?? Difference? boki15 Linux - Newbie 19 07-21-2010 04:41 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware

All times are GMT -5. The time now is 11:05 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration