LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-18-2015, 06:10 AM   #1
Frizi
LQ Newbie
 
Registered: Jan 2015
Posts: 4

Rep: Reputation: Disabled
How to remount root partition on system booted from external usb drive


Hello,
I have a setup with arch linux installed on external ssd as normal (non-live) installation.
I had to do that after my main laptop was broken. I don't want to put my drive into the new hardware, so I run it as external one for until the main will be repaired.

It boots as usual, everything is working fine until the usb drive reconnects for some reason (may be the physical connection problem). It probably mounts itself as read-only, the system freaks out and dies slowly from I/O errors process by process. I cannot issue commands from shell, even reboot. I have to physically reset the machine.

Is there a way to change its behaviour to auto remount as rw after reconnecting the USB and just keep running as usual?

I think the fstab is the file that i should modify, but cannot find a way how to do it properly. Please correct me if I'm wrong. My current fstab:
Code:
# 
# /etc/fstab: static file system information
#
# <file system>	<dir> <type> <options> <dump> <pass>
# /dev/sdb2
UUID=145b3c51-6531-48f3-a79b-fb1ae238c7ee / ext4 rw,relatime,data=ordered 0 1

# /dev/sdb1
UUID=AFC4-6899 /boot vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 2

/swapfile none swap defaults 0 0
 
Old 01-18-2015, 06:50 AM   #2
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
I don't think so.

I believe your problem is that the swapfile is also on root - and when that filesystem remounts RO (which normally CAN be repaired), the swapfile is now locked.... parts in memory and parts on disk...

And the activity outstanding on the swapfile can't be completed thus locking up the system.

You might try repartitioning and putting the swapfile as a self contained partition. I believe the swap activation would then complete (now being independent of the root filesystem) which may allow you to recover the system (a "mount -o remount,rw /" should work).
 
Old 01-18-2015, 07:50 AM   #3
Frizi
LQ Newbie
 
Registered: Jan 2015
Posts: 4

Original Poster
Rep: Reputation: Disabled
I would like to avoid mocking with this disk partition table for time being. As i have 4 gigs of ram on this temporary hardware and fairly lightweight system, I can disable the swap entirely and just keep my eye on memory usage while working. But after issuing "sudo swapoff -a" and effectively disabling all swap, the problem still occurs. After unplugging and plugging back again the system drive, it just dies. I even got a kernel panic for the first time on this system.
 
Old 01-18-2015, 10:01 AM   #4
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
Your test was a good one.

It is possible that root cannot be disconnected once services have been started. Even without swap, a swap operation (actually a page read) is done to load binaries.

It is possible that the disk driver hasn't got sufficient error recovery implemented.

I believe an NFS root can disconnect, and reconnect successfully, which is why I mention "hasn't got sufficient error recovery". NFS has loads of such recovery, but I don't know about local disk drivers having that much.

Last edited by jpollard; 01-18-2015 at 10:04 AM.
 
Old 01-18-2015, 01:26 PM   #5
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
There is some mount options that might change what is going on. From the mount manpage:
Code:
   errors={continue|remount-ro|panic}
              Define the behaviour when  an  error  is  encountered.   (Either
              ignore  errors  and  just mark the filesystem erroneous and con‐
              tinue, or remount the filesystem read-only, or  panic  and  halt
              the  system.)   The default is set in the filesystem superblock,
              and can be changed using tune2fs(8).
Since you mentioned getting a panic when you disabled swap, it is possible that the filesystem is directing the system to not continue. You might TRY (and note, this may cause other failures) to see what happens with "continue".
 
Old 01-19-2015, 01:44 PM   #6
Frizi
LQ Newbie
 
Registered: Jan 2015
Posts: 4

Original Poster
Rep: Reputation: Disabled
After testing it further, I've seen that random crashes happen a lot less frequently with swap off, but this may be just my luck. System still crashes after manual unplug though. Also system cannot wake from sleep, but there may be other causes (as slightly different hardware than usual).

As I want to be very cautious about my external drive, is it "safe enough" to set the errors=continue option? From what I understand, romount-ro is a safety valve in case something could go horribly wrong. And I still should be able to remount my partition from terminal into rw. From unknown reasons though, the disk cannot be even accessed, so it is probably not remounted at all, which effectively eliminates the possibility of issuing "mount" command. My wild guess is that not enough kernel/tools are prepared in RAM to recover from error. Is there a way to prepare the system for this situation? Just like some "live" distros can run without the usb stick after boot.
 
Old 01-19-2015, 04:51 PM   #7
Frizi
LQ Newbie
 
Registered: Jan 2015
Posts: 4

Original Poster
Rep: Reputation: Disabled
I've managed to get into text mode in the early stage of crash, and this is what i saw
Click image for larger version

Name:	IMG_0899.jpg
Views:	17
Size:	144.0 KB
ID:	17384
 
Old 01-19-2015, 04:54 PM   #8
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
Quote:
Originally Posted by Frizi View Post
After testing it further, I've seen that random crashes happen a lot less frequently with swap off, but this may be just my luck. System still crashes after manual unplug though. Also system cannot wake from sleep, but there may be other causes (as slightly different hardware than usual).

As I want to be very cautious about my external drive, is it "safe enough" to set the errors=continue option? From what I understand, romount-ro is a safety valve in case something could go horribly wrong. And I still should be able to remount my partition from terminal into rw. From unknown reasons though, the disk cannot be even accessed, so it is probably not remounted at all, which effectively eliminates the possibility of issuing "mount" command. My wild guess is that not enough kernel/tools are prepared in RAM to recover from error. Is there a way to prepare the system for this situation? Just like some "live" distros can run without the usb stick after boot.
There is... but it might take a good bit of work.

The key is the initrd file used at boot time. It is actually a very small root filesystem that gets copied into a RAMFS. Now once the real root filesystem is mounted, the initrd init then uses the "pivot" system call to exchange the initrd based root for the real root - then discards the initrd root and continues loading applications.

It should be possible to set up things such that the initrd root remains, and then uses the "real" root as an application mount (the way /usr was intended - an extension of the root). It will use up system memory (several MB, maybe as much 50? might even be more), but it would remain resident even if the USB disconnects, then reconnection events wouldn't need handling via page faults from the disk...

It would be a custom initrd, with more of the existing init. I know that root for most systems can be trimmed to about 3 GB, but that is too much memory to loose for many laptops.

I would expect the system to freeze, but hopefully recover since the udev handling would all be on the initrd (and hence in memory). Updating /etc might be tricky as you would have set it in the initrd used for booting (or have the initrd copy updated files from the real root). I don't know if a bind mount of /usr would be sufficient. A number of distributions have been putting things in /usr that really ought to be in the root directory.

Things like /tmp and /var might work (though the log files might cause a problem).

Slackware used to do this... They would boot the kernel from one floppy, and use a second floppy for root, and that second 1MB had sufficient tools to even install the system as well as recover one.

If you were willing to use a CD (or even DVD) as a root boot you should be able to put the everything else on the USB. I realize that is a bit of cribbing, but with a DVD you can have up to 3GB for root, with a CD up to 700 MB, and without using up your RAM.
 
Old 01-19-2015, 05:29 PM   #9
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
Quote:
Originally Posted by Frizi View Post
I've managed to get into text mode in the early stage of crash, and this is what i saw
That would be the log file problem.

Hopefully, it doesn't cause the log to get corrupted.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How best to remount external drive with permissions to access it? Steve175 Linux - Newbie 7 01-04-2015 01:14 PM
Will an external usb hard drive remount on wake from sleep? mturob Linux - Newbie 14 07-05-2011 10:10 PM
Having to remount an external drive every day Zeno McDohl Linux - Hardware 10 09-01-2009 02:00 PM
External USB hard drive unmounts, can't remount or figure out what's causing it adanedhel728 Ubuntu 7 04-21-2008 02:26 AM
ext3 partition on external USB hard drive mounted, but only root access allowed leontini Linux - Hardware 4 02-17-2006 02:54 AM


All times are GMT -5. The time now is 10:31 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration