LinuxQuestions.org - remote server mounts /usr as read-only. How to fix without physical access?

- Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)

- - remote server mounts /usr as read-only. How to fix without physical access? (https://www.linuxquestions.org/questions/linux-software-2/remote-server-mounts-usr-as-read-only-how-to-fix-without-physical-access-763529/)

remote server mounts /usr as read-only. How to fix without physical access?

All,
I'll be brief:
* /usr mounts as read only.
* The filesystem is Reiserfs
* fsck.reiserfs --fix-fixable reports that there is a bad superblock on the filesystem

Quote:

Will check consistency of the filesystem on /usr
and will fix what can be fixed without --rebuild-tree
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
bread: Cannot read the block (2): (Is a directory).
reiserfs_open: bread failed reading block 2
bread: Cannot read the block (16): (Is a directory).
reiserfs_open: bread failed reading block 16

reiserfs_open: the reiserfs superblock cannot be found on /usr.
Failed to open the filesystem.

If the partition table has not been changed, and the partition is
valid and it really contains a reiserfs partition, then the
superblock is corrupted and you need to run this utility with
--rebuild-sb.

* Obviously I cannot unmount /usr without losing my connection.
* Single user mode is out because I do not have physical access to the machine.

As I see it, to fix this problem, I have to do one of three things.
a. wait until I have access to the server.
b. move sshd so to a different partition (and reboot)
c. ??? (what does the wisdom of the crowd say?)

I am using openSuSE 11.0 Kernel 2.6.25.20-0.5

Thanks for reading this far!

Quote:

Originally Posted by thatto (Post 3727710)

* Obviously I cannot unmount /usr without losing my connection.

Well, you wouldn't lose your connection necessarily, it would just become a fairly boring connection. ;-) (sshd is memory resident if you're connected, so it doesn't matter where /usr goes.*) You'll still have /bin and /sbin, of course. Which is okay:

/sbin/umount, /sbin/mount, /sbin/fsck will all be available and should have all you need to fsck.

Code:

# First, kill everything using /usr:

lsof | grep /usr

kill <pids>

# Unmount /usr:

umount /usr

# fsck the problem:

fsck.resierfs <usr partition>

# remount, and enjoy us(e)r binaries again:

mount -a

# test a usr bin:

/usr/bin/awk -Wversion

* As a related example of this: If you hit rm -rf /* while on an ssh connection, your connection will continue, but as soon as you lose it, you'll need to reinstall the operating system via physical access.