LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 08-03-2009, 03:46 PM   #1
JohnE1
Member
 
Registered: Jun 2007
Location: Houston, TX
Distribution: Fedora, CentOS, Ubuntu
Posts: 66

Rep: Reputation: 16
Question fsck (e2fsck) questions


If you're a system admin responsible for Linux servers in racks, locked away with no keyboard or mouse directly connected, then I'm very interested in your replies to the following questions:


I need to repair a file system on a PC which I can see across the LAN, but of course, only while that PCs file system is running. Herein, lies the problem. Both PCs have Linux, but neither hardware has the serial console BOOT PROM (or serial console capability, to the best of my knowledge).

The file system checking utilities warn of the risk of EXTREME damage to mounted file systems. So, if the file system is not mounted, then the O/S is not running and no network devices are working, making the machine unreachable via Ethernet.

To my knowledge, that leaves 2 choices:

1. Boot from a CD/DVD locally plugged attached to the machine who's file system I want to fix, because a Live Linux can identify the attached drives and work on non-mounted file systems.

2. Connect via the serial console, if the hardware has the BOOT PROM support for this. While I do have serial console experience--from a Windows workstation connected to a Sun SPARC server--I do not know how to run or use fsck or e2fsck. Is this even an option between 2 Linux PCs?

So, how do all you sys admins repair file systems on servers you can't walk up to and use directly?

Suggestions, please?

Thanks in advance for your replies, and thanks for reading my long question!!
 
Old 08-03-2009, 05:21 PM   #2
JohnE1
Member
 
Registered: Jun 2007
Location: Houston, TX
Distribution: Fedora, CentOS, Ubuntu
Posts: 66

Original Poster
Rep: Reputation: 16
I found part of my answer, a way to repair the file system locally without having to boot from a Live Linux CD or DVD.

http://www.cyberciti.biz/tips/repair...le-system.html

Still like to know how to repair a file system remotely; via ssh would be great!
 
Old 08-03-2009, 06:39 PM   #3
hasienda
Member
 
Registered: May 2009
Location: Saxony, Germany
Distribution: Debian/GNU Linux
Posts: 36

Rep: Reputation: 18
Exclamation as often, it depends

Quote:
Originally Posted by JohnE1 View Post
I found part of my answer, a way to repair the file system locally without having to boot from a Live Linux CD or DVD.

http://www.cyberciti.biz/tips/repair...le-system.html

Still like to know how to repair a file system remotely; via ssh would be great!
So you already know, that it depends on the partition schema of the system. If the file system in question does _not_ hold the root partition, then it may be easy to unmount, check, remount without having to switch to single-user-mode, which may break networking as well (actually it _will_ do so i.e. with Debian/GNU Linux, but your distribution may handle this differently).

However I suspect you request repair of the file system on the root partition. That is the harder part and commonly done by physically accessing the system and booting a rescue (live) system from CD/DVD or USB mass storage.

But there _is_ an option to boot your system with it's native kernel and some custom kernel options (added to GRUBs menu.lst or LILOs lilo.conf). Have a look at
http://www.vanemery.com/Linux/Serial...l-console.html on serial consoles with Linux in general,
http://howtoforge.com/setting_up_a_serial_console about consoles at boot time involving boot loader configuration and
http://linuxjunkies.org/html/Remote-...ole-HOWTO.html with additional note on Null modem cables and
http://www.linuxjournal.com/article/7206 explaining use of a system as dedicated console concentrator for a number of other systems.

You'll see, there is no special hardware requirement, and I'd be surprised, if your systems lake serial console capability. Of course you'll need to have a serial connection installed between the system in question and another running system. But a serial port is (still) quit common, even if things like floppy drives and PS/2 ports are fading away. Apart from console related stuff you'll want to add another kernel option "root=/sbin/init" to stop the normal boot-up including mounting "/" read-write. USB-to-Serial converter will _not_ work, because USB may never be initialized, if not included in a special initial RAM-disc (often by making it kind of a mount dependency for root-fs or the like).

Well, while I do have these headless systems and used console on running systems, I must confess that I never tested the console for recovery by now. Hope, you still get some more points to start from.
 
Old 08-14-2009, 02:03 PM   #4
JohnE1
Member
 
Registered: Jun 2007
Location: Houston, TX
Distribution: Fedora, CentOS, Ubuntu
Posts: 66

Original Poster
Rep: Reputation: 16
Hasienda,

What a great post! Thanks for taking the time to share all that useful info with me (and everyone else on LQ)!!

I ended up switching into single user mode and running fsck.ext2 on the 2 ext2 partitions. After repairing the ext2 partitions, the swap partition tested good and the machine seems to be working fine now.

As for the hardware part of serial consoles, I've always used Sun's BOOT PROM, so I really have no experience with serial console connections on machines without those BOOT PROMS. I guess I was assuming they are required, since they contain the code to operate the server without the OS running. From what you said, sounds like I need to learn a lot more about GRUB, and maybe LILO, too.

Thank you very much for the links and comments and a great reply!!!
 
Old 08-15-2009, 01:27 AM   #5
settntrenz
Member
 
Registered: Aug 2009
Location: Orlando, Florida
Distribution: RHEL, Ubuntu
Posts: 49

Rep: Reputation: 19
If you've got the bankroll, a KVM-over-IP will allow you to get a console session on the box.

This product is an example for a single server solution. You may want to look at a more robust solution if you want to connect to multiple servers in rack.

http://www.lantronix.com/it-manageme...nx-spider.html

*I have no vested interest in that product, it is just one example of many available on the market.
 
  


Reply

Tags
fsck


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
fsck and e2fsck ygloo Linux - General 8 08-31-2006 02:19 PM
e2fsck and fsck.ext3 kpachopoulos Debian 4 02-09-2006 11:29 AM
Running fsck (or e2fsck) on root file system... xbill311x Linux - Newbie 2 01-02-2006 10:54 PM
Ambiguous error messages from fsck, but not from e2fsck jiffydude Linux - Software 2 09-12-2003 06:02 PM
fsck, e2fsck, mke2fs ?'s - RH9 blackx Linux - Newbie 6 05-23-2003 02:02 PM


All times are GMT -5. The time now is 06:25 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration