System crash - Shell prompt weirdness after attempting to repair - Suggestions?
 

We've had a bit of a problem here, and I'm trying to figure out what might have gone wrong.

The company I work for maintains a set of remote servers for one of our major clients. Yesterday, one of the remote sites (running SuSE 11.0) had their UPS backup system fail, and the server went down. After they tried to bring it back up, we could ping it, but not shell in via putty. From discussion on the phone, we determined that it came up into a shell prompt rather than the normal GUI prompt, and was apparently asking for the root password. We were able to talk someone there through login, and it came up to a (repair file system) prompt. This morning, we talked them through an fsck run (fsck -A -y), as found on a website reached by googling for the message they reported on the screen. ('type control d')

After that finished, we had them power the machine down and back up. It came back up to the original shell prompt, however, rather than the expected GUI prompt. It's now possible to ping the machine and shell in, and it looks like everything's in place (we've managed to snag backups for the work they did after the last daily backup they took), but the shell prompt in putty, which used to read 'DistrictX:~ #', now reads '-bash-3.2#'. And, again, the monitor physically attached to the machine in question is now showing a shell prompt login after startup, instead of going to the GUI login prompt. It does this every time the system is powered on and off.

My boss is worried that this might be a sign that something else in the system was corrupted, and that the fsck run may not have fixed it. I'm concerned as well; I've had systems boot to a shell prompt before, but I've always managed to get it back to the GUI prompt by rebooting. This one's a new one on me.

I'd thought that perhaps a runlevel setting somehow got shifted during the problem, but, well, below is output of a couple of commands:
If I understand correctly, runlevel 5 ought to mean the GUI should appear. As noted above, it doesn't.

Does anyone have any suggestions as to what might be the problem, or perhaps the names of other files I may need to check on in the system to determine what else, if anything, might be damaged?

Runlevel 5 contains an arbitrary set of links to various startup scripts, that's all. Your GUI scripts may be there, and by convention would, but it's best not to assume.

try the following command

# ps -ef | grep X

which will tell you what, if any GUI is running

Do you log in as root normally?
Do you know what services should be running?

What is the output of...

# cat /etc/fstab
# cat /etc/mtab
# cat /var/log/messages

What filesystem is it using ? If ext4 is it data=ordered / data=writeback ? See:
http://en.wikipedia.org/wiki/Ext4#Disadvantages

Thanks to you both for the advice. I'll try to look into it, but...

On the advice of a member of the area LUG mailing list, I tried rpm -V. It listed 99 files as missing, including what looked like most of X and several others, and many more files with numerous changes. On the basis of this, my boss decreed we would simply prep a new server to replace the old one and send the new one out. Given the way things work around here, after the new server arrives the old one will probably be sent back for a wipe/reclone. I'll try to work with these and find out what I can, but as things stand now, I'm afraid I'm going to have to label this one 'case closed: Terminal error'.

Unfortunately, that is sometimes the way of business. Much as we'd like to delve into this to see what the problem was, business continuity is the priority.