Out of no where I'm getting these errors:
Some numbers there variate, but that error continues over and over. The entire server is down. What's wrong, and how do I fix this?

I have made no changes to the server for over a month.

This is very bad. You also likely seeing in your system logs some other complaints from the kernel about communication failures with the disk itself.

There's a pretty good chance that disk is dying. It's probably time to break out smartctl and try the drive's self-tests, or make a backup of everything (actually, make a backup anyway) and grab Ultimate Boot CD to use the manufacturer's testing utilities to try to analyze and/or repair the disk.

Yeah I figured it's very bad. The system can still (partially) boot up.

I really need to figure out how to backup everything. Any suggestions? That's the priority.

you can boot from an install-cd, or use a live-cd distro to boot.

then mount the disk and save the data to another disk.
after that try to run fsck on that partition. I think you'll need
( at least ) the -c option for bad-block checking.
see "man fsck".

good luck

egag

Get a bootable CD. Use it to mount a remote NFS or CIFS (windows) filesystem that you can write to with enough space to backup the drive, or plug in a USB drive, or just put the dying drive in another machine that isn't broken (probably simplest/fastest).

Once you've gotten a *working* chassis and this hard drive together, you can use dd to begin a backup, and *hope* for the best. dd won't properly back the thing up without being given a bit of a shove tho'. You'll need to tell it a few special things, aside from just if=/dev/whatever of=/mnt/backupspace/image.img just in case...

ibs=512
obs=512

These set the size of each read and write dd does to 512 bytes, the idea of which is to get exactly one disk sector per pass of the loop. The man page is poorly worded, and it's likely that these are needed separately rather than just bs= which normally would set both at once.)

conv=noerror,notrunc,sync

This bit is important. Without noerror, the first bad block will bring dd to a screetching halt--not useful. I've always put notrunc in for the sake of paranoia, but it might not actually be needed. sync is needed to be sure that if the drive does happen to actually give you a partial read on a sector that it will fill in the output block that will be written with nulls so that it stays the right length of 512 bytes per block. When it finally grumbles to a halt, you're done and you should have an image of the disk. You can then chop it into pieces to turn it into partitions/filesystems that you can mount over the loop device, or just write it straight out to a replacement drive (of the exact same size and geometry). (Yeah you can clone one partition at a time into images, but I prefer not to sit around and wait for each one to complete.)

I would still strongly recommend using the manufacturer's diagnostics tools to at least test the drive electronics' sanity. If it's the cache memory on the hard drive or something that's gone bad, getting a clean image of the drive is going to be a bitch and involve reading each sector multiple times until you've gotten the same answer enough times to feel safe about it.

It's possibe that the problem is not with the plates of the harddrive, but with its circuit board. If this is the case, you might be able to get an identical harddrive and swap circuit boards. This can be a bit dicey, so don't try it unless everything else won't work, and you've followed evilDagmar's instructions.

Okay, I got a bootable CD and I'm in the command prompt now. fdisk -l lists hda1, hda2, hda5, hda6, hda7, hda8.

I could mount hda1, but the rest I couldn't. 6, 7 and 8 would get a "/dev/hda#: Input/output error" and then say to specify the filesystem type.