We put out a line of inexpensive web servers with a pre-packaged specialy web app loaded on it. (It controls some specialized machienery via a web application). We build the boxes on RedHat 8.0 distributions with minimal components - basically just apache, mysql, and php support.

Now that it's monsoon season here in the southwest, there are a lot of short term power outages, almost nightly. These power outages are corrupting our systems. There doesn't seem to be any hardware damage, it's just file system errors and problems due to the loss of power and improper shut down.

Linux has always been WAY more stable than windows, but I've killed power to windows machines hundreds of times over the years and never had any problem getting one to boot right back up. However, these linux boxes are having boot problems nearly any time they loose power.

Is there some way I can make a linux box more fault tolerant to this? I know adding a UPS power supply is the first obvious solution, and we will pursue this, but it bothers me that linux seems so finickey over improper shutdowns.

I'd like some startup options to specify that cause the OS to automatically diagnose and repair simple file errors on bootup. The console shows "improper shutdown, force check of file systems?" or something to that effect, but it doesn't perform this task by default.

I am using a combination of ext3 and ReiserFS partitions. The ReiserFS partitions seem to be just fine, but the ext3's are having problems.

Help???

Maybe look here: http://www.netadmintools.com/art178.html (especially the 'data=journal' option for Ext3). I haven't tried it myself, but it looks it may help.

Have you considered a UPS (Uninterruptable Power Supply)? Most of the modern ones can be setup to cleanly shutdown your system after a power failure (after a certain amount of failure time). Plus they condition the line, and if the PCs are connected to a circuit with heavy machinery (esp. motors), this will greatly increase the life of the PC equipment.

You've GOT TO, GOT TO (one more time) GOT TO have a UPS with Linux.

Dunno about Windows, but with Linux, writes to the disk are often cached, meaning they are not written immediatly to the disk, but are stored in memory, and written to the disk at intervals. This means that if the power goes out unexpectedly, your files can (and probably will) become corrupt.

You can get an inexpensive UPS for about 60 bucks. Look for one with a USB port and cable, which you can plug into your box. It's fairly straightforward to set up Linux to sense when the power goes out, and shut down safely.

One more time. You've GOT TO get a UPS. Anyone who does not have a UPS under Linux is begging for trouble.

You don't even need a 'modern' UPS to get your system to shut down after a power failure... you can use the dumbest UPS in the world, and still have your system shut down safely when the power goes out. Here's the idea:

http://www.tldp.org/HOWTO/UPS-HOWTO-7.html

The idea here is that you find an old serial modem, and hook it up. Tell 'powerd' that the serial modem is your UPS. When the modem loses power, 'powerd' does it's thing, and brings your system to a safe shut-down. This is not as nice as having a 'smart' UPS, but it works great for me!

Now that's cool. Would never thought to try that.

Thanks.

Thanks for the help. I'll check into the ext3 journal options, might not be a bad idea.

I didn't realized linux cached so much write data for the drive - this actually makes a LOT of sense over windows, it also explains why our systems are getting messed up.

I didn't realize it was so easy to get a UPS to tell the linux box that power had gone away. Now if I can get a UPS to tell more than one PC that power went away (usually we hook up 2 or 3 of them, and getting that many UPS units is kinda silly) I'll be all set.

Thanks again for some good info.

There are many different kinds of UPS. The ones that cost less than 100 US dollars are stand-by UPS. These are not truly UPS because when the power goes out it switches off for a few milliseconds to switch to the battery. A true UPS device is an in-line. These UPS are always hooked up to the battery. If the power goes out or the voltage levels goes lower than 100 volts, your system will still get the normal AC voltage 120/240.

You can always change to a different filesystem like reiserfs, xfs, jfs....

You must have a UPS. Our power doesn't go out it just blinks a lot. I have a home system and my UPS went out. It's will be here soon. Linux usually empties cache about every 30 to 60 seconds depending on how much data there is.
I plan to buy one made by Cyberpower. It's about $150.00. It's a bit of overkill but since I have a large power hungry 19" monitor. . . .

I read somewhere that you can make a whole network of systems shutdown when power fails. I think it's upsmon? I'm not sure. If not pm me and I'll look to see if I can find it. It's on my system somewhere, I think.

Good luck.