I'm trying to find out what filesystem should I use if I know in advance that a running Linux system is going to have the power pulled from underneath it.

I'd read about tux2, but it looked like all the patent problems with NetApp had run it off the road

I basically need a filesystem that after a hard crash isn't going to need manual intervention to bring it back up. I've been using reiser for a while now, but even with reiser I've experienced problems with sudden power outtages and had to run fsck.reiser on the partitions to fix things.

Does such a thing exist?

I do not know of any filesystem that is impervious to being corrupted by a power outage.

Can you explain your application and why you want to pull power instead of a normal shutdown?

An OS loaded to RAM or ROM based system would work.

I have requirements that don't permit me to use a UPS, and the power is very unpredictable quite often dropping without warning.

What I had in mind was something along the lines of whatever Nomad Jukeboxes, or TiVo like set top boxes that can have their power pulled from without notice and still power back on without any disk corruption.

Anyone know how they do that?

Educated guess: they mount their FS read-only?
As they are set-top boxes there shouldn't be a need
to write to disk?


Cheers,
Tink

I don't think that TiVo boxes mount their partitions read only, how else would they be able to store the tv that they record to HD?

Hmmm ... maybe a partitioning scheme that will only have
that particular data-stream mounted on a writable partition?

Stabbing in the dark here, but really if they are writable they
will have the same problem you do...

Anything else is PHYSICALLY impossible.


Cheers,
Tink

I see where you're coming from tink, but I'm thinking there's something more to it (perhaps a bespoke FS - I'm joining you in stabbing in the dark here ).

The reasoning behind there being more to it is that in consumer devices like the Nomad jukeboxes or TiVo boxes is that the end user isn't a computer user, they don't know about what's going on under the covers, they just expect it to work. Now, I've never ever heard of a TiVo or a Nomad (or iPod, etc.) having to be returned to the manufacturer because the batteries were pulled out of the thing or the power lead being yanked out not giving it a chance to shutdown gracefully - it just works, no evidence of an fsck, no lag in getting the device to start up again, they all just giddy on up

I like the idea of mounting the OS partition as read only, I'm **assuming** (I know I know..an ass outta.....) that you could pull the power on a read only partition as much as you like and not do any damage to the partition. Now we've just got to figure how to make sure that the pesky partition that you want to write data to doesn't get corrupted after a power down..

Has anyone worked on systems like this before that could cast some light upon our speculations?

Thanks for the replies tink, they've got me thinking about new directions

Well ... one possibility I can think of would be a set-up
where the machine e.g. streams video to a RAM-DISK,
and once a certain level of that thing is filled up puts
a HDD partition into a RW state, writes the block, flushes
the buffers and goes RO again ... would make the chance
of a failure during write-access pretty small (not impossible).

Just think about it logically: even if those other devices
had static RAMs in the GB size range, once the tuner
stops sending a continuous data-stream it'd STILL be
chopped-off at that stage and need some sort of consistency
check ... unless of course there's a setting telling an FS
to ignore silly things as unclean shutdowns, and just
forfeit whatever was going on while it crapped out. That
should be quite possible with a journaling FS like Reiser
or the likes.


Cheers,
Tink

I don't think that a Nomad Jukebox nor TiVo Operating Systems are on the hard drive. I'm sure that they use ROMS or flash ROMs similar to the USB flash memory systems. Think of these devices as embedded processors and not desktop computers.

The storage drive on a TiVo probably does have a formatted filesystem. It is possible to write raw data to an like a file stream with fixed bytes. If recording data and power is removed the last frame of data is automatically overwritten. I use data acquisition sytems of this type.

Yeah, I understand the strategy of framing for DVR's, but even before you get to overwrite the last partially written frame that's only going to be permissible if at the FS level you've not screwed up something pretty bad (every journaling FS I've looked at suffers from the possibility of corruption).

One technique that would work would be to have 3 partitions:

partition A - OS partition - read only
partition B - workspace1
partition C - workspace2

Partition B & C would be initially set read only and the application caches data waiting to be written for a set period. The app then mounts partition B read write, writes its data to partition B and then unmounts. Once unmounted it repeats the procedure to partition C.

This way, on boot up you could check to see if either of the partitions were corrupt, if a problem is found format the bad partition and copy over the data from the good partition. By perhaps splitting partition B & C over many partitions the need to copy over data would be dropped and you would just suck up the loss of potentially a gig or two of data.

What do you think?

oh...I should have put in there that the last post was in lieu of some FS that's corrupt proof

Other than a RAM/ROM based system or read-only FS access, I would say michaelk has the best idea.

I designed a miniature server device that used a RAMDisk stored OS and Compact Flash as a storage media. For shutdown, the system would just remount everything read-only (it didn't unmount them) and cut the power.

Point is that, yes, you can kill the power to a read-only mounted file system with negligible risk of corruption. Unless there is some sort of hardware failure, anyway.

oops sorry tink, I missed your last post and the idea I put forward was kinda in the same manor that you said just using two hd' partitions instead of ramdisk. Made me feel kinda like the manager guy in the fedex adverts who does the hand wiggle thing when he repeats the same suggestion

Thanks guys for being a good sounding board