Hey,
I'm working on an application that requires a large amount of storage space and I want to handle storage `in-house` (Much cheaper than, say, S3) so we will have multiple servers (Initially 4) with large amounts of storage (6TB each).
The storage will need to be very flexible and configurable, each piece of data should be replicated on at least 2 servers and must be easily readable/writable from ether an API of a UNIX device/file/folder like a normal drive, I don't mind which. We must also be able to easily offload content to our HTTP CDN (Edgecast), it doesn't need to have built in HTTP support but if it doesn't I'm going to have to write something to get the files onto HTTP so they can be pulled by the CDN.

I've looked at a lot of solutions including
Eucalyptus Walrus
OpenStack Object Storage
MogileFS
MongoDB GridFS (I'm not sure why, it just sounded cool =) )
and some others which I can't remember

All the servers will be running RHEL 6, they have 4x1.5TB drives which will be RAID1'd into a single partition. All the servers have 1GB/s connections between them and 100MB/s connections to the internet with unlimited bandwidth. They have 2x2.66ghz processors.

I understand there isn't a single, perfect answer but it would be nice to get some pointers.
-Sam

When I read the header my first association was DRDB. MongoDB GridFS looks completely different and supplies only a distributed database for MongoDB when I get it right but you need a complete file system?

Really what we're looking for is an internal system like Amazon S3, I know Walrus provides exactly that but I didn't know if there was something a bit easier to configure/maintain.