hello,

well i'm looking for a potential new file system that is more suitable for the storage of static files such as backups and so forth.
my major problem with ext* is that it has this overhead on the raw storage device therefore reducing the amount of data that can actually be stored.

now personally , i'm yet to become an expert on file systems which is why i need your advice.

is there anything else other than ext that is stable and reliable to enough the serve as a format for storing large static files?
if so then what are the tradeoff ?
and please elaborate on this issues or provide links to articles that fully explain what is what..

thanks

Explain this statement.
There are innumerable threads/articles re the supposed benefits of various f/s architectures and their implementation(s). Usually written by a proponent of the particular one being touted.
Most these days maintain metadata for data consistency. Non-journalled have very little benefit these days.

RAID5+LVM+XFS so you can do snapshots, have data integrity and redundancy and the ability to resize the volume without taking it down first. It's a popular combination for this kind of storage due to the flexibility and security. I am also looking forward for BTRFS, but it is not as proven than ext3 or XFS. I don't agree with other member that said it is that dangerous and unstable, my own tests prove the opposite, but I would not trust it for critical data yet.

I have used BTRFS on Ubuntu 10.10 Alpha and have to say it really does the job for me. I can easily create snapshots of directories (including /) and easily store extremely large files without the FS getting corrupted. It does have kind of a performance hit, though: On a BTRFS drive, it takes hours just to apt-get upgrade or apt-get install 100 packages.

in short it means , that on ext FS's you can store less than data than the actual hdd size because there is a significant overhead over file written , i've noticed that ext wastes around 10% of disk space...

here is an experiment that you might do , try to format a hdd with fat and then with ext and compare the free space available ..

so the question was , is there a stable and reliable filesystem that can save data efficiently like fat and reliably as ext ?


hi , well redundancy is assured and flexibility is not really essential , what i'm optimizing for is maximum capacity.

so the question , is XFS for capacity efficient in contrast to ext ?

i'm afraid i'm NOT interested in experimental FS's , i'm running a production site and i can't afford risks (at least not too much)

cheers

BTRFS certainly is. It does not take up any drive space itself (but the tools to modify it (btrfsctl, btrfsck) certainly do). And if that's not enough, BTRFS also allows file and folder compression to actually make the files and directories smaller.

^^Yea, but it is experimental, even if it is backed by Oracle and RedHat.
XFS will do, it is the best FS for large files, but lack the compression ability of ZFS or BTRFS. ZFS is not experimental, but the OS running it is dead, so it is not the best idea, even if it is the best File System.

As syg00 said
If you just want max storage and are willing to forego the benefits of journalling, try ext2 (effectively ext3 without journalling).
Note also that by default most fs creation cmds reserve a certain amt of storage on disk for easing potential fragmentation issues (ie blocks used temporarily for re-ordering) and also for the root user who may need a small scratch area to sort things out when the users fill it up.
This space is normally configurable by the relevant parameter to the mkfs cmd. Again, if you want to reduce it you can, but it may not be wise.

Elv13
News to me that Solaris is dead(!) ... care to provide a citation...? Also available as a userland FS in Linux.
http://en.wikipedia.org/wiki/ZFS

OpenSolaris is dead, quote are everywhere. It's experimental in FreeBSD and Solaris does not worth paying for just for ZFS. And (speculation) Oracle may make Solaris SPARC only, so they complete their vertical market. But they need a lot of money to keep SPARC viable against POWER or x86_64 based multi CPU XEON beasts. SXE (Solaris Express Edition) is not an OS, but more of a compiled code dump.
[/offtopic]

okay , thanks for the input folks

eventually , i may deploy ext2

cheers

Use ext3 or ext4, ext2 is far too unstable for precious data. It can get badly corrupted with any power failures. The journal does not create that much trouble for big files.

That sounds like too much, maybe you should research the e2mkfs/tune2fs man page (-m) and set the space reserved for root to 0% first. I could be wrong, but I think you are "in search of a problem for this solution".

There will be still an overhead for the journal, but every actual fs has that problem, unless you want to use fat or ext2.

I'd only consider it safe as long as you can guarantee 24/7 power supply with 0% error. My own experience (from years ago, anyway, it takes time for me to forget this kind of thing).

About btrfs, well, I am one of those that look at it with interest, but I wouldn't use it for anything serious yet. For those talking about stability, btrfs has had many nasty problems recently, the last one in the recent 2.6.34 series (we are at 2.6.35.2 now, so it hasn't been long ago) about data corruption and free space calculation. Sure it's promising, but it's not mature and I disregard the usage for backups. The whole point of backups is reliability, otherwise just leave the files in your main fs and try to be careful. It's not good when your backup media is more experimental than your main system just to be cool.

ext reserves 5% by default, not 10. If you get the impression it is more, you may not have noticed that the size of your hard drive is not what it is in the first place. Hard drives provide 7% less space than advertised * (thus a 1TB drive has only 930GB of usable space. Deduct another 5% reserved by the fs and yes, you would lose about 10% in all. But changing file systems is not going to recuperate the "missing" 7%.

XFS does not reserve 5%, more like a few dozen megabytes on a large drive. I have found it to be reliable but you need to make sure that you enable barrier (the default) unless you have a hardware raid controller. The real advantage of XFS is raw speed; it is easily 25% faster than ext except when you are handling many small files, which requires some customization to make it perform as well as ext.

--------------------------------------------------------------------------------------------------------------
* Which is actually false: 1 kilobyte is 1000 bytes, not 1024 (just as a kilogram is 1000 grams, not 1024). In other words, hard drive manufacturers provide exactly what they promise, it's just that most users are confused because most other hardware (and software) does define kilobyte as 1024 (more properly: kibibyte).

Actually, at the beginning of this year Oracle bought out Sun and decided to charge $$ for security updates to Solaris, which basically breaks OpenSolaris. Simply put, Solaris is not dead but proprietary, and as we all know, proprietary operating systems/file systems are horrid.

For big files on big (huge) modern disks, ext3/4 will create way more inodes than you need.

Do "man mkfs.ext4" and read it.

Then look at /etc/mke2fs.conf

You probably want to do "mkfs -t ext4 -T largefile4" or something like that.