I'm working on a new project, for which I will need to access a large (anywhere up to 1000's) quantity of very small files (probably a median size of under 100B). I need to maintain access permissions (read and write for owner, group, and other), although executable permissions are not necessary. I must be able to arrange these files in a hierarchy structure. It is likely that files sharing a directory will most often be used together.

So, I realize that I have two main options - an archive or a file system. If I choose a file-system, it will be loop-mounted. I considered a file system with a very small block size. If I chose a file system, I think I would either use ext2 for its simplicity or reiserFS for its claims of good performance with small files. If I choose an archive, I like the compression of zip, but the portability of tar.

Suggestions? Could a database be a better solution? The primary goal is good access times and reducing the number of disk reads.

P.S. Is it possible to use a file lock on a file inside an archive? It would be a nice feature, but it can be done without.

Okay, I've changed my needs bit for this project and, I think, effectively answered my own question. Instead of a large quantity of small files, I'll really just need a smaller quantity of more average sized files. However, the ability to lock a file has gone from "would be nice" to a requirement. I think that rules out an archive, so I'm back to a file system. I'll probably be using ext2, unless reiserFS really blows me away.

bonnie++ is supposed to be good for testing FS performance. Please post back what you choose & why, particularly any testing you do.