Hello...

I have a folder in which I store letters that I have written, creatively called 'letters'. Within this folder I have more folders for each person or company that I communicate with, which obviously contain the relevant files/letters.

I have however, over the last number of months, started to ignore this folder structure and have begun to store letters directly within the letters folder without dropping them into sub-folders.

In order to make this manageable I have changed how I name files and now employ a specific naming routine... in the case of letters, I use the following naming convention:

recipient-date_of_letter_(year.month.day)-subject.extension

This method of a more-or-less flat file structure seems very natural to me.

Taking this to extremes this could result in a single partition having say 100,000 files stored flatly instead of within a hundred (for arguments sake) or so folders/directories.

The question is, is a flat file structure less, equally, or more efficient than a directory based structure, putting aside the delay that would occur as a graphical file manager attempts to display the contents of a flat structure with so many files?

Any thoughts on the efficiency and any pros and cons would be very welcome.

Thanks
Jamie.

It depends quite a bit on which filesystem you are using. EXT2/3 in general don't perform as well with (very) large numbers of files as more advanced filesystems like XFS and JFS; ReiserFS specifically is supposed to be very good with large directory trees comprised of many small files.

That said, 100K files isn't really a lot in the grand scheme of things; there are mail servers out there which are storing many times that number of files. All of the filesystems should be able to cope with that number, and the real world performance impact will likely be very small. I would still look into using XFS or ReiserFS, but that is more personal taste than anything.

If i am not mistaken, putting files into folders simply means that the file entries have codes added to associate them with directories (folders)---which are actually files themselves. The associaton of files with folders does not affect how much space is used.

I would organize thing into folders based on what make you most efficient on the user side.

Bingo.

I also tend to place like files in single folders, and name them in such a manner that I can type 3-4 characters into my file browser and quickly find the file I want. I find that this method is the fastest and most efficient for me.

The main performance bottleneck may be whatever you're using to browse the files. While the file system driver may have no particular issues with thousands of files in one directory, your file browser might bog down.

This can be a particularly annoying performance issue if you're browsing over a network or over the internet.

Possibly 100,000 loose files of different written letters and papers you've created over time? If I were you, I'd probably create me a database of these in MySQL or PostgreSQL, maybe SQLite even to organize these and to access them.

The larger the num of files, the better it is to structure eg use sub-dirs. Its easier to manage.