Review your favorite Linux distribution.
Go Back > Forums > Linux Forums > Linux - General
User Name
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.


  Search this Thread
Old 09-15-2008, 01:03 PM   #1
LQ Newbie
Registered: Jul 2006
Location: Scotland
Distribution: Gentoo
Posts: 21

Rep: Reputation: 0
Is it more efficient to store files flat or in folders?


I have a folder in which I store letters that I have written, creatively called 'letters'. Within this folder I have more folders for each person or company that I communicate with, which obviously contain the relevant files/letters.

I have however, over the last number of months, started to ignore this folder structure and have begun to store letters directly within the letters folder without dropping them into sub-folders.

In order to make this manageable I have changed how I name files and now employ a specific naming routine... in the case of letters, I use the following naming convention:


This method of a more-or-less flat file structure seems very natural to me.

Taking this to extremes this could result in a single partition having say 100,000 files stored flatly instead of within a hundred (for arguments sake) or so folders/directories.

The question is, is a flat file structure less, equally, or more efficient than a directory based structure, putting aside the delay that would occur as a graphical file manager attempts to display the contents of a flat structure with so many files?

Any thoughts on the efficiency and any pros and cons would be very welcome.

Old 09-15-2008, 01:24 PM   #2
LQ Guru
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
It depends quite a bit on which filesystem you are using. EXT2/3 in general don't perform as well with (very) large numbers of files as more advanced filesystems like XFS and JFS; ReiserFS specifically is supposed to be very good with large directory trees comprised of many small files.

That said, 100K files isn't really a lot in the grand scheme of things; there are mail servers out there which are storing many times that number of files. All of the filesystems should be able to cope with that number, and the real world performance impact will likely be very small. I would still look into using XFS or ReiserFS, but that is more personal taste than anything.
Old 09-15-2008, 02:43 PM   #3
LQ Veteran
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
If i am not mistaken, putting files into folders simply means that the file entries have codes added to associate them with directories (folders)---which are actually files themselves. The associaton of files with folders does not affect how much space is used.

I would organize thing into folders based on what make you most efficient on the user side.
Old 09-16-2008, 07:55 AM   #4
Registered: Oct 2004
Location: Southeast, U.S.A.
Distribution: Fedora (Desktop), CentOS (Server), Knoppix (Diags)
Posts: 934

Rep: Reputation: 38
Originally Posted by pixellany View Post
I would organize thing into folders based on what make you most efficient on the user side.

I also tend to place like files in single folders, and name them in such a manner that I can type 3-4 characters into my file browser and quickly find the file I want. I find that this method is the fastest and most efficient for me.
Old 09-16-2008, 12:21 PM   #5
Senior Member
Registered: Apr 2004
Location: Baton Rouge, Louisiana, USA
Distribution: Debian 9 Stretch
Posts: 2,174
Blog Entries: 8

Rep: Reputation: 340Reputation: 340Reputation: 340Reputation: 340
The main performance bottleneck may be whatever you're using to browse the files. While the file system driver may have no particular issues with thousands of files in one directory, your file browser might bog down.

This can be a particularly annoying performance issue if you're browsing over a network or over the internet.
Old 09-16-2008, 02:03 PM   #6
LQ Guru
Registered: Jan 2001
Posts: 24,149

Rep: Reputation: 235Reputation: 235Reputation: 235
Possibly 100,000 loose files of different written letters and papers you've created over time? If I were you, I'd probably create me a database of these in MySQL or PostgreSQL, maybe SQLite even to organize these and to access them.
Old 09-16-2008, 08:33 PM   #7
LQ Guru
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,406

Rep: Reputation: 2396Reputation: 2396Reputation: 2396Reputation: 2396Reputation: 2396Reputation: 2396Reputation: 2396Reputation: 2396Reputation: 2396Reputation: 2396Reputation: 2396
The larger the num of files, the better it is to structure eg use sub-dirs. Its easier to manage.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
LDAP authentication and flat files bx.s Linux - General 5 10-27-2006 04:18 AM
flat files or xml and css? trscookie Linux - General 1 01-17-2006 09:24 AM
Samba can create new files and folders but access denied in any new folders k.king Linux - Networking 2 01-15-2006 06:14 AM
Using flat files or tables with IPTables? tarballed Linux - Security 1 12-09-2004 04:08 PM
Flat files vs Databases coolman0stress Programming 5 09-08-2003 09:23 PM > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 02:56 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration