LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > General
User Name
Password
General This forum is for non-technical general discussion which can include both Linux and non-Linux topics. Have fun!

Notices

Reply
 
LinkBack Search this Thread
Old 05-20-2004, 06:08 AM   #1
neranjana
Member
 
Registered: Jul 2003
Location: Sri Lanka
Distribution: Mandrake
Posts: 99

Rep: Reputation: 15
Maximum number of files in a directory


Hi

Can anybody tell me how many files could be created in one directory? I just need a rough figure. 10,000? 1,000,000? 1,000,000,000,?

Thanks

Neranjana
 
Old 05-20-2004, 06:27 AM   #2
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 194Reputation: 194
I would think it depends on your hard drive space actually. Most configurations if your using Linux use 4k inodes. So each directory or file at minimum would take up 4k of space by default.

So lets make it easy, doing some simple math and hopefully I did it right as its way to early, if you had a 2 GB partition using 4k inode allocations, you could fit 500,000 files on the partition.

So your question I think is irrelevant to directories, it all depends on your drive itself, size of inodes and size of all the files, etc.

Lets just put it this way, too hard to determine, so if it was a multiple choice quiz question, I'd choose: "Not enough information provided to answer"

 
Old 05-20-2004, 07:00 AM   #3
dave_starsky
Member
 
Registered: Oct 2003
Location: UK, Manchester
Distribution: Gentoo (2.6.10-r4) & Ubuntu
Posts: 145

Rep: Reputation: 16
i'm sure you can have as many files as you want in a directory. A directory is merely a file which stores information about the other files which are "within" it.
 
Old 05-20-2004, 05:22 PM   #4
SciYro
Senior Member
 
Registered: Oct 2003
Location: hopefully not here
Distribution: Gentoo
Posts: 2,038

Rep: Reputation: 51
it also depends on the file system used, reiserfs can support a lot... others can only support a few ..
 
Old 05-20-2004, 09:36 PM   #5
neranjana
Member
 
Registered: Jul 2003
Location: Sri Lanka
Distribution: Mandrake
Posts: 99

Original Poster
Rep: Reputation: 15
This is a question to trickykid.

Suppose I have ext3 and I have 80 GB partition. Suppose the size of the file is immaterial. (I would not run out of disk space.) But there are some problems. As you have mentioned, if it takes 4k to allocate a file then if I have say 100 files of the size of 10k then that would mean it would take 1MB for the files. Suppose this is not important. But the directory would take 4*100 k just to keep the info about the files, wouldn't it? I've heard that there is a limit of 4GB for a file in Linux. So it would mean that any directory could only be as big as 4GB (this is without the actual files inside it. Just the file names.) So it would mean that 4,000,000/4 would be the maximum number of files that a directory could keep inside it, wouldn't it?

Am I on track or am I way off the mark?
 
Old 05-20-2004, 09:53 PM   #6
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 194Reputation: 194
Quote:
Originally posted by neranjana
This is a question to trickykid.

Suppose I have ext3 and I have 80 GB partition. Suppose the size of the file is immaterial. (I would not run out of disk space.) But there are some problems. As you have mentioned, if it takes 4k to allocate a file then if I have say 100 files of the size of 10k then that would mean it would take 1MB for the files. Suppose this is not important. But the directory would take 4*100 k just to keep the info about the files, wouldn't it? I've heard that there is a limit of 4GB for a file in Linux. So it would mean that any directory could only be as big as 4GB (this is without the actual files inside it. Just the file names.) So it would mean that 4,000,000/4 would be the maximum number of files that a directory could keep inside it, wouldn't it?

Am I on track or am I way off the mark?
No, you could have a directory with three 4gb files that reside in that one directory. Just cause a files reside in a directory doesn't mean that all the files are limited to 4gb in this case you explained, the filesystem doesn't work like that..
 
Old 05-21-2004, 12:19 PM   #7
dave_starsky
Member
 
Registered: Oct 2003
Location: UK, Manchester
Distribution: Gentoo (2.6.10-r4) & Ubuntu
Posts: 145

Rep: Reputation: 16
The size of the files doesn't matter, the directory is a file which contains references to the files, so it just has to store the address for the file. (i think thats how it works anyway)

plus, everything is inside the / directory
 
Old 04-14-2009, 12:48 PM   #8
Lazypete
LQ Newbie
 
Registered: May 2008
Posts: 7

Rep: Reputation: 0
Actually as trickykid said no, you can have 3 files of 4GB in the same dir. Its actually a physical limitation. 4GB is the maximum of a 32bit system, because the maximum number you can write is 2^32 = 4 294 967 296
so you can't say I would like to request bit number 5 000 000 000, that number can't be sent over a 32bit bus.

On a 64 bit system that barrier is lifted to a much higher value 2^64

There are two things you must consider when figuring the file system.
The space a file use, its data part, and the commonly forgotten aspect that this file also need adressing in the file allocation table.

There is usually a maximum number of file in a single directory, I know there is one in FAT16 and FAT32. I just dont know if there are in other filesystem.

Maybe the way ext2, ext3, ext4 or reiserfs are build there is no such limit.
 
Old 04-14-2009, 01:36 PM   #9
sundialsvcs
Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,054

Rep: Reputation: 954Reputation: 954Reputation: 954Reputation: 954Reputation: 954Reputation: 954Reputation: 954Reputation: 954
Actually, what I would suggest doing is ... use a database, or databases. That's basically what you're doing anyway, but you're doing it the hard way. If you start running into the general vicinity of "a design limit," back off.
 
Old 04-16-2009, 11:49 AM   #10
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Rep: Reputation: 114Reputation: 114
I almost bit on this too, but I reviewed the thread, saw the dates, and said "this is a really moldy oldie".
 
Old 04-16-2009, 10:46 PM   #11
ProtoformX
Member
 
Registered: Feb 2004
Location: Canada
Distribution: LFS SVN
Posts: 334

Rep: Reputation: 33
Quote:
Originally Posted by Lazypete View Post
Actually as trickykid said no, you can have 3 files of 4GB in the same dir. Its actually a physical limitation. 4GB is the maximum of a 32bit system, because the maximum number you can write is 2^32 = 4 294 967 296
so you can't say I would like to request bit number 5 000 000 000, that number can't be sent over a 32bit bus.

On a 64 bit system that barrier is lifted to a much higher value 2^64

There are two things you must consider when figuring the file system.
The space a file use, its data part, and the commonly forgotten aspect that this file also need adressing in the file allocation table.

There is usually a maximum number of file in a single directory, I know there is one in FAT16 and FAT32. I just dont know if there are in other filesystem.

Maybe the way ext2, ext3, ext4 or reiserfs are build there is no such limit.
That would be wrong my friend, sorry to intrude but 32bit systems since the Pentium Pro are limited to 64gigs, PAE (Physical Address Extension) can extend the upper limit using 36bit addressing instead of 32.

Also this only applys to memory, not hard drive space, if that was the case everyone who didn't have 64bit systems would be limited to a 4096mb hard drive. Since this is not the case the amount of data on a harddrive is irreverent, even if it where, you could always limit the addressing to 32 or even 36bits and have a zillion bit addressing, it would be horribly slower, but can be done via look up table. See unreal mode programming, you basically enter 32bit addressing space save the memory table in the stack and return to 16bit real mode reload the stack and you are now using a 16bit mode with 32bit addressing...so now you have all the benefits of 32bit protected mode with the speed of 16bit real mode. but you can only access 64kb of memory at a time (not a huge deal for an assembly coder).
 
Old 09-30-2009, 01:30 AM   #12
demerson3
Member
 
Registered: Sep 2004
Location: Los Angeles
Distribution: debian stable
Posts: 43

Rep: Reputation: 15
JUST READ WIKIPEDIA! (read both ext2 and ext3, reiserfs, etc.)

All the answers are there!

There is a LOT of misinformation in this thread (e.g. 4 GB max file size -- not true)
 
Old 05-13-2010, 03:09 AM   #13
Frustin
Member
 
Registered: May 2002
Location: Essex, UK
Distribution: Debian, Redhat, AIX 5L
Posts: 511

Rep: Reputation: 30
Quote:
Originally Posted by demerson3 View Post
JUST READ WIKIPEDIA! (read both ext2 and ext3, reiserfs, etc.)

All the answers are there!

There is a LOT of misinformation in this thread (e.g. 4 GB max file size -- not true)
right, so tell us; using the information on wikipedia, how many?
 
Old 05-13-2010, 12:00 PM   #14
smeezekitty
Senior Member
 
Registered: Sep 2009
Location: Washington U.S.
Distribution: M$ Windows / Debian / Ubuntu / DSL / many others
Posts: 2,208

Rep: Reputation: 147Reputation: 147
All of you act like wikipedia is so untrustworthy yet its so well maintained, i think you are more untrustworthy.
 
Old 05-13-2010, 01:33 PM   #15
demerson3
Member
 
Registered: Sep 2004
Location: Los Angeles
Distribution: debian stable
Posts: 43

Rep: Reputation: 15
Quote:
Originally Posted by Frustin View Post
right, so tell us; using the information on wikipedia, how many?
Are you serious?

The maximum is 392,471,693. And five eighths. With a cherry on top.

/sarcasm

It's not so simple as that: the number is different for every partition, and is equivalent to the maximum number of inodes for that particular partition, which depends on the file system type, file system size, and block size. Most people will run out of drive space before they run out of inodes, so this is rarely an issue. If it's that important to you, go read wikipedia and figure it out for yourself. I'm not going to try and summarize what's there because you are too lazy to do some research. If you have an unusual use case where you might potentially run into the limit, then you'll need to understand what wikipedia says on the subject anyway.

You're a big boy, you can read and do math without my help.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Number of files in directory bullium Linux - General 13 08-31-2011 08:50 AM
Limit of maximum files/direstories per directory? sindri Linux - Software 3 11-14-2004 11:56 AM
finding number of files in a directory pantera Programming 2 08-20-2004 10:17 AM
Maximum Number of Directory Entries & Performance aig Linux - General 1 07-09-2004 07:36 AM
finding the number of files in each directory ridertech Programming 1 06-02-2004 10:26 PM


All times are GMT -5. The time now is 10:56 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration