LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 10-06-2003, 12:46 PM   #1
dman65
Member
 
Registered: Sep 2003
Posts: 61

Rep: Reputation: 15
File Systems and Numerous files per directory


I have been in the process of trying to replace a Windows NT 4.0 server with a Linux server for the past couple of weeks.

The Windows NT server has 923,176 files in a directory.

After getting everything worked out with Samba I started copying files over to it. At first the transfer was going pretty well with the network showing anywhere from 40-60% utilization (100Mbit ethernet). However, as more and more files were copied the rate started to slow.

I am assuming this is happening because Linux is trying to insert the new entries into some type of directory structure. I have tried copying the files to an NT server and there is no corresponding decrease in transfer speed as the number of files grows.

Is there any kind of file system for Linux that does not have this directory entry problem? Or is this problem related to something totally different? (kernel 2.4.18) I have used EXT3 and XFS file systems so far and neither seems to work well with large numbers of files.

I tried splitting these files up so that there were no more than 10,000 entries in any directory and the transfer speed remained up around 18GB per hour, but all of our existing software expects those files to be in the same directory.

Thanks for any assistance.
 
Old 10-06-2003, 04:32 PM   #2
teval
Member
 
Registered: Jul 2003
Location: Toronto, Canada
Distribution: Gentoo
Posts: 720

Rep: Reputation: 30
Have you tried ReiserFS? I heard it's good with that kind of thing.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to delete files with 0k file size in a directory justgiver Linux - Newbie 4 01-28-2008 04:56 AM
Changing file permissions within a directory for all files. cajunaggie Linux - Newbie 2 02-06-2005 01:53 AM
How to do recursive file copy of directory for specific files? Arodef Linux - Newbie 4 06-29-2004 05:35 PM
Massively renaming numerous similar files khermans Linux - General 12 06-29-2004 02:34 PM
recursive file permissions does not change new files in same directory PAB Linux - Newbie 2 03-08-2004 12:27 PM


All times are GMT -5. The time now is 09:04 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration