LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 07-12-2004, 06:36 AM   #1
BlueKnight
Member
 
Registered: Jul 2004
Distribution: Slackware
Posts: 32

Rep: Reputation: 15
"Really" large partitions, files and filesystems


Hi,

I have tried to sort out what would be the best, or at least a fairly good, filesystem to be used in my fileserver. I have a couple of 200GB IDE drives and a couple of 80GB drives and would like to know how to partition and format the drives in a good way.

The threads at LQ and other sites have been somewhat useful, but I have not really got sorted out a couple of things. Firstly, the fileserver will mainly be used for storing files from a couple of hundred megabytes to five gigabytes of files. So shortly, a few big files. Secondly, the fileserver will be a 2x500MHz PIII with 512MB RAM used only for this.

The things that have left unanswered are:

1) ReiserFS is said to be fast when dealing with small files (<4kB), but what about big files?

2) How (un)reliable is XFS? The issue I am addressing is the usage of memory and not writing to disk too often. Could this be a problem?

3) What about having really huge partitions like making one 200GB drive to have only one partition? Pros and cons?

4) JFS? Anyone with experience in using this filesystem?

To conclude what I am really asking is whether there is a good filesystem for making a fileserver storing really huge files on huge partitions and then share these on a LAN using Samba? Performance is essential and also to be able to have as big partitions as possible.

Thank you in advance!
 
Old 07-12-2004, 07:10 PM   #2
J.W.
LQ Veteran
 
Registered: Mar 2003
Location: Boise, ID
Distribution: Mint
Posts: 6,642

Rep: Reputation: 87
Welcome to LQ BlueKnight. This reference article does a pretty good job describing the differences between file system types, and I think would answer many of your questions. Good luck with it -- J.W.
 
Old 07-13-2004, 08:30 AM   #3
BlueKnight
Member
 
Registered: Jul 2004
Distribution: Slackware
Posts: 32

Original Poster
Rep: Reputation: 15
J.W: Thank you for the link! Interesting reading even if it was somewhat faliliar to me. The real question I am having is if there are people with practical experience of using either filesystem in a file server. XFS seems very tempting, but as I do not know if there are too many downsides with it I don't want to start using it without knowing about them. I guess I could use the maximum block size with ReiserFS as I mainly have big files.. anybody with some tips or experience to share?
 
Old 07-13-2004, 11:51 AM   #4
Tuttle
Senior Member
 
Registered: Jul 2003
Location: Wellington, NZ
Distribution: mainly slackware
Posts: 1,291

Rep: Reputation: 52
i have been using reiser for a couple of years now with zero errors, but I have heard only good things about xfs. I would go for xfs.
 
Old 07-20-2004, 02:14 AM   #5
BlueKnight
Member
 
Registered: Jul 2004
Distribution: Slackware
Posts: 32

Original Poster
Rep: Reputation: 15
I have been using ReiserFS for quite some time and I have not any direct complaints to raise. However, I cannot really say if this is a good choice as I have only had ReiserFS and Ext2/3. Therefore, I would like to hear some comments from people that have fileservers or web-servers or such and what kind of setup you are using and so on and so forth. It would be nice to get some real information how the different filesystems work in the real environments and not just to read some statistical reports.

I am sure there are a lot of people here that have good experience of various filesystems in high-trafficed environments who can give their oppinion about each of these.

Thank you in advance!
 
Old 07-20-2004, 04:22 AM   #6
MS3FGX
LQ Guru
 
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 361Reputation: 361Reputation: 361Reputation: 361
I setup a 120 GB file server and setup the drive as a slave, with a single partition and formated with ReiserFS.

It ran fine, but I was looking to get all the speed possible.

I did a custom compile of 2.6.5, compiling in XFS support and then I reformatted the drive to XFS.

The end users and myself noticed an immediate increase in speed with large files and directories (1 GB and above). I then did some transfers and timed them, and found an increase of around 20% on the user's end.

I have been running this kernel and FS combination on the server for around 3 months now, without any problems.

I would highly suggest XFS and 2.6.x if you are going to be setting up a file server, especially if you think you will be dealing with some large transfers.
 
Old 07-20-2004, 06:31 AM   #7
BlueKnight
Member
 
Registered: Jul 2004
Distribution: Slackware
Posts: 32

Original Poster
Rep: Reputation: 15
Thank you MS3FGX! This does sound really interesting, but then on the other hand what are the speeds we are talking about and what kind of underlying network setup do you have? Switches, NICs et cetera.

Have you ever had problems with the drives if the system hangs, reboots or in some other way does not shut down correctly?

This is my configuration as of now:

2xPIII 500MHz
Slackware 10
Custom kernel 2.6.7
ReiserFS on two IDE 200GB drives
Linux is on a SCSI-drive.

I have a couple of 80GB drives which I could format with XFS and try out the speed (benefits) with. I just have to find some time to do this..
 
Old 07-20-2004, 01:43 PM   #8
jong357
Senior Member
 
Registered: May 2003
Location: Columbus, OH
Distribution: DIYSlackware
Posts: 1,914

Rep: Reputation: 52
XFS does handle very large files quite nicely. Thats one of the things it was designed for... I don't run a server but have been using XFS for about a year now and have had no complaints with it what-so-ever... Very stable. I've had the power go out a couple times in the midst of editing files and it somehow always manages to save them... Well, my 2 cents... Sorry to butt in....

Last edited by jong357; 07-20-2004 at 01:45 PM.
 
Old 07-20-2004, 10:11 PM   #9
MS3FGX
LQ Guru
 
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 361Reputation: 361Reputation: 361Reputation: 361
I have my server on a UPS, so it has never shutdown unexpectedly. Hasn't crashed yet either, so I can't say how it handles a system failure.

The network is nothing fancy, mainly clients at 100 Mbps through 10/100 switches (around 100 clients at 100 Mbps), and a few clients over a 11 Mb wireless link.

There are about 40 clients over 4 different 128 K ISDN lines, but their connection is so slow that I don't even count them in my tests.

These are Windows clients by the way, with Samba 3.0.4 on the server side.
 
Old 07-21-2004, 12:34 AM   #10
BlueKnight
Member
 
Registered: Jul 2004
Distribution: Slackware
Posts: 32

Original Poster
Rep: Reputation: 15
Thank you both of you for sharing you experience! I will have to test XFS at some point to see if I get some speed increase.

Anyone with some negative input regarding XFS?
 
Old 07-21-2004, 05:14 AM   #11
Frustin
Member
 
Registered: May 2002
Location: Essex, UK
Distribution: Debian, Redhat, AIX 5L
Posts: 512

Rep: Reputation: 30
when i was testing some very large RAID boxs (14+ bay) i used a combination of iozone:

http://www.iozone.org/

and Bonnie:

http://www.textuality.com/bonnie/

If you decide to look for benchmarks already published, make sure they are up to date as the resierfs and xfs have moved along significantly in the past couple of years.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
add "Artist" and "Album Title" to mp3 files powah Linux - Software 2 04-05-2005 03:04 AM
Hang at "Unmounting remote filesystems" lagartoflojo Slackware 7 03-07-2005 02:13 PM
External USB HDD"Mounting local filesystems: /dev/sdb1 is not a valid block device" manudath Linux - Hardware 3 12-13-2004 09:59 PM
why iam getting problems like "...too many files opened" or "segmentation fault" naren_0101bits Linux - Newbie 2 07-19-2004 12:20 PM
"X-MS" cant open because "x-Multimedia System" cant access files at "smb&qu ponchy5 Linux - Networking 0 03-29-2004 11:18 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 07:25 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration