LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 12-27-2007, 05:31 AM   #1
carpman
Member
 
Registered: Jan 2002
Posts: 54

Rep: Reputation: 15
optimise xfs for large partition


Hello, ok i am building new 64bit system with Gentoo and will have 650gb partition for storing my digital images. file will range in size from 3mb to 200mb with average about 20mb.


The partition is on a raid 1 array, the hardware for array is areca sata pciE raid card with 256mb ram, 500mhz processor, drives are 2x750gb Hitachi 7k1000. The 650gb array is set 128k stripe size.


Now i have read and used the IBM article for setting up XFS but never for array this size.

The guide talks about allocation groups


Quote:
The second option lets you enhance the performance of your new filesystem by telling mkfs.xfs to minimize the number of allocation groups that are created. Normally, mkfs.xfs chooses the number of allocation groups automatically, but from my experience it usually picks a number that is a bit too high for most general-purpose Linux workstations and servers. As you'll recall from my previous article, allocation groups let XFS perform multiple metadata operations in parallel. This comes in handy on high-end servers, but too many allocation groups do add a bit of overhead. So rather than let mkfs.xfs choose the number of allocation groups for your filesystem, specify a number by using the -d agcount=x option. Set x to a small number, something like 4, 6, or 8. You'll need at least one allocation group for every 4 GB of capacity in your target block device.


but following that a 650gb xfs partition will have very high number for allocations groups, is there max or should go with 1 for every 4gb?


If do then allocation group number would be in the thousands!!



Any tips for setting up xfs for this partition?

cheers

Last edited by carpman; 12-27-2007 at 05:39 AM.
 
Old 12-27-2007, 03:05 PM   #2
jay73
Guru
 
Registered: Nov 2006
Location: Belgium
Distribution: Ubuntu 11.04, Debian testing
Posts: 5,019

Rep: Reputation: 130Reputation: 130
If the average file size is only 20MB, you have other things to worry about than allocation speed, especially on a computers with moderate specs (xfs tends to require a good deal of RAM to perform well) . Having too many allocation groups tends to become non-productive - if not counterproductive - beyond a certain point as performance will be affected by increased search activity.
It may be more interesting to focus on the area where xfs tends to lag a bit:file deletion. Xfs is noticeably slower in this respect than ext3 or reiser,especially when it comes to smaller files. Remember that xfs is optimized for large files, i.e. hundreds of megabytes. The good news is that you can get noticeable performance gains by manipulating the number and size of logbuffers. Just prepare your partition(s) with mkfs.xfs -l size=64m (or even 128m) and mount them with noatime,nodiratime,logbufs=8. Xfs file systems also tend to perform better if the journal is placed on a partition of its own.
 
Old 12-28-2007, 04:07 AM   #3
carpman
Member
 
Registered: Jan 2002
Posts: 54

Original Poster
Rep: Reputation: 15
Hello and thanks for reply, i am not worried about deletions times but good read then write speed would be nice.

I have amd64 3700mhz with 2gb ram soon to be 4gb so system spec is not an issue.

The IBM guide i refer to is at www.ibm.com/developerworks/library/l-fs10.html


I normally use following to create xfs FS, setting agcount= as per partition size.

Quote:
mkfs.xfs -d agcount=4 -l size=32m /dev/hda1

Which as you can see already uses logbuffer setting but i can up amount as you suggest for logbuffers but i am still not sure about agcount= setting, how many for a 650gb partition?

Also can speed be improve but using other xfs settings such as block size?

cheers
 
Old 12-28-2007, 05:46 PM   #4
jay73
Guru
 
Registered: Nov 2006
Location: Belgium
Distribution: Ubuntu 11.04, Debian testing
Posts: 5,019

Rep: Reputation: 130Reputation: 130
http://everything2.com/index.pl?node_id=1479435

Maybe you should use something like Bonnie to find out what works best for you.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Resize xfs partition fibster Slackware 1 09-28-2007 09:40 PM
Formatting partition with XFS. duffmckagan Linux - Software 1 10-30-2005 01:58 PM
Mounting my XFS partition monks1975 Fedora - Installation 1 08-04-2005 03:52 AM
XFS Partition problem qwerty_int Linux - Software 0 09-20-2004 01:05 PM
mounting a partition XFS ??? osat3ch Linux - General 3 08-30-2004 10:57 PM


All times are GMT -5. The time now is 08:13 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration