LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 01-31-2011, 06:42 PM   #1
genderbender
Member
 
Registered: Jan 2005
Location: US
Distribution: Centos, Ubuntu, Solaris, Redhat
Posts: 396

Rep: Reputation: 31
Filesystem for 45TB of storage


That's right, 45TB - here are some of the options I've considered, would like some input:

Debian+JFS - lots of guides on using this vast ammount of storage with debian and JFS, however I view debian as a linux kernel used to fix broken linux systems, also not had much experience with JFS. Command line could be complicated when fixing faulty disks (most people have little experience with faulty disks as it's a command we use so little).

Centos+EXT4 - Again, no native ZFS support, EXT4 is fairly new, not much documentation on such a vast ammount of disk space.

Opensolaris+ZFS - Fairly proficient with solaris, still no gui or documentation, benefits of ZFS make it ideal for such a large filesystem, ZFS also natively supported in the kernel, no weird hacks to get it working, solaris have a similar product which they have working well, however still cli only and read somewhere that multipliers support has only recently been introduced. Iscsi support also natively present.

Openfiler - Never used openfiler, however it has full iscsi support and a gui and I suspect decent documentation, also ZFS present in the kernel, although not a native addition.

FreeNAS + ZFS - little worried as I view freenas as a solution to peoples movie+mp3 storage and not a super efficient system, however it has vast documentation and a strong kernel (freebsd) and ZFS support with iscsi, and a command line, my biggest worry is fixes via the command line, I HATE freebsd and really struggle when using it, initiating those disks could be hard.

Planning on getting a storage pod from backblaze (documentation here. The how to's at the bottom dont specify an operating system but hints at running ZFS on fuse (not ideal).

Last edited by genderbender; 01-31-2011 at 06:51 PM.
 
Old 01-31-2011, 06:45 PM   #2
AlucardZero
Senior Member
 
Registered: May 2006
Location: USA
Distribution: Debian
Posts: 4,824

Rep: Reputation: 615Reputation: 615Reputation: 615Reputation: 615Reputation: 615Reputation: 615
ZFS has a GUI in the Oracle Web Console + the ZFS Administration applet.

What the heck do you mean by "no native command line running"?

I have no opinion on JFS, FreeNAS, or OpenFiler as I haven't used them. Of the others, ZFS on *Solaris would be the best IMO. Else, ext4 on CentOS 6 (which isn't out yet.. but should be real soon now?). ext4 is old enough now IMO.

The major benefit to ZFS and ext4 in this scenario would be the speedy fsck times (none in the case of ZFS).
 
1 members found this post helpful.
Old 01-31-2011, 06:49 PM   #3
genderbender
Member
 
Registered: Jan 2005
Location: US
Distribution: Centos, Ubuntu, Solaris, Redhat
Posts: 396

Original Poster
Rep: Reputation: 31
Quote:
Originally Posted by AlucardZero View Post
ZFS has a GUI in the Oracle Web Console + the ZFS Administration applet.

What the heck do you mean by "no native command line running"?

I have no opinion on JFS, FreeNAS, or OpenFiler as I haven't used them. Of the others, ZFS on *Solaris would be the best IMO. Else, ext4 on CentOS 6 (which isn't out yet.. but should be real soon now?). ext4 is old enough now IMO.

The major benefit to ZFS and ext4 in this scenario would be the speedy fsck times (none in the case of ZFS).
Typo. I'll run EXT4 in journalled or writeback mode, it won't fsck. Looking at freenas if it has native ZFS support, opensolaris is mysecond choice.

Thanks for your advice.
 
Old 01-31-2011, 06:53 PM   #4
genderbender
Member
 
Registered: Jan 2005
Location: US
Distribution: Centos, Ubuntu, Solaris, Redhat
Posts: 396

Original Poster
Rep: Reputation: 31
Scrap that, no RAID 6 on freenas!
 
Old 01-31-2011, 06:57 PM   #5
genderbender
Member
 
Registered: Jan 2005
Location: US
Distribution: Centos, Ubuntu, Solaris, Redhat
Posts: 396

Original Poster
Rep: Reputation: 31
No ZFS in openfiler. Looks like it's gonna be opensolaris...
 
Old 01-31-2011, 08:32 PM   #6
jefro
Moderator
 
Registered: Mar 2008
Posts: 20,670

Rep: Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333
BSD or Solaris for zfs would be my choice. In fact might want to look at openIndiana project. Last looked pretty good. Big learning curve on it though to be useful. BSD's may be a bit easier. Don't forget to think about iSCSI target's.
 
1 members found this post helpful.
Old 01-31-2011, 09:18 PM   #7
eveningsky339
Member
 
Registered: Mar 2010
Location: Western Maine
Distribution: PCLinuxOS (LXDE)
Posts: 466

Rep: Reputation: 51
There is a FreeBSD port of ZFS, is there not? I don't personally have experience using the file system, but OpenSolaris isn't looking so good after the Oracle takeover. FreeBSD may be a safer bet.
 
1 members found this post helpful.
Old 01-31-2011, 09:44 PM   #8
genderbender
Member
 
Registered: Jan 2005
Location: US
Distribution: Centos, Ubuntu, Solaris, Redhat
Posts: 396

Original Poster
Rep: Reputation: 31
I'm not at all concerned about the future of opensolaris, oracle are continuing support for it (after lots of dispute). My biggest concern is multiplier support; it's only recently been written into the kernel, which means there's a good chance whatever RAID cards I pick won't be supported. Someone recommended nexenta to me (opensolaris with debian style commands) could be well suited given that it's slightly more user friendly than solaris (not really a concern for me but could be for new users). Still doesn't address my concerns with sata multipliers though.

I think ZFS should be my prime focus here, slightly fearful of running BSD with ZFS as my understanding is that ZFS is a propriety (i.e code owned by oracle) and thus anything but solaris will be hacks of some kind. I've read some stories about people putting ZFS fuse filesystem on top of things like red hat and hitting major performance issues.

Trying to avoid anything with a big learning curve as support for this kind of thing will be patchy, out of the guides I've read; mostly people are using debian and JFS, so at least if I run into problems I know people have a similar setup... I just think debian and JFS isn't a brilliant setup really. I'll probably end up trying lots of OSs, it's a real bity about freeNAS not having RAID 6 support, would of been absolutely ideal for my needs.

Last edited by genderbender; 01-31-2011 at 09:47 PM.
 
Old 02-01-2011, 04:10 PM   #9
jefro
Moderator
 
Registered: Mar 2008
Posts: 20,670

Rep: Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333Reputation: 3333
Try this maybe. http://openindiana.org/

As to the code of ZFS, it is not exactly free but in fact really usable. http://en.wikipedia.org/wiki/Common_...bution_License ZFS on BSD I believe to be real ZFS and not a hack at all.

Last edited by jefro; 02-01-2011 at 04:14 PM.
 
1 members found this post helpful.
Old 02-01-2011, 05:21 PM   #10
genderbender
Member
 
Registered: Jan 2005
Location: US
Distribution: Centos, Ubuntu, Solaris, Redhat
Posts: 396

Original Poster
Rep: Reputation: 31
That looks really slick, I suspect I'm going to end up trying lots of different combinations: Openfiler, centos, openindiana (or some other solaris based distro) and the most suitable freeBSD distro are all looking promising. Having RAID 6 is the most important thing, followed by usability, and then ZFS and finally extras such as iscsi.

Big thanks for everyones help, this has been a really interesting learning curve, seem like every OS has pitfalls and benefits, where as initially I thought this would be entirely based on preference.
 
Old 02-01-2011, 06:51 PM   #11
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,582

Rep: Reputation: 3504Reputation: 3504Reputation: 3504Reputation: 3504Reputation: 3504Reputation: 3504Reputation: 3504Reputation: 3504Reputation: 3504Reputation: 3504Reputation: 3504
Why the RAID6 importance ?. Have you seen the zfs demos on youtube ?. Pretty impressive.
 
Old 02-02-2011, 03:43 AM   #12
genderbender
Member
 
Registered: Jan 2005
Location: US
Distribution: Centos, Ubuntu, Solaris, Redhat
Posts: 396

Original Poster
Rep: Reputation: 31
Quote:
Originally Posted by syg00 View Post
Why the RAID6 importance ?. Have you seen the zfs demos on youtube ?. Pretty impressive.
Faster, easier to manage, maximum redundancy (like to happen given the vast number of disks).

Last edited by genderbender; 02-02-2011 at 03:45 AM.
 
Old 02-02-2011, 11:46 AM   #13
twisted_wire
LQ Newbie
 
Registered: Feb 2011
Posts: 1

Rep: Reputation: 0
What's wrong with Solaris?

Solaris 10 - not OpenSolaris is available from Oracle/Sun as a free download:

"www(dot)oracle(dot)com/technetwork/server-storage/solaris/downloads/index(dot)html"

It has a nice GUI interface or cli as you choose. RHEL works with ZFS as well, I believe.

The SunFire x4500 shipped with Solaris and ZFS I'm pretty sure...I know they had Solaris on them.
and the 48 500gig drives were part of a cluster at the installation I was at.

Personally, I'd go with Solaris, or OpenSolaris...Oracle may have bought them out but most of the people are still the same.

Good Luck!
 
Old 02-02-2011, 01:52 PM   #14
genderbender
Member
 
Registered: Jan 2005
Location: US
Distribution: Centos, Ubuntu, Solaris, Redhat
Posts: 396

Original Poster
Rep: Reputation: 31
It's the same problem with solaris and open solaris and solaris express and anything built on the solaris kernel - lack of support for sata multipliers. The sunfireX4500 had a custom built motherboard without sata multipliers. From what I understand redhat has no support for ZFS (I haven't thoroughly investigated this yet). Running ZFS on fuse apparently results in really crap/boarder line unusable performance.

I'll give some form of opensolaris a shot, the indianna thing mentioned previously looks like it may have more support for 'other' hardware so maybe worth a shot. I'm fairly competant with solaris products so I'm more than happy using it over (most) other unix/linux systems (exception being redhat, centos and possibly ubuntu).
 
Old 02-02-2011, 02:06 PM   #15
Skaperen
Senior Member
 
Registered: May 2009
Location: WV, USA
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,234
Blog Entries: 28

Rep: Reputation: 161Reputation: 161
We've been using XFS on our 40TB RAID 5 arrays with no problem. Any reason XFS is not in your choices?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: This week at LWN: Linux Storage and Filesystem Workshop, day 2 LXer Syndicated Linux News 0 04-28-2009 10:10 AM
LXer: This week at LWN: Linux Storage and Filesystem workshop, day 1 LXer Syndicated Linux News 0 04-24-2009 05:41 AM
Filesystem Type for large network storage smudge|lala Linux - General 2 01-18-2007 11:22 AM
choosing a filesystem for storage jimk Linux - General 4 06-13-2005 01:39 AM
Storage Partition Filesystem question for Dual boot Slackware and Windows XP Razza2004 Linux - General 4 03-31-2004 07:14 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 06:38 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration