LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 05-31-2006, 09:08 PM   #1
tormentum
LQ Newbie
 
Registered: May 2003
Location: Perth, Australia
Distribution: Slackware, Ubuntu
Posts: 18

Rep: Reputation: 0
Scalable, cheap disk storage? (Hypothetical, interested in ideas)


At the moment this is completley hypothetical and i'm just interested in your ideas and feedback, and also expanding my knowledge on the subject:

I'm wondering how you would go about creating a scalable and cheap (ie: cheaper than a SAN or massive SCSI array) array of SATA disks. Currently I have a box at home with 7 x 320GB SATA disks arranged in a RAID-5 array, giving about 1.9TB formatted. However, to resize an array of this size is headache inducing to say the least (thank god for UPS's!)

I'm interested in a solution that involves several machines and is scaleable, so I can add more nodes at a later date. It should be some sort of storage cluster, as it were. It doesn't necessarily have to actually be one big disk array internally, but from the outside (Samba access), it should appear as one "disk/folder". My thoughts on the matter is having several "slave nodes" which are much like my current system and simply have a large capacity RAID array on them, and then having a "master node" that can merge all the array's into one logical "drive", or use the storage provided by these arrays, and use some sort of filesystem (such as MogleFS to store the files, while using Samba to serve them up across the network.

What are your ideas? If I need to clarify my "requirements", let me know

Torm
 
Old 06-01-2006, 02:36 PM   #2
farslayer
Guru
 
Registered: Oct 2005
Location: Willoughby, Ohio
Distribution: linuxdebian
Posts: 7,228
Blog Entries: 5

Rep: Reputation: 189Reputation: 189
I would use a hardware RAID contrroller that allows hot swapping and array expansion on the fly such as the 3ware controllers.. add drive,issue expand array command, walk away... job is done.


otherwise I suppose a NAS solution using something like NASLite or freeNAS would work with multiple machines..
 
Old 06-08-2006, 02:12 PM   #3
jantman
Member
 
Registered: Nov 2005
Location: New Jersey, USA
Distribution: SuSE
Posts: 492

Rep: Reputation: 31
Aside from the jealosy of 1.9 Tb....

Personally, the idea of having something that appears to be one "folder" at 1.9 Tb scares the sh** out of me. It seems like a point for too much failure, and too much misuse if it's anything other than a one-user system.

IMHO, anything that combines that much space into one logical "place" is just adding unneeded code, lag time, and complexity. If they're 320Gb drives, why not just have hda1, hda2, hda3, etc. in the old-fashioned way, mount them to folders with meaningful names, and call it a day?

On the useful side of advice, if you're storing that much data, you need a strong, reliable, fault-tolerant file system. I would recommend against ext2, or any other Linux default. though they may be journalling, they're not the best out there in terms of crash-worthiness.

I'd suggest OpenSolaris or one of the BSD systems (FreeBSD or NetBSD).

There's also a distro called FreeNAS which is a NAS package based on FreeBSD, and has a strong filesystem.
 
Old 06-08-2006, 10:00 PM   #4
madluther
Member
 
Registered: Aug 2004
Distribution: LFS
Posts: 350

Rep: Reputation: 30
Sounds like LVM2 at http://evms.sf.net is what you're looking for, it features scalable logical volumes, resizable partitions, snap copies, bad block relocation, partition table types that support > 2TB partitions, most of the modern file systems and a lot more. It works well with HA-clustering and as an iSCSI target.

HTH

Mad.
 
Old 06-09-2006, 02:34 AM   #5
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,986
Blog Entries: 11

Rep: Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880Reputation: 880
Quote:
Originally Posted by jantman
IMHO, anything that combines that much space into one logical "place" is just adding unneeded code, lag time, and complexity. If they're 320Gb drives, why not just have hda1, hda2, hda3, etc. in the old-fashioned way, mount them to folders with meaningful names, and call it a day?
Because those won't give him the a) kick and b) redundancy
that a RAID-5 system does. Striping the data over several disks
IMPROVES performance, it doesn't deteriorate it (as a rule of
thumb, anyway. Happy to discuss the differences it makes to
e.g. an Oracle installation to have more disks laid out differently,
but I wouldn't want to miss out on redundancy in either way.)

Another good reason for wanting such large space is e.g. for
movie-remastering in a larger scale :}



Cheers,
Tink
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Disk Storage (accross network) podollb Linux - Software 2 01-20-2006 12:31 AM
Disk Storage Advice Needed vdi_nenna Linux - Hardware 1 06-26-2003 07:42 PM
Building a cheap network-attached storage system Travis86 Linux - Networking 5 03-14-2003 12:42 PM
alternate paths to disk storage da_kidd_er Linux - Hardware 0 02-06-2003 10:18 AM
Config disk storage limit vwhk Linux - General 0 01-04-2002 02:09 AM


All times are GMT -5. The time now is 05:25 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration