LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 04-23-2008, 10:05 PM   #1
Swakoo
Member
 
Registered: Apr 2005
Distribution: Red Hat / Fedora / CentOS
Posts: 508

Rep: Reputation: 30
Solution for storage purposes


Hi guys,

I am planning for a storage solution for a media company, for them to store large amount of their magazine work on it. Estimated required space per year is almost 3TB.

I'm thinking is it even feasible to use linux as a base operating system with huge-ass amount of data space behind it...

just thinking wildly now.. not sure abt the actual implementation.. hence I thought i ask you guys, to see if there's any suggestions

thanks!
 
Old 04-24-2008, 03:35 PM   #2
keratos
Member
 
Registered: May 2007
Location: London, UK.
Distribution: Major:FC8. Others:Debian;Zenwalk; Arch; Slack; RHEL.
Posts: 544

Rep: Reputation: 30
is it backing store or online store you need?

for online store access I would recommend SCSI for speed but 3TB could be cost prohibitive

for backing store, why not go for a rack case with USB2 HDDs chained mapped under LVM ... supported natively in latest kernels/distros.
 
Old 04-24-2008, 03:42 PM   #3
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
Or you could try RAID with regular SATA drivers, which are cheaper than SCSI ones. But, do not use RAID0, it increases the probability of data loss. Use RAID1 and up.
 
Old 04-25-2008, 12:07 AM   #4
raskin
Senior Member
 
Registered: Sep 2005
Location: Russia
Distribution: NixOS (http://nixos.org)
Posts: 1,893

Rep: Reputation: 68
And maybe you should somehow implement a daemon to periodically read all data - so corrupted sectors get restored using RAID redundancy before some more data about the same data block is lost.
 
Old 04-25-2008, 08:13 AM   #5
jakev383
QmailToaster Developer
 
Registered: Dec 2005
Location: Burlington, NC
Distribution: CentOS, Voyage, Debian, Fedora
Posts: 220

Rep: Reputation: 31
Quote:
Originally Posted by Swakoo View Post
Hi guys,

I am planning for a storage solution for a media company, for them to store large amount of their magazine work on it. Estimated required space per year is almost 3TB.

I'm thinking is it even feasible to use linux as a base operating system with huge-ass amount of data space behind it...

just thinking wildly now.. not sure abt the actual implementation.. hence I thought i ask you guys, to see if there's any suggestions

thanks!
I have a 2TB backup server here, using IDE drives. I would use RAID5, but I don't know what your requirements are. RAID5 will give you redundancy (if 1 drive fails, your data is not lost as in LVM) and fast reads, but writing is slower since it would be writing to all the drives.
What will your accesses be like? As keratos asked, is it for backup purposes, or will the users be accessing data from it? (honestly, either way I would probably use RAID5 - I hate to lose data, and a mirroring setup takes too many drives IMHO). If a lot of the users (20+) will be mounting the drives via NFS then I would use either a BSD distro or a Solaris distro as those seem to be a little better at handling NFS that Linux (speed wise).
Not sure what the limitations are as far as how many TB's the kernel/file system will handle, but if you go extremely large you may want to look at things like iSCSI and fibre channel. Grab a Linux magazine and look at the ads in there for the NAS solutions. Most of the larger companies I do work for end up using a NAS box of one form or another.
 
Old 04-25-2008, 05:26 PM   #6
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 199Reputation: 199
What's your budget and what are the access requirements? Are users needing to access such data? Are you planning this as a large file server? What type of files are we talking about? I'd imagine magazine as in articles and a lot of text right? If that's the case, you'll probably have a lot of duplication involved. You may want to look into deduplication products. This requires less physical drive space that crams a lot of data by de-duping it, etc. Other than that and if you want the generic option, a SAN or NAS is the only option if you know you'll be growing at 3TB a year. Also with this amount of data, I hope you have a backup plan in place.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Large storage solution question Millenniumman Linux - Hardware 8 09-02-2007 10:12 AM
Virtualusers for emailing purposes Swakoo Fedora 4 11-21-2006 02:23 AM
Sun NFS/SMB storage solution monz Solaris / OpenSolaris 3 04-22-2006 02:08 PM
Best Linux for...security purposes? :P blackjack90 Linux - Newbie 2 02-01-2006 10:25 AM
Which Distro Would Suit My Purposes Best? johnleemk Linux - Newbie 5 10-02-2003 05:57 AM


All times are GMT -5. The time now is 06:41 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration