Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I am planning for a storage solution for a media company, for them to store large amount of their magazine work on it. Estimated required space per year is almost 3TB.
I'm thinking is it even feasible to use linux as a base operating system with huge-ass amount of data space behind it...
just thinking wildly now.. not sure abt the actual implementation.. hence I thought i ask you guys, to see if there's any suggestions
Or you could try RAID with regular SATA drivers, which are cheaper than SCSI ones. But, do not use RAID0, it increases the probability of data loss. Use RAID1 and up.
Distribution: approximately NixOS (http://nixos.org)
Posts: 1,900
Rep:
And maybe you should somehow implement a daemon to periodically read all data - so corrupted sectors get restored using RAID redundancy before some more data about the same data block is lost.
I am planning for a storage solution for a media company, for them to store large amount of their magazine work on it. Estimated required space per year is almost 3TB.
I'm thinking is it even feasible to use linux as a base operating system with huge-ass amount of data space behind it...
just thinking wildly now.. not sure abt the actual implementation.. hence I thought i ask you guys, to see if there's any suggestions
thanks!
I have a 2TB backup server here, using IDE drives. I would use RAID5, but I don't know what your requirements are. RAID5 will give you redundancy (if 1 drive fails, your data is not lost as in LVM) and fast reads, but writing is slower since it would be writing to all the drives.
What will your accesses be like? As keratos asked, is it for backup purposes, or will the users be accessing data from it? (honestly, either way I would probably use RAID5 - I hate to lose data, and a mirroring setup takes too many drives IMHO). If a lot of the users (20+) will be mounting the drives via NFS then I would use either a BSD distro or a Solaris distro as those seem to be a little better at handling NFS that Linux (speed wise).
Not sure what the limitations are as far as how many TB's the kernel/file system will handle, but if you go extremely large you may want to look at things like iSCSI and fibre channel. Grab a Linux magazine and look at the ads in there for the NAS solutions. Most of the larger companies I do work for end up using a NAS box of one form or another.
What's your budget and what are the access requirements? Are users needing to access such data? Are you planning this as a large file server? What type of files are we talking about? I'd imagine magazine as in articles and a lot of text right? If that's the case, you'll probably have a lot of duplication involved. You may want to look into deduplication products. This requires less physical drive space that crams a lot of data by de-duping it, etc. Other than that and if you want the generic option, a SAN or NAS is the only option if you know you'll be growing at 3TB a year. Also with this amount of data, I hope you have a backup plan in place.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.