LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices

Reply
 
Search this Thread
Old 11-16-2004, 10:31 AM   #1
nutthick
Member
 
Registered: Jun 2004
Distribution: Slack
Posts: 214

Rep: Reputation: 30
4Tb Server


I've been asked to look into the possibility of hosting a 4Tb server for dishing out files through a website. I'm not too keen on a Windows box, due to the licensing limitations, so was thinking about a linux box. Does anyone have any advice, or experience of doing this?

Last edited by nutthick; 11-16-2004 at 11:03 AM.
 
Old 11-16-2004, 11:06 AM   #2
hkb33
Member
 
Registered: Sep 2004
Location: Raleigh NC
Distribution: Fedora / RHEL
Posts: 171

Rep: Reputation: 30
Be careful of the block size an filesystem limits...some distributions (like Red Hat) will only support a block size of 1TB...therefore a filesystem maximum of 1TB. 2TB is possible, but it might be pushing it. Don't try to use all 4Tb on one big filesystem unless you want to expect problems.
 
Old 11-16-2004, 11:08 AM   #3
rjlee
Senior Member
 
Registered: Jul 2004
Distribution: Ubuntu 7.04
Posts: 1,990

Rep: Reputation: 67
I've never built a server on that scale, but most distros will run the apache web server out of the box, so getting ~4Tb's worth of files online isn't much harder than getting them all in one directory structure.

For a that size, I'd have one partition for all of the files [EDIT: given the previous post, check that the distro will support this first. I think most newer kernels will, but it may depend on the filesystem.], and mount it as something like /usr/share/httpdocs, and point your documentroot in httpd.conf to that address. If you're going to be saving files to the web-partition a lot, then use a journeling filesystem (I suggest reiserfs), otherwise don't (use ext2) as you'll use the disk space more efficiently.

If you want to stream over protocols other than HTTP, then Apache won't help you; you'll need to look at a more appropriate server (try searching www.freshmeat.net)

You might also want to consider using software RAID; distros like SuSE will set that up out of the box (in an advanced install), and you can gain speed and/or reliability savings by trading off disk space. RAID (or LVM) will also let you combine multiple partitions into one big one. Hardware RAID is an option, but check the distro's HCL (hardware compatibility list) to make sure that it will actually work (I say that from experience!).

Hardware: Get a box with one processor, and don't go for the fastest you can find. The disk or network speed will probably be the limiting factor. 64-bits won't help you here (unless you want to move the entire 4Tb of data into one tmpfs (in-memory) partition, in which case if you want more than 4Tb of RAM, so you'll get a speed benefit from the increased native address space).

Last edited by rjlee; 11-16-2004 at 11:09 AM.
 
Old 11-16-2004, 11:21 AM   #4
nutthick
Member
 
Registered: Jun 2004
Distribution: Slack
Posts: 214

Original Poster
Rep: Reputation: 30
The files to be moved will be 30-300Mb in size. Demand has yet to be established, I think this is more of somebody's pipe dream, rather than something that is needed, but if the money man wants it... It will be about 5 years worth of work going onto the server initially, with more added as it's created. The idea is to produce two identical 4Tb systems, one purely as a backup (seems the only way of backing up something that size cheaply).
 
Old 03-18-2005, 11:50 AM   #5
EdR
LQ Newbie
 
Registered: Sep 2004
Distribution: RHEL AS3, AS4 / Vector
Posts: 22

Rep: Reputation: 15
Filesystem limits

Can someone offer some more clarification on the filesystem size limits?

I am running RH ES3.0 and as stated in this thread, the filesystem size is 1TB. I want to use a Direct Attached Storage unit that is 2TB and expandable to 4.8 TB. It would be using a Fiber Channel or scsi interface. I have previously used a Network Storage device and it didn't have the size limitations but unfortunately it doesn't offer the throughput that I need.

My question is - Does the filesystem size limit of 1TB = Total disk space( server disks + DAS) or can I split the DAS into multiple 1TB volumes in addition to my disk space on my server?
 
Old 03-18-2005, 03:09 PM   #6
Electro
Guru
 
Registered: Jan 2002
Posts: 6,042

Rep: Reputation: Disabled
I suggest XFS instead of ReiserFS if you are going to store files with a size of 30 to 300 MB. XFS comes in real handy if using LVM or EVMS because it can handle million terabytes of space when using a 64-bit processor and kernel 2.6.x with big block support. An Opteron system will work well for now and any future upgrades.

I use XFS on my 120 GB hard drives and it has very high data throughput.
 
Old 03-29-2005, 01:19 AM   #7
nutthick
Member
 
Registered: Jun 2004
Distribution: Slack
Posts: 214

Original Poster
Rep: Reputation: 30
I had to go with an XServe system in the end. The files turned out to be some strange customised format that only an Apple server could understand. Netatalk and Windows Appletalk just got totally confused.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How the DNS-server is connected to work of a web-server and a mail-server? ukrainet Linux - Newbie 2 01-10-2005 09:18 PM
Relay Server<-> GatewayServer<->filtering server: PostFix, amavis,spamassassin, cyrus admore Linux - General 0 11-02-2004 08:46 AM
can we configure a Linux server with mail server,file server and web server kumarx Linux - Newbie 5 09-09-2004 06:21 AM
Unable to access my ssh server and ftp server from the Internet, but smtp works foxone Linux - Networking 1 05-28-2004 05:17 PM
mkfs and mounting large (1.4TB) hardware raid array. Frustin Linux - General 1 05-20-2002 06:04 AM


All times are GMT -5. The time now is 07:56 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration