LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 06-03-2007, 04:03 AM   #1
Tabemasu
LQ Newbie
 
Registered: Jun 2007
Posts: 1

Rep: Reputation: 0
Linux Cluster/Server configuration


Hi

I am working for a research group that requires to do a large amount of data analysis (each of these files could be up to 1 gig in size). We're planning on buying up to 10 pc with linux on them to do these scientific processing (matlab, etc ) but what would be the best server configuration ? Prefereably, we would like to maintain all the data on one server and send it out to the linux box for processing automatically. Would a beuwolf cluster be a good approach or does anyone have a better suggestion ? The worst case scenario is that we drop the file manually into these linux box and use remote desktop to run them.

Thanks, Dave
 
Old 06-03-2007, 09:40 PM   #2
PatrickNew
Senior Member
 
Registered: Jan 2006
Location: Charleston, SC, USA
Distribution: Debian, Gentoo, Ubuntu, RHEL
Posts: 1,148
Blog Entries: 1

Rep: Reputation: 48
I don't think beowulf is the way to go, that is more form trying to take advantage of SMP, which matlab probably doesn't. I'd suggest just hooking the computers together in whatever home-network style you want. From there you have two options - set up an ftp server on one computer and have it serve the files to the other computers or set up an NFS (or samba) share with the files on it and allow the other computers to mount that. With files around 1 gig I would recommend the ftp server, but the choice is up to you.

Advantages of the ftp:
+less network load
+cannot corrupt work if two computers simultaneously open the same file

Advantages of the nfs:
+slightly simpler to use
+provides your changes back to your colleagues faster.

Any self-respecting linux distro has an ftp server in its repository or on the install CD. Likewise, any decent distro can do NFS for you too.
 
Old 06-05-2007, 02:03 AM   #3
p_s_shah
Member
 
Registered: Mar 2005
Location: India
Distribution: RHEL 3/4, Solaris 8/9/10, Fedora 4/8, Redhat Linux 9
Posts: 237
Blog Entries: 1

Rep: Reputation: 34
Check http://openmosix.sourceforge.net. It mounts remote filesystem as a local drive, provides faster access, distributes load between multiple systems.
Try to implement and check.
Tell us if you find any problem.
 
Old 06-11-2007, 11:19 AM   #4
brianmcgee
Member
 
Registered: Jun 2007
Location: Munich, Germany
Distribution: RHEL, CentOS, Fedora, SLES (...)
Posts: 399

Rep: Reputation: 40
Maybe a NFS diskless sharedroot cluster [1] could be useful for your setting. A NFS server provides a common root filesystem for the diskless clients. They boot the exact same operating system and form a single system image on filesystem basis. All results that are stored on the filesystem are instantly available to all cluster nodes. The read/write access is competively for all nodes. New nodes can be added very easily because they boot the same root filesystem. The configuration of the whole cluster is like a single linux box.

[1] http://www.open-sharedroot.org/docum...oot-mini-howto
 
Old 06-11-2007, 11:42 PM   #5
simplyA
LQ Newbie
 
Registered: Apr 2007
Location: Iran
Distribution: CentOS, SUSE, Debian, RedHat
Posts: 14

Rep: Reputation: 0
Hi,

I would strongly recommend you drbd:
http://www.drbd.org/

simplyA
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Penguin Computing's Scyld ClusterWare(TM) HPC Offers Enhanced High Availability and Most Advanced Linux Cluster Virtualization, Enabling Broader Cluster Use LXer Syndicated Linux News 0 04-06-2006 06:03 PM
Removing survivable cluster configuration Almost Lost Linux - Enterprise 1 12-27-2005 05:46 PM
Linux server cluster lovelysandu Linux - Networking 0 06-02-2004 05:46 AM
New To linux Want help with server configuration Sideshow Linux - Networking 2 02-12-2004 04:04 PM
NIC Configuration in Linux Server shyam Linux - Hardware 1 08-20-2003 05:46 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 10:49 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration