LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-14-2013, 04:49 AM   #1
Axelvid
LQ Newbie
 
Registered: Nov 2013
Posts: 2

Rep: Reputation: Disabled
Building a Rock Cluster for FDS6 parallel computation


Hello

I am building a cluster for Fire dynamics simulator (FDS6) parallel computation. I have access to about 20 Dell Precision (dual core) computers and have about USD $5000 budget for my cluster.

- What kind of cluster would you recommend (I have heard about Rock cluster)?
- What kind of processors (computers) would you recommend?
- Do i need to have equally powerful slave processors?
- Does the master processor have to be more powerful than the slave processors?


Thanks in advance

Best regards
Axel
 
Old 11-14-2013, 11:27 PM   #2
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,225

Rep: Reputation: 2521Reputation: 2521Reputation: 2521Reputation: 2521Reputation: 2521Reputation: 2521Reputation: 2521Reputation: 2521Reputation: 2521Reputation: 2521Reputation: 2521
You might want to post tthis question in only ONE forum

Your OTHER identical post in "Linux - Server" would be a better place
 
1 members found this post helpful.
Old 11-16-2013, 01:11 AM   #3
btmiller
Senior Member
 
Registered: May 2004
Location: In the DC 'burbs
Distribution: Arch, Scientific Linux, Debian, Ubuntu
Posts: 4,284

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
Since the other thread has been closed ... here goes my attempt at an answer. Note that I have no experience with FDS6, but I have quite a bit of experience building and running beowulf clusters.

- ROCKS is supposedly an easy way to build a Beowulf cluster with pre-rolled scripts. If the style of the cluster you want to build and your management style synergizes well with the "ROCKS" way of doing things, it will probably work well. If not, you'll find it a real pain. Note that there are other choices, e.g.Perceus and Warewulf). Personally, I just like to build all the nodes from scratch myself (using Kickstart + Puppet with Scientific Linux). It's really not as hard as some people lead you to believe, and you have the advantage of being able to know and tune how everything is set up. If you're even a semi-experienced Linux admin, setting up a basic beowulf cluster without using pre-rolled scripts is really not that difficult.

- We don't know your application. You should benchmark it against several commercially available processors and see what gives you the best computations per dollar range. You might also try asking other users of FDS6 or its manufacturer (if commercial) or developers (if open source) what they recommend. Chances are that they will know better than random people on a message board who have never heard of the application :-). Also think about whether you will be running computations in parallel over the network (e.g. using MPI). If so, network performance may matter way more than CPU performance. Questions to ask in that case are whether the code is bandwidth-sensitive or latency-sensitive or, worst of all, both. If it's latency sensitive, you may never get good inter-node parallel performance without investing lots and lots of money into specialized low latency network hardware (special purpose Ethernet or InfiniBand). Also what kind of I/O ingest/output rates are required. If FDS6 reads/writes massive data files, the fastest processors and networks in the world won't help you if the code is constantly stuck performing I/O to slow as molasses disk systems. Building an efficient and cost-effective HPC systems is all about finding what your workload needs, doing as well as possible at meeting those needs, and skimping as much as possible on other stuff to save money. Just asking "what's the best processor?" is a bit like "what kind of engine do I need in my car?" It depends on what kind of car you want to build, what gas mileage is required, how fast does it needs to go, etc.

- If you plan to run parallel calculations accross nodes, they should be matched in speed (since faster processors do no good when combined with slower ones in a parallel calculation, generally). There are some exceptions to this rule, but they're few and far between. As a rule of thumb, you want things as homogenous as possible. If you will not be running parallel computations between multiple nodes, then this is less critical, but still homogeneity will help ease the management and administration burden (only having one type of memory means you can keep you can keep a few spares on hand when [not if] a DIMM fails).

- Not necessarily. It depends on which tasks are going to be running on the master node. If it will be running lots of set-up jobs, serving a file system to the cluster via NFS, and running a batch scheduler for the rest of the cluster, then you want it to be fairly beefy. If it's just handling user logins while the rest of the cluster does the "real work", then not so much. In some of the clusters I've built, the master node is about the slowest in the whole operation.

Other things to consider:

- ECC RAM is good if you can afford it. I've seen too many crap numerical results courtesy of passing cosmic rays.
- Think about how much memory each node will need.
- Think about your power and cooling requirements if you plan to put the cluster in a confined space (e.g. a network closet). Over heating can and does cause component failure. Also, consider the cost of the electricity to run the thing...
- It's best not to cheap out if at all possible; low quality components have higher failure rates.
- If your code will run well on GPUs, consider buying a few of these rather than a big cluster. You might get just as good performance for much less money.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
parallel computation linux cluster management tool ztdep Linux - Networking 4 03-14-2013 07:25 AM
Setting up a cluster with old computers (for computation) Feynman Linux - Networking 13 10-11-2011 07:34 AM
Rock Cluster storage er.surendersharma Linux - Server 3 06-09-2010 10:42 AM
How to add application to Rock Cluster kumar.manoj412 Linux - Server 0 06-07-2010 10:15 AM
Rock cluster vs Conga on Centos kumar.manoj412 Linux - Server 2 06-01-2010 09:36 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration