LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 03-09-2016, 11:47 AM   #1
andrew_cz
LQ Newbie
 
Registered: May 2006
Distribution: Cygwin
Posts: 10

Rep: Reputation: 0
Understanding ROCKS clusters


I understand that installing and running a ROCKS cluster involves running it on several computers. Once install on every computer, it combines the cpu cores of those computers so that you can run programs faster using all the cores available...a supercomputer.

Am I understanding this correctly so far?

My real question is, can I run any software that I can compile and have it use all those cores? Or does the software need to be written especially to support clusters, or more specifically, rocks clusters?
 
Old 03-11-2016, 10:13 AM   #2
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,665
Blog Entries: 4

Rep: Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945
Yes, to a certain extent, any application intended for use in a high-performance computing cluster must be designed with the architecture of clustering firmly in mind. (And, as usual, the documentation doesn't say too much about that, because the people who use such things the most are steeped in that mode of thinking, and with that kind of problems.)

The task, itself, must be such that it can meaningfully be distributed among a large number of high-performance computers connected by a relatively slower network infrastructure, such that adding more computing hardware to the problem does enable the overall problem to be solved more quickly; to "scale up" without fundamentally changing the software itself.

One quintessential example of this is weather forecasting. Many forecast models subdivide the sky into regions, performing identical calculations in each region, and rolling-up the data from smaller regions into larger ones, such that the essential nature of the work is both that each compute-node is able to operate independently and the overall process "scales up" naturally. Of course, this is because the forecast-modeling algorithm was designed for clustering.

Other examples include ##CLASSIFIED##, ##CLASSIFIED##, and of course (let us surely not forget ...), ##CLASSIFIED##.

The controller for a compute-node is, of course, designed to make maximum possible computational use of the computing resources ("cores") available on that particular node, and to insulate the compute-node applications from dependencies as to just exactly what this-or-that node physically looks like. The workload dispatcher must dispatch a unit of work to a cluster-node that it knows is capable of supporting it; otherwise, the node "blooms where it is planted."

The node itself must also be sure that the necessary data is available where it needs to be, and that results are correctly retrieved and correctly handled, and that no compute-node is ever starved (or drowned) by data. This, too, is an application design issue that must be considered from the ground up.

If the application is not designed for clustering, then yes, you might be able to run it on a cluster, but the cluster would not really be a cluster to you. It would be nothing more than a batch-job runner.

Last edited by sundialsvcs; 03-11-2016 at 10:17 AM.
 
Old 03-15-2016, 01:27 PM   #3
andrew_cz
LQ Newbie
 
Registered: May 2006
Distribution: Cygwin
Posts: 10

Original Poster
Rep: Reputation: 0
Thank you for the wonderful explanation. I guess there goes that project out the window :-(
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Formation of Clusters in NS 2 surya.prakash Programming 5 05-27-2015 04:12 AM
[SOLVED] Need some help in learning clusters. RaviTezu Linux - Newbie 5 01-08-2013 08:46 AM
Linux HA / Clusters? gdizzle Linux - Enterprise 1 10-24-2012 04:14 PM
LXer: Rocks clusters make sense for educational environments LXer Syndicated Linux News 0 08-19-2008 04:00 AM
clusters edman007 Linux - Software 4 09-28-2005 08:52 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 10:14 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration