LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 03-28-2012, 06:16 AM   #1
markyd
LQ Newbie
 
Registered: Dec 2011
Location: Shepperton - UK
Distribution: Lubuntu & Puppy ... mostly ;-)
Posts: 25
Blog Entries: 16

Rep: Reputation: Disabled
High Performance Computing - Discussion


I have been reading a lot about High Performance Clustering (HPC) but it appears only to be relevant to application specifically designed to take advantage of it, is there a distro that can treat each node in a cluster as a seperate process or processor so tha any application can use the cluster? Or am I being a bit to fanciful here?

regards .... MarkyD
 
Old 03-28-2012, 07:59 PM   #2
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,921

Rep: Reputation: 3618Reputation: 3618Reputation: 3618Reputation: 3618Reputation: 3618Reputation: 3618Reputation: 3618Reputation: 3618Reputation: 3618Reputation: 3618Reputation: 3618
Well, yes but you are in such a wide field we can't say for sure what you want or need.

We use hundreds of clusters each day for a specialized industrial task. A master node passes a task to nodes as they become free. The node performs a task and sends results back to the master node. In our use the nodes are 8-way standalone servers so we don't send to a core as such but to a nfs node. The software on each node uses the entire server for it's task generally.


Other common HPC setup's so similar things but they break up a single large task into smaller parts then send them to a node. This differs from our use which is thousands of tasks to be worked.

Sun company had a neat trick where they were able to install a client application to almost any workstation anywhere in the world. The idea was some big company could use the power of every workstation they owned to do processing. Each node could be set for amount of task or times or such to allow the normal user to continue to work.

For the most part, you can't easily use an HPC. You can set them up in a virtual machine or a small cluster of systems in your home if you wish but to use it for some work is difficult unless you are using some scientific program or very technical program that was made to run in a HPC engine.

Common cluster distros may be Rocks, Cluster knoppix. I think something like openmosix (I forget). There are a few others. Some of the enterprise level distro's offer a choice.

Last edited by jefro; 03-28-2012 at 08:00 PM.
 
Old 03-29-2012, 01:01 AM   #3
markyd
LQ Newbie
 
Registered: Dec 2011
Location: Shepperton - UK
Distribution: Lubuntu & Puppy ... mostly ;-)
Posts: 25

Original Poster
Blog Entries: 16

Rep: Reputation: Disabled
Thanks for taking the time to reply, its appreciated and kind of confirmed what I thought ... you have to admit though it would be cool and useful to make the power of the cluster available to the a normal distro front end.

What fueled this was when I was thinking about virtualisation, normal technologies distribution VM's to idividual nodes but what I was thinking is having a mutinode cluster running virtual box and then VM's woulod then be spread accross the cluster. Thus adding harware to the cluster to improver performance and capacity would be seemless.

Did think it was a bit ambitious :-)

MarkyD
 
Old 03-30-2012, 12:33 PM   #4
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 15.2
Posts: 1,339

Rep: Reputation: 260Reputation: 260Reputation: 260
There are some flavors of clusters for HPC: one is to submit jobs with a batch queuing system to route tasks to a node in the cluster (like ROCKS), but you can also set it up by hand. Even this could be parallelized if your software supports it to use more than one node for each job by PVM or MPI. What you are looking for sounds like a single system image which was offered by OpenMOSIX, but IIRC each process was still bound to a node and could only use local memory. It’s now LinuxPMI. There is Kerrighed to offer a single system image across a bunch of machines too.
 
Old 02-08-2013, 12:24 AM   #5
422vishal
LQ Newbie
 
Registered: Feb 2013
Posts: 4

Rep: Reputation: Disabled
I'm a final year diploma student in cse and i'm trying to make and implement an infrastructure which would serve as file server, web server, mail server, news server, database server on old systems on linux-ubuntu based platform.I researched a quite bit and then came to know about this Kerrighed cluster. Would it be of any use and fisible if i build a Kerrighed cluster and deploy these services on them. Any i/p would be helpful for the project.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Best distro to implement a High Performance Computing Cluster iperetta Linux - Newbie 6 04-08-2011 01:53 PM
High Performance Computing chj0848 Linux - Newbie 2 09-28-2010 11:39 PM
high performance computing cluster - child nodes speed test m2azer Linux - Networking 1 06-23-2009 03:56 AM
New User and Linux High Performance computing environment venki_nyn LinuxQuestions.org Member Intro 3 02-26-2009 08:27 AM
High Performance Computing humbletech99 Linux - General 9 11-13-2008 01:29 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 03:17 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration