Best distro to implement a High Performance Computing Cluster
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Best distro to implement a High Performance Computing Cluster
Hi, guys!
I'm new in this forum and in Linux universe. I'm trying to implement a HPC Cluster with 5 computers (Intel Core 2) in my lab at university to start GA development and I've been searching for information, but I couldn't reach any conclusion.
Has anybody heard of CHAOS (Clustered High Availability Operating System), a Linux distribution produced within the Livermore Computing center at the Lawrence Livermore National Laboratory? Is it available for downloading? To find more about it: https://computing.llnl.gov/linux/projects.html
I've also seen several Red Hat distro posts for Cluster implementation and I got in contact with some researchers who use Slackware distro for the same. I've also got a book where the writer uses CentOS. What I'm trying to say is: depending on the source, one particular and different distro cames up.
So, I'm here in this forum to try to understand what's the best distro to HPC Clusters or, at least, gain some knowledge to understand the specific differences among the distros I mentioned above regarding HPC Clusters. Is there somewhere a article or study comparing distros? Regarding clusters, is there any known study comparing distros and their perfomance?
In my opinion one can do everything with every distribution. Most people will take the distribution which they are most familiar with. I'd recommend to chose a distribution with an active and helpful community. http://www.linuxquestions.org/questions/slackware-14/ http://www.slackware.org
Take a look at Gentoo as well.
As a new Linux user, I would have thought a (pre-)packaged solution would be a better fit. A (very) quick google turned up some candidates - like this. The gentoo project referenced has some good info, but might be jumping in the "deep end" if you have no previous gentoo install experience. The doco looks a couple of years old as well. Also note the references/links to OSCAR and Rocks in the lwn article.
If you had to have absolute performance, gentoo would be worth considering with a bit of time/experience.
Redhat would be more interested in high availability clusters than high performance I would think, although I've never looked into it. Centos is merely the commercial Redhat repackaged as freeware - seems CHAOS was of similar lineage, but according to the Wikipedia article CHAOS may have died.
Many thanks, markush, for your point of view, and syg00, for your advices.
I think Wikipedia's Chaos is a different one, while it's from Midnight Code (here) and the other is from LLNL (here), what makes me more confused :-P
Well, I'm start again from your points of view. Now, I need to know the essential differences between distros. I mean, why so many distros if the core system is the same? Is there some real difference that could make my application less effective? Or do I need only to run proper applications?
the big difference is what syg00 stated. One part of the distributions is highly preconfigured and almost everything will work out of the box, e.g. network, sound etc.
The other part of the distributions is less configured, one has to do much configuration by himself. Normaly I'd suggest that a linux-newbie gets started with one of the preconfigured distributions. But in your case it will be necessary to do much configuration on the system by yourself.
Slackware and Gentoo belong to the distributions which have to be manually configured. Slackware is very easy to install, the installation of Gentoo is more difficult (for a newbie).
But I'd recommend: just try it out and make your own experience.
Go to http://distrowatch.com and you can search for the parameters "clusters" and "high performance". They list 10 distros, with descriptions, so you can take it from there.
Many thanks, markush, for your point of view, and syg00, for your advices.
I think Wikipedia's Chaos is a different one, while it's from Midnight Code (here) and the other is from LLNL (here), what makes me more confused :-P
Well, I'm start again from your points of view. Now, I need to know the essential differences between distros. I mean, why so many distros if the core system is the same? Is there some real difference that could make my application less effective? Or do I need only to run proper applications?
Thanks for all!
I don't think you're going to find too many differences at the distro level. IMHO, don't worry about the distro - focus on your application requirements. Do you already have a single threaded application that you're now trying to parallelize? If so, how would you like to accomplish that? Is it by running your monolithic application concurrently on different data sets or are you trying to break apart your single thread into concurrent tasks working on one set of data? These should lead you in a better direction than worrying about whether to choose Red Hat or Slackware...
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.