LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 11-16-2009, 10:17 AM   #1
luigi
LQ Newbie
 
Registered: Aug 2001
Location: Italy
Distribution: kubuntu
Posts: 27

Rep: Reputation: 15
Cool Beowulf-like cluster


Hi all,
I would like to build a small beowulf like cluster with few machines that I have here around from the pc-farm. The problem is that up to now I have only been able to find docs and links to old/unupdated/dead projects...

Do you have some hints? I slighly prefere debian-based systems, but any (reasonably recent....) distro would be ok.

Thank you!
Luigi


PS: I hope this is the correct forum for such question...
 
Old 11-16-2009, 07:39 PM   #2
MS3FGX
Guru
 
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 351Reputation: 351Reputation: 351Reputation: 351
Well, "cluster" is a pretty general term. What exactly would you like to do with it? You need to have some kind of application in mind before you get started to figure out what sort of software you are looking at.

Beowulf is an advanced system for mathematical applications, but one could argue it is no more or less a cluster than two Debian boxes passing jobs between each other via SSH.
 
Old 11-17-2009, 03:50 AM   #3
luigi
LQ Newbie
 
Registered: Aug 2001
Location: Italy
Distribution: kubuntu
Posts: 27

Original Poster
Rep: Reputation: 15
Hi MS3FGX,
I have a bunch of number-crunching applications that need to crunch few Tbytes of data in a "embarassing parallel" way (to say it in HPC jargon).

I did already the ssh trick between two machines but, as you know, there are lots of "little problems" like disk sharing, handling a machine/job failure, etc etc that must be handled and I would like not to reinvent the wheel...
So before extending it to more machines I would really be happy to try some beowulf clustering, i.e. the same game in pratice, but much simpler stuff for the end users.

What do you think? do you have some pointers to active projects?

Thanks!
Luigi

Last edited by luigi; 11-17-2009 at 03:52 AM.
 
Old 11-17-2009, 02:01 PM   #4
MS3FGX
Guru
 
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 351Reputation: 351Reputation: 351Reputation: 351
So I assume you are going to be writing the software for this cluster then?

In that case, you might want to look at OSCAR. I haven't used it personally, but I have heard good things. Basically is a complete toolkit to create and maintain Beowulf-style clusters on common Linux distributions. It handles all the mundane details that you mentioned like inter-node communication and leaves you free to just write up your code.

As I understand it, OSCAR can even create disk images with your code and configuration that are suitable for booting your nodes.
 
Old 11-18-2009, 07:15 AM   #5
luigi
LQ Newbie
 
Registered: Aug 2001
Location: Italy
Distribution: kubuntu
Posts: 27

Original Poster
Rep: Reputation: 15
Hi MS3FGX,
thank you for the link
I have just tried OSCAR on a VirtualBox image with ubuntu 9.04, but I am not able to have it running: is it possible that OSCAR supports only ubuntu up to 8.04, debian up to 4, etc and not newer distros?
Any hint to have it running on a newer system?

Regards!
Luigi
 
Old 11-18-2009, 08:30 AM   #6
LinuxLover
Member
 
Registered: Feb 2004
Distribution: Solaris 10/11 , RHEL 6 ,AIX 7.1
Posts: 185

Rep: Reputation: 32
Look at the video of Beowolf cluster

http://www.wbitt.com/hpcc-supercomputing-cbts.html
 
  


Reply

Tags
beowulf, cluster, hpc


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Beowulf cluster suggestions poonippi Linux - Software 3 07-18-2008 10:08 PM
cluster beowulf grened23 Linux - Networking 0 09-11-2006 08:13 AM
Beowulf cluster questions etmiserie Linux - General 3 06-29-2004 09:07 PM
RH Beowulf Cluster and SETI kajensen Red Hat 1 09-06-2003 07:23 AM
Beowulf Cluster or Network CCParrish Linux - General 3 12-24-2000 10:22 AM


All times are GMT -5. The time now is 02:37 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration