LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-26-2010, 12:24 AM   #1
dritjon
LQ Newbie
 
Registered: Aug 2010
Posts: 25

Rep: Reputation: 0
Help with cluster


I need to build a cluster with vmware with 2 linux nodes but I'm a bit of a begginer.
I have this questions:

Wich vmware to use (server, workstations)?
Do I need a server OS? Wich linux OS to use?
A step-by-step how to create the cluster?
How do I test it? How do I run a program in paralel?

Please somebody help me. It's very important
 
Old 08-27-2010, 01:54 AM   #2
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello and Welcome to LinuxQuestions,

VMWare is one of the possibilities you have when you want to set up a virtualization cluster. You have other possibilities like KVM, VirtualBox, Xen and so on.

If you're just a starter and want to get going real fast, first read about virtualization so that you at least understand some of the terminology (Wikipedia Virtualization for example).

Next you can turn to a site like http://www.howtoforge.com where you'll find easy to follow step by step guides. Basically those howtos are almost copy/paste. One possible solution is this:
Xen cluster management with Ganeti on Debian Lenny.

You first install your nodes with Debian Lenny and then you follow the guide at the link above.

Hope this helps and good luck.

Kind regards,

Eric
 
Old 08-27-2010, 11:51 AM   #3
dritjon
LQ Newbie
 
Registered: Aug 2010
Posts: 25

Original Poster
Rep: Reputation: 0
That was a helpful link, but very complicated.

I just need to try paralel computin, and I thought a simple cluster could help.
Just need 2 linux nodes in a cluster and execute a software in both of them at the same time.

Can I do this?
 
Old 08-27-2010, 11:58 AM   #4
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello,

Not with the setup(s) mentioned above as far as I know. You can use the virtual machines you create over the two nodes but if you want to do what you call parallel computing that's way further than just setting up a virtual environment. I'm not sure if this is what you mean but have a look at this link:
Hadoop distributed computing

Is this what you mean? If not, then please explain a bit more in detail what you mean by parallel computing?

Kind regards,

Eric
 
Old 08-28-2010, 07:17 AM   #5
dritjon
LQ Newbie
 
Registered: Aug 2010
Posts: 25

Original Poster
Rep: Reputation: 0
Hey man, thanks again.

What I wanted to do was this: http://www.ibm.com/developerworks/li...ry/l-cluster2/

But I need a more specific help for every step, since I'm a beginner.
 
Old 08-28-2010, 07:24 AM   #6
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hi,

I just read through the site you mentioned and understand a bit better what you want. Disregard my previous post mentioning Hadoop, you don't need that.

It seems to me that the site explains what needs to be done pretty completely. Can you indicate what you don't understand? What hardware have you available for your project? Let's take it from there, start with what you have, install what you need, and off to testing

Kind regards,

Eric

Last edited by EricTRA; 08-28-2010 at 07:36 AM.
 
Old 08-28-2010, 11:04 AM   #7
dritjon
LQ Newbie
 
Registered: Aug 2010
Posts: 25

Original Poster
Rep: Reputation: 0
I have a dual core @ 1.7 GHZ, 2 giga RAM and Vista on my pc.

Because I cant find another PC, I wanted to use vmware.

Now, my problems are:
1. what vmware to use (esx, workstation) ?
2. Can I use any linux OS (ubuntu, fedora, etc) or do I need a specific linux os?
3. I dont understand the OSCAR part. Is it the next step of the tutorial, or is it an alternative of the tutorial?

Then I will try the project and let you know my problems.

Thanks 4 your time!
 
Old 08-28-2010, 12:37 PM   #8
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello,

First of all, do you want to keep your Windows and make your computer dual boot? Or do you want to install a product like VMWare in Windows and run your necessary Linux machines in that virtual environment?

If yes, then download your Linux distro of choice and install it. Most current Linux distros handle installing a dualboot system pretty well if Windows was installed first.

If no, download your Linux distro of choice, install it wiping Windows clean and condemning it to cyberspace (BEST CHOICE ).

Above is the first step/decision you need to take. Then comes the next one.

Basically you can use any Linux distro, perform a full install with GUI (Graphical User Interface) if you don't feel comfortable with the command line, install a product like VMWare Server or VirtualBox and then install your two nodes with Linux in newly created virtual machines.

That's the simplest way if you're new. If you want to jump in and get to know Linux you can use a product like Xen, running at the command line will teach you a lot. If you want to get a feeling of Xen, they have a great LiveDVD that you can start from without touching your harddisk, just to see if you like it.

As indicated on the website, the first option is for:
Quote:
As noted earlier, this is a bare-bones cluster, and much of it is based on doing manual work by making sure machines can communicate with each other (ssh is configured, MPICH is manually copied, etc.).
Of course there are ways to automate lots of it after installation but that will be a bit confusing if your new to Linux.

The second option (OSCAR) is an alternative option, where open source packages are installed to facilitate the maintenance of your cluster.
Quote:
Clearly, it will be hard to maintain the cluster above. It is not convenient to copy files to every node, set up SSH and MPI on every node that gets added, make appropriate changes when a node is removed, and so on.
Fortunately, excellent open source resources can help you set up and manage robust production clusters. OSCAR and Rocks are two examples. Most of the things we did to create our cluster are handled by these programs in an automated manner.
You can pick either, the second one will be easier to maintain because software will take care of almost everything if correctly configured. The first one will take up more of your time maintaining but in my opinion will teach you a lot in a short time.

So, now it's up to you to make some decisions on what you want to do. Looking forward to your questions.

Kind regards,

Eric
 
Old 08-28-2010, 03:05 PM   #9
dritjon
LQ Newbie
 
Registered: Aug 2010
Posts: 25

Original Poster
Rep: Reputation: 0
Again, thanks for your time!

First of all, I want to use vmware, not a dual-boot. I guess vmware esx will do just fine?

Second: I'd prefer using OSCAR, since I dont have much time ( I have a deadline ), but I have never used it.

Can you help me with this, a step-by-step?

Again, a billion thanks!
 
Old 08-28-2010, 03:27 PM   #10
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello,

Do you want to use a graphical interface or do you prefer command line?
In other words, will this computer be dedicated for your project or is it going to be used also for your daily computer needs, hence needing a GUI. And what distro would you use?


Kind regards,

Eric
 
Old 08-28-2010, 05:05 PM   #11
dritjon
LQ Newbie
 
Registered: Aug 2010
Posts: 25

Original Poster
Rep: Reputation: 0
I will use GUI.

(PS. What exactly do u mean by computer? the first node of the cluster?)

I want to use Ubuntu, or openSUSE. Can I use them?

Also, wich is better: vmware or virtual box?

Last edited by dritjon; 08-28-2010 at 05:20 PM.
 
Old 08-29-2010, 12:39 AM   #12
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello,

By computer I mean your pc. You can use Ubuntu if you like. Here's a link to an easy to follow step by step guide to install VMWare on Ubuntu:
Install VMWare Server 2.0.2 on Ubuntu 10.04.
There's even a link on how to install Ubuntu if you're not sure about some options.

I've used both VMWare (and still do at work) and VirtualBox and I liked VMWare more. But the choice is up to you. Since you're new to Linux and want to get started with your project as fast as possible, I suggest you go with VMWare and follow the guide indicated in the link above.

Of course if you have any doubt following that guide, post your questions here.

Kind regards,

Eric
 
Old 08-29-2010, 05:17 AM   #13
dritjon
LQ Newbie
 
Registered: Aug 2010
Posts: 25

Original Poster
Rep: Reputation: 0
But on my PC I have Vista.

I thought I was going to install Ubuntu as guest with vmware on a Vista host.
 
Old 08-29-2010, 05:32 AM   #14
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello,

You can do that too, but I advice against it because Vista will be the host system and is not very good at it. The performance will be a lot better if you install Ubuntu along with Vista making your pc dual-boot. Linux takes up a lot less resources then Vista. Since you're planning on installing two virtual machines with Ubuntu and then the environment you indicated, I'm afraid you'll have bad experiences if you do it all using Vista as a host system.

You can try it by using the wubi installer that will install Ubuntu 'inside' your Windows Vista installation if you want to but I'm sure the performance will be very poor once you start installing VMWare in the Ubuntu system and next two virtual machines to setup your environment. If you do it like that you'll have 3 machines running on top of Vista with only 2Gb RAM.

My advice is to make it dual boot if you want to keep Vista but again, the choice is up to you.

Kind regards,

Eric
 
Old 08-29-2010, 05:37 AM   #15
dritjon
LQ Newbie
 
Registered: Aug 2010
Posts: 25

Original Poster
Rep: Reputation: 0
So you are sayin: install Ubuntu dual-boot -> install vmware on ubuntu -> create the 2 virtual machines on vmware .

PS what vmware should I use? Does vmware workstation work for this project?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Linux Terminal Server Project Cluster Edition (LTSP-Cluster) LXer Syndicated Linux News 0 02-07-2010 03:50 AM
LXer: Cray to Resell Moab Cluster Suite from Cluster Resources LXer Syndicated Linux News 0 02-04-2008 06:10 PM
Cluster accounting failed at 135593 (0x211a9): missing cluster in $Bitmap fakie_flip Linux - Software 1 01-02-2008 03:08 AM
LXer: Cluster Programming: Explicit Implications of Cluster Computing LXer Syndicated Linux News 0 12-26-2006 08:54 PM
LXer: Hitting the Cluster Wall - A Study in Cluster Optimization LXer Syndicated Linux News 0 06-27-2006 12:33 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 08:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration