LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Virtualization and Cloud
User Name
Password
Linux - Virtualization and Cloud This forum is for the discussion of all topics relating to Linux Virtualization and Linux Cloud platforms. Xen, KVM, OpenVZ, VirtualBox, VMware, Linux-VServer and all other Linux Virtualization platforms are welcome. OpenStack, CloudStack, ownCloud, Cloud Foundry, Eucalyptus, Nimbus, OpenNebula and all other Linux Cloud platforms are welcome. Note that questions relating solely to non-Linux OS's should be asked in the General forum.

Notices

Reply
 
Search this Thread
Old 05-10-2010, 04:41 AM   #1
tarekeldeeb
LQ Newbie
 
Registered: Jun 2009
Posts: 7

Rep: Reputation: 0
Is there a PVM-Xen thing for transparent HPC?


Hello LQ community,

I have been a reader at this forum for years, but this is my first post!

I would like to ask about a thought (maybe a day-dream) I had.

I have a set of EDA binary tools (no source code), the tools need so much CPU runtime + too many RAM, the tools make use of multicore CPUs. Those tools are too old, and run under old linux distros. I use Xen virtual machines to run thos tools in. Whenever I get a better PC with more cores and RAM, I can feel the speedup. Now I want to make a much larger speed boost.

Here comes my day-dream:
I am asking about compiling Xen modules with pvm (parallel virtual machine) and installing this Xen-PVM on 10 physical PCs. 1 is PVM master, and 9 slaves. I can then deploy 1 Xen VM on the master and it runs on 10 PCs. No recoding for the tools. PVM manages the messages between Xen master/slave modules.

Now I can have my HPC without any coding.

Is this possible?

Last edited by tarekeldeeb; 05-10-2010 at 04:43 AM.
 
Old 05-11-2010, 05:05 PM   #2
tarekeldeeb
LQ Newbie
 
Registered: Jun 2009
Posts: 7

Original Poster
Rep: Reputation: 0
bump!
 
Old 05-12-2010, 10:48 AM   #3
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 91
Very interesting concept.

I'm not 100%, but I'm pretty sure that if you PVM your 10 machines - and then install RHEL on top of the PVM; then RHEL (and hence Xen) would be "unaware" of the fact that you have 10 machines underneath.

Thus imho setting up a VM would allow you to give the CPU and Memory of all 10 machines to this VM - the best way to verify I think is probably to try it out?

As of RHEL 5.4 Red Hat have discontinued the Xen kernel; so that would be the only schlep.

Maybe one of the Guru's or Snr Members have tried this...

Cheers,
Yas
 
Old 05-12-2010, 11:53 AM   #4
tarekeldeeb
LQ Newbie
 
Registered: Jun 2009
Posts: 7

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by alli_yas View Post
Very interesting concept.

I'm not 100%, but I'm pretty sure that if you PVM your 10 machines - and then install RHEL on top of the PVM; then RHEL (and hence Xen) would be "unaware" of the fact that you have 10 machines underneath.
hello yas,
what do u mean with PVM the 10 machines? compile xen with PVM? compiler kernel? Install a module?

I'm totally new to this field!

Anyways, thanks for your help.
 
Old 05-13-2010, 02:26 AM   #5
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 91
Hi there

I'm no expert with PVM - and have only read about it. From my rudimentary understanding PVM allows you to "combine" physical hardware to appear as a single machine to an OS?

So if the above holds true; you "combine" your 10 machines; which would have RHEL/CentOS or the like installed on the single "combined" machine?

Thereafter you setup a Xen VM; and allocate it a chunk of hardware available (the CPU and Memory of the 10 machines should be available?).

And you now have a Xen VM sharing hardware from 10 machines? I don't see why you can't do it - though you'd need to test it to see if it works.

Not sure about PVM, but for Xen you wouldn't need to compile the kernel or install any kernel modules - you would need to install the RHEL Xen kernel and the xen packages required for virtualization.

CHeers,
Yas
 
Old 05-13-2010, 04:45 AM   #6
tarekeldeeb
LQ Newbie
 
Registered: Jun 2009
Posts: 7

Original Poster
Rep: Reputation: 0
Hello Yas,

Yes, you got my point, I want to combine the resources.
But you did not get the challenging point.

Quote:
Originally Posted by alli_yas View Post
Hi there
I'm no expert with PVM - and have only read about it. From my rudimentary understanding PVM allows you to "combine" physical hardware to appear as a single machine to an OS?
PVM does not make magic, it provides a framework for message passing between PCs. You need to re-code your tools in a master/slave manor and compile with PVM.

This is why I see this task too complicated, as I have to split the Xen into master/slave and recompile with PVM.
If this succeeded, then I can easily install a guest virtual-machine on the master xen, and have access to all 10 PCs resources.

I am asking about a project/package that does this Xen trick for me.
 
Old 05-14-2010, 12:33 AM   #7
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 91
Hi

I kind of understand what you're trying to achieve - unfortunately my understanding of PVM is a bit limited so I won't be able to give you a definite answer.

Cheers,
yas
 
Old 10-31-2010, 12:51 PM   #8
galg
LQ Newbie
 
Registered: Oct 2010
Posts: 4

Rep: Reputation: 0
Sorry for bumpping.

Hi alli_yas,
Did you find it possible? If so, what have you done to make it work?

Thanks.
 
Old 11-01-2010, 03:45 AM   #9
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 91
Hi galg,

Unfortunately I didn't really get the time to attempt to test it. Based on discussions I've had with colleagues/friends I doubt it will work though.
 
Old 11-01-2010, 11:45 AM   #10
galg
LQ Newbie
 
Registered: Oct 2010
Posts: 4

Rep: Reputation: 0
But something is not clear to me. isn't PVM the same as MPI (Message Passing Interface) ? If it is than it's just an interface for a cluster between the nodes, it's not an interface that allows you install OS on it (i.e linux) - isn't it?
Can you give me a hand (guide or anything else) with the installation of PVM (master & slaves) and how to install the OS on it later (Should I use Xen for installing the OS?)

Thanks ;-)
 
Old 11-02-2010, 01:25 AM   #11
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 91
Hi galg,

I'm no expert with PVM and haven't used it personally - I've only read up on it a little.

With regards to setting up virtualization hosts/guests; Xen is a good tool; however if your hardware supports it, perhaps you should investigate using KVM. There are numerous guides out there on how to install KVM and setup guests; maybe give one of those a try and I'll help when you get stuck?

Good Luck
 
  


Reply

Tags
hpc, transparent, xen


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Video Tutorial: How to Setup a Virtual Supercomputer Center or HPC Cluster using Xen Teo En Ming Linux - Virtualization and Cloud 0 10-31-2009 07:05 AM
Using Xen VMs for Development & Testing of Supercomputer and HPC Cluster MPICH Apps Teo En Ming Linux - Virtualization and Cloud 0 10-30-2009 06:45 AM
Xen Transparent Bridging jamied66 Linux - Software 2 07-15-2009 03:12 PM
LXer: Managing Xen With Xen-Tools, Xen-Shell, And Argo LXer Syndicated Linux News 0 11-05-2006 12:21 PM
Microsofts new thing.........Xen! Quivver Programming 3 02-18-2004 10:33 PM


All times are GMT -5. The time now is 10:07 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration