Is there a PVM-Xen thing for transparent HPC?
 

Hello LQ community,

I have been a reader at this forum for years, but this is my first post!

I would like to ask about a thought (maybe a day-dream) I had.

I have a set of EDA binary tools (no source code), the tools need so much CPU runtime + too many RAM, the tools make use of multicore CPUs. Those tools are too old, and run under old linux distros. I use Xen virtual machines to run thos tools in. Whenever I get a better PC with more cores and RAM, I can feel the speedup. Now I want to make a much larger speed boost.

Here comes my day-dream:
I am asking about compiling Xen modules with pvm (parallel virtual machine) and installing this Xen-PVM on 10 physical PCs. 1 is PVM master, and 9 slaves. I can then deploy 1 Xen VM on the master and it runs on 10 PCs. No recoding for the tools. PVM manages the messages between Xen master/slave modules.

Now I can have my HPC without any coding.

Is this possible?

bump!

Very interesting concept.

I'm not 100%, but I'm pretty sure that if you PVM your 10 machines - and then install RHEL on top of the PVM; then RHEL (and hence Xen) would be "unaware" of the fact that you have 10 machines underneath.

Thus imho setting up a VM would allow you to give the CPU and Memory of all 10 machines to this VM - the best way to verify I think is probably to try it out?

As of RHEL 5.4 Red Hat have discontinued the Xen kernel; so that would be the only schlep.

Maybe one of the Guru's or Snr Members have tried this...

Cheers,
Yas

hello yas,
what do u mean with PVM the 10 machines? compile xen with PVM? compiler kernel? Install a module?

I'm totally new to this field!

Anyways, thanks for your help.

Hi there

I'm no expert with PVM - and have only read about it. From my rudimentary understanding PVM allows you to "combine" physical hardware to appear as a single machine to an OS?

So if the above holds true; you "combine" your 10 machines; which would have RHEL/CentOS or the like installed on the single "combined" machine?

Thereafter you setup a Xen VM; and allocate it a chunk of hardware available (the CPU and Memory of the 10 machines should be available?).

And you now have a Xen VM sharing hardware from 10 machines? I don't see why you can't do it - though you'd need to test it to see if it works.

Not sure about PVM, but for Xen you wouldn't need to compile the kernel or install any kernel modules - you would need to install the RHEL Xen kernel and the xen packages required for virtualization.

CHeers,
Yas

Hello Yas,

Yes, you got my point, I want to combine the resources.
But you did not get the challenging point.

PVM does not make magic, it provides a framework for message passing between PCs. You need to re-code your tools in a master/slave manor and compile with PVM.

This is why I see this task too complicated, as I have to split the Xen into master/slave and recompile with PVM.
If this succeeded, then I can easily install a guest virtual-machine on the master xen, and have access to all 10 PCs resources.

I am asking about a project/package that does this Xen trick for me.

Hi

I kind of understand what you're trying to achieve - unfortunately my understanding of PVM is a bit limited so I won't be able to give you a definite answer.

Cheers,
yas

Sorry for bumpping.

Hi alli_yas,
Did you find it possible? If so, what have you done to make it work?

Thanks.

Hi galg,

Unfortunately I didn't really get the time to attempt to test it. Based on discussions I've had with colleagues/friends I doubt it will work though.

But something is not clear to me. isn't PVM the same as MPI (Message Passing Interface) ? If it is than it's just an interface for a cluster between the nodes, it's not an interface that allows you install OS on it (i.e linux) - isn't it?
Can you give me a hand (guide or anything else) with the installation of PVM (master & slaves) and how to install the OS on it later (Should I use Xen for installing the OS?)

Thanks ;-)

Hi galg,

I'm no expert with PVM and haven't used it personally - I've only read up on it a little.

With regards to setting up virtualization hosts/guests; Xen is a good tool; however if your hardware supports it, perhaps you should investigate using KVM. There are numerous guides out there on how to install KVM and setup guests; maybe give one of those a try and I'll help when you get stuck?

Good Luck :)