Hi
Currently our cluster is based on AMD opteron processors. We are going to add more nodes however I want to know is it good to add Intel xeon processors. They both claim to be x86 compatible however I don't know if they are really math each other.

Does a program compiled on Intel xeon really works on opteron (amd64)? any feedback is appreciated.

This is not the definitive answer, but I believe that AMD pays license fees to Intel so that AMD can legally make chips that understand the x86 instruction set -- which was created by Intel. Whether the systems are 'compatible' is too broad a question.

There a variety of facts that may be helpful:
* you can't just pop out an Intel and put an AMD chip in its place in your motherboard (or vice versa). AMD and Intel have different pin configurations and require different motherboard chipsets.
* you couldn't just take a hard drive with linux or windows installed on it from an Intel machine and pop it into an AMD machine and expect the OS to run (or vice versa). The configuration of files on the hard drive (drivers, etc.) would not be set up properly for the other machine.
* can they both run windows? yep. linux? yep. OSX? Yep, with some tweaking. Apache? yep. IIS? Yep. AFAIK, any software you can run on an Intel chip will also run on an AMD chip. I believe the executables are even identical -- although 64-bit apps won't run on a 32-bit OS.
* can they all live on the same network? Yep, as long as their software is compatible.

You need to ask more specific questions.

Hi,

If a piece of software is compiled for a specific processor it will not (or badly) run on a different kind of processor. In general software is compiled to run on AMD and Intel (assuming both are for instance x86 compatible).

If you have any doubt you should ask the developers of the software you are using. The cluster software itself should be ok if you use out-of-the-box clustering software.

Hope this helps.

This should work. The only problem I see here it the initrd in Linux: it has the wrong drivers included. But this is not a matter of the used processor but of the used chipset of the mainboard. Once you adjusted it there shouldn't be an issue.

+1

If you compile software on your own: some compilers default to optimize for the one on which they are executed. E.g. with Portland compiler I got an Illegal Instruction when executing an application compiled on a recent AMD but executed then on a former Opteron. There are of course switches to tell the compiler which instruction set to use, or make it a plain x86_64 w/o any optimization.

Are you using any queuing system in your cluster? E.g. in GridEngine you get the variable $ARC at runtime to tell the job on which platform your are. Now assume you have from one and the same application two binaries: one optimized for Intel and one for AMD. When you set up identical directory structures for both, you can have a shared /opt/em64t/my_app_foober and /opt/amd64/my_app_foobar.

When now in the job script /opt/$ARC/my_app_foobar is executed, you always get the correct binary for the machine your job was scheduled to.

So there is no problem having a mixture of machines. NB: parallel job should be set up to get only machines from one type though.

Yes I am using Torque/Maui and my concern is that, the job manager submit a job on a node that is optimized for another node. For example I compiled and optimized my code for opteron but the job manager submit it to an Intel node.

Also in Torque/Maui you can setup certain features of a machine and request this resource. So like:Or the admin defines two queues and you have to submit to the correct one. The admin just has to setup things accordingly.

Are you using x86 (32 bit) architecture, or are you using x86_64 (aka AMD64, aka em64t) architecture?

"xeon" is a marketing term with no technical meaning. The "xeon" name covers a very wide range of very different Intel CPU's, so saying "xeon" tells us nothing useful about which Intel CPU you mean.

Within the x86 (32 bit) architecture, GCC can compile for a specific model in a way that makes code that would not work on older models. I don't know whether any of those options would make code that doesn't work on an amd64 capable chip.

I suspect you are safe: any amd64 capable chip running x86 (32 bit) code supports a superset of the common optional features of x86. I'm just not certain of that.

Within x86_64 architecture, you are even safer. I don't think GCC has even optional support for any feature that exists in one x86_64 model and doesn't exist in another x86_64 model.

SneakyImp: Actually, Intel traded with AMD for the x86_64 design after the Itanium chip wasn't as successful as Intel had hoped.

I think that you are making a problem, but it is related to a problem that might exist.

Essentially, all of the x86 series of chips has capabilities (and AMD and Intel are not capabilities), and, over time, the capabilities have tended to increase. All of the chips have, at least in some mode or another, the capabilities of the earliest chips, but they have added more capabilities in the more recent chips. So code compiled for a plain vanilla 32 bit x86 will run on a more advanced x86, it just won't take advantage of the newer software features.

Now the newer features don't really make that much of a difference to speed except in certain exceptional cases, so, if you compile for a sufficiently old version of an x86 that only has capabilities that all of the target chips have, the code will run, irrespective of whether the CPU doing the running is AMD or Intel.

You didn't have to do that. Given that if you hadn't done that, but followed the idea of optimising for a 'lowest common denominator' chip there wouldn't be a problem, only you can decide whether creating this problem was actually worthwhile. (Note that if your Intel chips have a superset of the selected Opteron's capabilities, you haven't created a problem, as should be obvious.)

@jschiwal
Err, not quite (pedantry alert): it was actually the instruction set and not the chip design. Essentially this means the 'what' of what the chip does, as seen by a software engineer, rather than the 'how' of how it goes about it.

In any case, the situation with the 64 bit instruction set is (currently) a lot friendlier than the 32 bit version; when AMD came out with the 64 bit version they did a lot of clean up and tidy up work, and so, at the time that it came out, there was only one single instruction set (as opposed to an instruction set, plus half a dozen, or more, significant optional add-ons) worth supporting, and that applies to chips from both manufacturers.

One possible exception comes to mind; Intel has been adding AES-acceleration instructions to its latest chips, and those instructions do make an appreciable difference to the speed of AES operations. This is a very specialised field, however, and I have no idea of the state of support in GCC for these (and the odds are quite strong that you probably aren't doing anything AES-related), so it is probably irrelevant to you.

Thanks all
That is great. I will check that