Although there are no
sane ways to avoid having to compile separate MPI and non-MPI versions of the program, I did find
a way for OpenMPI version 1.4.
For my own sanity, the following example only includes support for the functions MPI_Init(), MPI_Finalize(), MPI_Comm_rank(), MPI_Comm_size(), and constants MPI_SUCCESS and MPI_COMM_WORLD; others can be added with a bit of work, based on the OpenMPI header files.
Code:
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
/* <openmpi/mpi.h> replacement headers, definitions, and dummy replacements.
*/
#define MPI_SUCCESS 0
typedef void *MPI_Comm;
static int dummy_mpi_init(int *argc_ptr, char ***argv_ptr) { return MPI_SUCCESS; }
static int dummy_mpi_finalize(void) { return MPI_SUCCESS; }
static int dummy_mpi_comm_rank(MPI_Comm c, int *ptr) { *ptr = 0; return MPI_SUCCESS; }
static int dummy_mpi_comm_size(MPI_Comm c, int *ptr) { *ptr = 0; return MPI_SUCCESS; }
MPI_Comm MPI_COMM_WORLD = NULL;
int (*MPI_Init)(int *, char ***) = dummy_mpi_init;
int (*MPI_Finalize)(void) = dummy_mpi_finalize;
int (*MPI_Comm_rank)(MPI_Comm, int *) = dummy_mpi_comm_rank;
int (*MPI_Comm_size)(MPI_Comm, int *) = dummy_mpi_comm_size;
/* Load OpenMPI if available, and OMPI_COMM_WORLD_SIZE is set
* (only mpirun/orterun should set that environment variable).
* Returns 1 if it thinks real OpenMPI is available.
*/
int load_openmpi(void)
{
if (!getenv("OMPI_COMM_WORLD_SIZE"))
return 0;
void *const libmpi = dlopen("libmpi.so.0", RTLD_NOW | RTLD_GLOBAL | RTLD_DEEPBIND);
void *const librte = dlopen("libopen-rte.so.0", RTLD_NOW | RTLD_GLOBAL | RTLD_DEEPBIND);
void *const libpal = dlopen("libopen-pal.so.0", RTLD_NOW | RTLD_GLOBAL | RTLD_DEEPBIND);
if (!libmpi || !librte || !libpal) {
if (libpal) dlclose(libpal);
if (librte) dlclose(librte);
if (libmpi) dlclose(libmpi);
return 0;
}
MPI_COMM_WORLD = dlsym(libmpi, "ompi_mpi_comm_world");
MPI_Init = dlsym(libmpi, "MPI_Init");
MPI_Finalize = dlsym(libmpi, "MPI_Finalize");
MPI_Comm_rank = dlsym(libmpi, "MPI_Comm_rank");
MPI_Comm_size = dlsym(libmpi, "MPI_Comm_size");
return 1;
}
int main(int argc, char **argv)
{
int result, process, processes;
load_openmpi();
result = MPI_Init(&argc, &argv);
if (result != MPI_SUCCESS) {
fprintf(stderr, "MPI_Init() failed (error %d).\n", result);
return 1;
}
result = MPI_Comm_rank(MPI_COMM_WORLD, &process);
if (result != MPI_SUCCESS) {
MPI_Finalize();
fprintf(stderr, "MPI_Comm_rank() failed (error %d).\n", result);
return 1;
}
result = MPI_Comm_size(MPI_COMM_WORLD, &processes);
if (result != MPI_SUCCESS) {
MPI_Finalize();
fprintf(stderr, "MPI_Comm_size() failed (error %d).\n", result);
return 1;
}
if (processes > 0)
printf("MPI process %d of %d\n", process + 1, processes);
else
printf("Non-MPI process\n");
fflush(stdout);
MPI_Finalize();
return 0;
}
Compile the above sources using gcc, not mpicc. For example, if the above is in
test-mpi.c use
Code:
gcc -Wall -O3 -fomit-frame-pointer -ftree-vectorize -ldl -o test-mpi test-mpi.c
When the program is run, the variable
processes will be 0 if MPI is not available, and positive (number of parallel processes) if MPI is available.
Because the MPI_ functions above are actually function pointers, there is a small overhead to calling them, compared to normal OpenMPI programs. It is only measurable if you call these functions several hundred times a second. The function pointers default to safe dummy functions, so that you can always call them, whether MPI is available or not.
The
load_openmpi() function will try to dynamically load the three libraries and look up the actual symbols, if and only if the environment variable OMPI_COMM_WORLD_SIZE is set. Hopefully mpirun/orterun will keep setting that.
The dummy functions are written based on the MPI spec, and are therefore not a problem. The dynamic library file names, internal types, and symbol names are very specific to OpenMPI (version 1.4 here), and are a bit of a pain: they have to be parsed from the OpenMPI header files down to the binary implementation level. For example, note how the
MPI_Comm type is actually just a pointer, and
MPI_COMM_WORLD "constant" is actually a pointer to
ompi_mpi_comm_world symbol. On the other hand, this level must stay unchanged in future versions of the library too, otherwise all other previously compiled programs using OpenMPI will break too. (While compatibility issues can be fixed for other programs by recompiling against the new version of OpenMPI, such changes need to be addressed in the source code for programs like my example above -- not so trivial.)
The reason I posted this code is that a very similar mechanism can be used for dynamically loading plugins or other shared libraries. For plugins, you scan the plugin library, and dlopen() each one using the path and not just file name. If you make sure the binary level interface does not change, this should work quite well. It's just MPI that makes all of this difficult.