Open MPI

Message Passing Interface (MPI) is a standard application programming interface (API) for efficient interprocess communication. It is widely used in many existing research codes. It can be used to write parallel programs which span multiple computers or which reside on a single multiprocessor.

Open MPI

The default (and preferred) MPI implementation on our clusters is Open MPI. It's free, open source, production-quality MPI-2 implementation.

Note: Do not confuse the Open MPI library with the OpenMP interface.

Documentation

Open MPI web site

Frequently Asked Questions

Instructional videos and presentations

How it's spelled

It's two words (not one) — Open MPI. It's also frequently abbreviated "OMPI" (pronounced "oom-pee").

Modulefile

openmpi

Versions

Default is 1.2.9 built with PGI. More recent versions also available.

Benefits of Open MPI

• Supports all available types of interconnections (Ethernet, Myrinet, InfiniBand, on-host shared memory, etc.).
• The fastest available interconnection will be used by default.
• Better performance, especially on large shared-memory nodes.
• No need to recompile your application on every cluster.
• Better integration with Sun Grid Engine.
• Better integration with parallel debuggers (see TotalView).
• User friendly, easy to use.

Commands

You can use the following standard commands:

 mpif90, mpif77, mpicc, mpiCC (or mpicxx or mpic++), mpirun

You can also browse man pages for the main Open MPI commands, for example:

 $ man mpif90
 $ man mpicc

Also you can get description and syntax for many MPI functions, for example:

 $ man MPI_Init
 $ man MPI_Send

Running production jobs

Below is an example of a submission script for an MPI job. There is no need to specify the list of hosts and/or the number of processes for the mpirun command because Open MPI will obtain this information directly from Grid Engine. You should use the ompi* parallel environment for Open MPI jobs.

  #$ -cwd
  #$ -pe ompi* 4
  #$ -l h_rt=01:00:00

  mpirun my_parallel_application

Running interactive jobs

Main page: Interactive Jobs

Below is the short reference on how to run MPI job interactively. For a complete description, please read Interactive Jobs.

Interactive session through Grid Engine:

$ qrsh -cwd -l h_rt=00:10:00,test=true -pe "ompi*" 4 mpirun myapplication

If you prefer to get an interactive login shell and then run the job manually, you just need to specify the name of your favorite shell as a job name, like so:

$ qrsh -cwd -pe "ompi*" 8 -l h_rt=00:30:00 bash

Alternative builds of Open MPI

Some users may want to use alternative builds of Open MPI to compile their applications. We provide several versions of Open MPI built with PGI, Intel and GCC compilers. In order to use them, you should load a specific modulefile. All Open MPI modulefiles are mutually exclusive. In order to minimize user's confusion and prevent version conflict, we suggest users to load an Open MPI modulefile interactively before compiling an application and then again in the submission script to run it.

Further reading

• Parallel Jobs
• Interactive Jobs
• Test Jobs
• Memory Management
• Tips on writing Job Scripts