Up: MPI and Threads
Next: Clarifications
Previous: MPI and Threads
In a thread-compliant implementation, an MPI process is a process
that may be multi-threaded.
Each thread can issue MPI calls; however, threads are not
separately addressable: a rank in a send or receive call identifies a
process, not a thread. A message sent to a process can be received by
any thread in this process.
Rationale.
This model corresponds to the POSIX model of interprocess
communication: the fact that a process is multi-threaded, rather than
single-threaded, does not affect the external interface of this
process.
MPI implementations where MPI `processes' are POSIX threads
inside a single POSIX process are not thread-compliant by this
definition (indeed, their ``processes'' are single-threaded).
( End of rationale.)
Advice to users.
It is the user's responsibility to prevent races when threads within the
same application post conflicting communication calls. The user can
make sure that two threads in the same process will not issue
conflicting communication calls by using distinct communicators at each
thread.
( End of advice to users.)
The two main requirements for a thread-compliant implementation are listed
below.
1. All MPI calls are thread-safe,
i.e.,
two concurrently
running threads may make MPI calls and the outcome will be as if the
calls executed in some order, even if their execution is interleaved.
2. Blocking MPI calls will block the calling thread only, allowing
another thread to execute, if available.
The calling thread will be blocked until the
event on which it is waiting occurs. Once the blocked communication is
enabled and can proceed, then the call will complete and the thread
will be marked runnable, within a finite time.
A blocked thread will not prevent progress of other runnable threads
on the same process, and will not prevent them from executing MPI
calls.
Example
Process 0 consists of two threads. The first thread
executes a blocking send call MPI_Send(buff1, count, type,
0, 0, comm), whereas the second thread executes a blocking receive
call MPI_Recv(buff2, count, type, 0, 0, comm, &status),
i.e.,
the first thread sends a message that is
received by the second thread. This communication should always
succeed. According to the first requirement, the execution will
correspond to some interleaving of the two calls. According to the
second requirement, a call can only block the calling thread and
cannot prevent progress of the other thread. If the send call went
ahead of the receive call, then the sending thread may block, but this
will not prevent the receiving thread from executing. Thus, the
receive call
will occur. Once both calls occur, the communication is enabled
and both calls will complete. On the other hand, a
single-threaded process that posts a send, followed by a matching
receive, may deadlock. The progress requirement for multithreaded
implementations is stronger, as a blocked call cannot prevent progress
in other threads.
Advice
to implementors.
MPI calls can be made thread-safe by executing only one at a time,
e.g., by protecting MPI code with one process-global lock. However,
blocked
operations cannot hold the lock, as this would prevent progress of
other threads in the process. The
lock is held only for the duration of an atomic, locally-completing
suboperation such as posting a send or completing a send, and is released
in between.
Finer locks can provide more concurrency, at the expense of higher
locking overheads.
Concurrency can also be achieved by having some of the MPI protocol
executed by separate server threads.
( End of advice to implementors.)
Up: MPI and Threads
Next: Clarifications
Previous: MPI and Threads
Return to MPI-2.2 Standard Index
Return to MPI Forum Home Page
(Unofficial) MPI-2.2 of September 4, 2009
HTML Generated on September 10, 2009