4.9. Persistent Communication Requests

Often a communication with the same argument list (with the exception of the buffer contents) is repeatedly executed within the inner loop of a parallel computation. In such a situation, it may be possible to optimize the communication by binding the list of communication arguments to a persistent communication request once and then repeatedly using the request to start and complete operations.In the case of point-to-point communication, the persistent communication request thus created can be thought of as a communication port or a ``half-channel.'' It does not provide the full functionality of a conventional channel, since there is no binding of the send port to the receive port. This construct allows reduction of the overhead for communication between the MPI process and communication controller, but not of the overhead for communication between one communication controller and another. It is not necessary that messages sent with a persistent point-to-point request be received by a receive operation using a persistent point-to-point request, or vice versa.

There are also persistent collective communication operations defined in Section Persistent Collective Operations and Section Persistent Neighborhood Communication on Process Topologies. The remainder of this section covers the point-to-point persistent initialization operations and the start routines, which are used for persistent point-to-point, partitioned point-to-point, and persistent collective communication operations.

A point-to-point persistent communication request is created using one of the five following calls. These point-to-point persistent initialization calls involve no communication.

Creates a persistent communication request for a standard mode send operation.

Creates a persistent communication request for a buffered mode send operation.

Creates a persistent communication request for a synchronous mode send operation.

Creates a persistent communication request for a ready mode send operation.

Creates a persistent communication request for a receive operation. The argument buf is marked as OUT because the user gives permission to write on the receive buffer by passing the argument to MPI_RECV_INIT.

A persistent communication request is inactive after it was created---no active communication is attached to the request.

A communication that uses a persistent communication request is started by the function MPI_START.

The argument, request, is a handle returned by any of the initialization procedures for persistent point-to-point communication (the previous five procedures), or for partitioned point-to-point communication (see Section Partitioned Point-to-Point Communication), or for persistent collective communication (see Sections Persistent Collective Operations and Persistent Neighborhood Communication on Process Topologies). The associated request should be inactive. The request becomes active once the call is made.

If the request is for a ready mode send operation, then a matching receive operation should be started before the call is made. The communication buffer should not be modified after the call, and until the operation completes.

The call is local, with similar semantics to the nonblocking communication operations described in Section Nonblocking Communication. That is, a call to MPI_START with a request created by MPI_SEND_INIT starts a communication in the same manner as a call to MPI_ISEND; a call to MPI_START with a request created by MPI_BSEND_INIT starts a communication in the same manner as a call to MPI_IBSEND; and so on.

The execution of MPI_STARTALL has the same effect as the execution of MPI_START for each of the array elements in some arbitrary order. MPI_STARTALL with an array of length one is equivalent to MPI_START.

A communication started with a call to MPI_START or MPI_STARTALL is completed by a call to MPI_WAIT, MPI_TEST, or one of the derived functions described in Section Multiple Completions. The request becomes inactive after successful completion of such call. The request is not deallocated and it can be activated anew by an MPI_START or MPI_STARTALL call.

A persistent communication request is deallocated by a call to MPI_REQUEST_FREE (Section Communication Completion). The call to MPI_REQUEST_FREE can occur at any point in the program after the persistent request was created. However, the request will be deallocated only after it becomes inactive. Active receive requests should not be freed. Otherwise, it will not be possible to check that the receive has completed. Collective operation requests (defined in Section Nonblocking Collective Operations and Section Nonblocking Neighborhood Communication on Process Topologies for nonblocking collective operations, and Section Persistent Collective Operations and Section Persistent Neighborhood Communication on Process Topologies for persistent collective operations) must not be freed while active. It is preferable, in general, to free requests when they are inactive. If this rule is followed, then the functions described in this section will be invoked in a sequence of the form,

where * indicates zero or more repetitions. If the same persistent communication request is used in several concurrent threads, it is the user's responsibility to coordinate calls so that the correct sequence is obeyed.

Inactive persistent requests are not automatically freed when the associated communicator is disconnected (via MPI_COMM_DISCONNECT, see Releasing Connections) or the associated World Model or Sessions Model is finalized (via MPI_FNIALIZE, see Finalizing MPI, or MPI_SESSION_FINALIZE, see Session Creation and Destruction Methods). In these situations, any further use of the request handle is erroneous. In particular, freeing associated inactive request handles after such a communicator disconnect or finalization is then impossible.

Advice to users.

Persistent request handles may bind internal resources such as MPI buffers in shared memory for providing efficient communication. Therefore, it is highly recommended to explicitly free inactive request handles, using MPI_REQUEST_FREE, when they are no longer in use, and in particular before freeing or disconnecting the associated communicator with MPI_COMM_FREE or MPI_COMM_DISCONNECT or finalizing the associated session with MPI_SESSION_FINALIZE. ( End of advice to users.)
A send operation started with MPI_START can be matched with any receive operation and, likewise, a receive operation started with MPI_START can receive messages generated by any send operation.

Advice to users.

To prevent problems with the argument copying and register optimization done by Fortran compilers, please note the hints in Sections Problems With Fortran Bindings for MPI--Comparison with C. ( End of advice to users.)