MPI includes a variant of the reduce operations where the result is returned to all MPI processes in a group. MPI requires that all MPI processes from the same group participating in these operations receive identical results.
| MPI_ALLREDUCE(sendbuf, recvbuf, count, datatype, op, comm) | |
| IN sendbuf | starting address of send buffer (choice) |
| OUT recvbuf | starting address of receive buffer (choice) |
| IN count | number of elements in send buffer (non-negative integer) |
| IN datatype | datatype of elements of send buffer (handle) |
| IN op | operation (handle) |
| IN comm | communicator (handle) |
If comm is an intra-communicator, MPI_ALLREDUCE behaves the same as MPI_REDUCE except that the result appears in the receive buffer of all the group members.
Advice
to implementors.
The all-reduce operations can be implemented as a reduce, followed by a
broadcast. However, a direct implementation can lead to better performance.
( End of advice to implementors.)
The ``in place'' option for intra-communicators is specified by passing the
value MPI_IN_PLACE to the argument sendbuf
at all processes.
value MPI_IN_PLACE to the argument sendbuf
at all MPI processes.
In this case,
the input data is taken at each MPI process from the receive buffer,
where it will be replaced by the output data.
If comm is an inter-communicator, then the result of the reduction of the data provided by MPI processes in group A is stored at each MPI process in group B, and vice versa. Both groups should provide count and datatype arguments that specify the same type signature.
The following example uses an intra-communicator.
Example
A routine that computes
the product of a vector and an array that are distributed across a
group of MPI processes and returns the answer at all nodes (see also
Example Predefined Reduction Operations).
