This chapter discusses the MPI topology mechanism. A topology is an extra, optional attribute that one can give to an intra-communicator; topologies cannot be added to inter-communicators. A topology can provide a convenient naming mechanism for the processes of a group (within a communicator), and additionally, may assist the runtime system in mapping the processes onto hardware.
As stated in Chapter Groups, Contexts, Communicators, and Caching , a process group in MPI is a collection of n processes. Each process in the group is assigned a rank between 0 and n-1. In many parallel applications a linear ranking of processes does not adequately reflect the logical communication pattern of the processes (which is usually determined by the underlying problem geometry and the numerical algorithm used). Often the processes are arranged in topological patterns such as two- or three-dimensional grids. More generally, the logical process arrangement is described by a graph. In this chapter we will refer to this logical process arrangement as the ``virtual topology.''
A clear distinction must be made between the virtual process topology and the
topology of the underlying, physical hardware. The virtual topology can be
exploited by the system in the assignment of processes to physical processors,
if this helps to improve the communication performance on a given machine. How
this mapping is done, however, is outside the scope of MPI. The description
of the virtual topology, on the other hand, depends only on the application,
and is machine-independent.
The functions that are described in this chapter
deal only with machine-independent mapping.
Rationale.
Though physical mapping is not discussed, the
existence of the virtual topology information may be used as advice by
the runtime system.
There are well-known techniques for mapping grid/torus structures to
hardware topologies such as hypercubes or grids. For more
complicated graph structures good heuristics often yield nearly optimal
results [32]. On the other hand,
if there is no way for the user to specify the logical process
arrangement as a ``virtual topology,'' a random mapping is most
likely to result. On some machines, this will lead to
unnecessary contention in the interconnection network.
Some details about predicted and measured
performance improvements that result from good process-to-processor
mapping on modern wormhole-routing architectures can be found in
[10,11].
Besides possible performance benefits, the virtual topology can
function as a convenient, process-naming structure, with
significant
benefits for program readability and notational power in
message-passing programming.
( End of rationale.)