This chapter discusses the MPI virtual topology mechanism. A virtual topology is an extra, optional attribute that one can give to an intra-communicator; virtual topologies cannot be added to inter-communicators. A virtual topology can provide a convenient naming mechanism for the MPI processes of a group (within a communicator), and additionally, may assist the runtime system in mapping the processes onto hardware.
As stated in Chapter Groups, Contexts, Communicators, and Caching, a group in MPI is an ordered set of n process identifiers (henceforth MPI processes). Each MPI process in the group is assigned a rank between 0 and n-1. In many parallel applications, a linear assignment of integer ranks to the MPI processes does not adequately reflect the logical communication pattern of the MPI processes (which is usually determined by the underlying problem geometry and the numerical algorithm used). Often the MPI processes are arranged in topological patterns such as two- or three-dimensional grids. More generally, the logical MPI process arrangement is described by a graph. In this chapter we will refer to this logical MPI process arrangement as the virtual topology.
A clear distinction must be made between the virtual topology and the topology of the underlying, physical hardware. The virtual topology can be exploited by the system in the assignment of processes to physical processors, if this helps to improve the communication performance on a given machine. How this mapping is done, however, is outside the scope of MPI. The description of the virtual topology, on the other hand, depends only on the application, and is machine-independent. The functions that are described in this chapter deal with machine-independent mapping and communication on virtual topologies.
Rationale.
Though physical mapping is not discussed, the existence of the virtual topology information may be used as advice by the runtime system. There are well-known techniques for mapping grid/torus structures to hardware topologies such as hypercubes or grids. For more complicated graph structures good heuristics often yield nearly optimal results [51]. On the other hand, if there is no way for the user to specify the logical process arrangement as a virtual topology, a random mapping is most likely to result. On some machines, this will lead to unnecessary contention in the interconnection network. Some details about predicted and measured performance improvements that result from good process-to-processor mapping on wormhole-routing architectures can be found in [13,14].
Besides possible performance benefits, the virtual topology can
function as a convenient, process-naming structure, with
significant
benefits for program readability and notational power in
message-passing programming.
( End of rationale.)