373. Permanent Data Movement

A Fortran compiler may implement permanent data movement during the execution of a Fortran program. This would require that pointers to such data are appropriately updated. An implementation with automatic garbage collection is one use case. Such permanent data movement is in conflict with MPI in several areas:

• MPI datatype handles with absolute addresses in combination with MPI_BOTTOM.
• All nonblocking MPI operations if the internally used pointers to the buffers are not updated by the Fortran runtime, or if within an MPI process, the data movement is executed in parallel with the MPI operation.

Example Using separated variables for overlapping communication and computation to allow the protection of nonblocking communication with the ASYNCHRONOUS attribute.

USE mpi_f08  
REAL :: b(0:101)     ! elements 0 and 101 are halo cells 
REAL :: bnew(0:101)  ! elements 1 and 100 are newly computed 
INTEGER :: i 
CALL separated_sections(b(0), b(1:100), b(101), bnew(0:101)) 
i=1 ! compute leftmost element  
  bnew(i) = function(b(i-1), b(i), b(i+1))  
i=100 ! compute rightmost element  
  bnew(i) = function(b(i-1), b(i), b(i+1))  
END 
 
SUBROUTINE separated_sections(b_lefthalo, b_inner, b_righthalo, bnew)  
USE mpi_f08  
REAL, ASYNCHRONOUS :: b_lefthalo(0:0), b_inner(1:100), b_righthalo(101:101) 
REAL :: bnew(0:101)  ! elements 1 and 100 are newly computed 
TYPE(MPI_Request) :: req(4) 
INTEGER :: left, right, i 
CALL MPI_Cart_shift(...,left,right,...) 
CALL MPI_Irecv(b_lefthalo (  0), ..., left,  ..., req(1), ...) 
CALL MPI_Irecv(b_righthalo(101), ..., right, ..., req(2), ...) 
! b_lefthalo and b_righthalo is written asynchronously. 
! There is no other concurrent access to b_lefthalo and b_righthalo.  
CALL MPI_Isend(b_inner(  1),     ..., left,  ..., req(3), ...) 
CALL MPI_Isend(b_inner(100),     ..., right, ..., req(4), ...) 
   
DO i=2,99  ! compute  only elements for which halo data is not needed 
  bnew(i) = function(b_inner(i-1), b_inner(i), b_inner(i+1))  
  ! b_inner is read and sent at the same time. 
  ! This is allowed based on the rules for ASYNCHRONOUS. 
END DO 
CALL MPI_Waitall(4,req,...) 
END SUBROUTINE