Coarray Fortran (CAF) is an extension of Fortran language and offers a simple interface for parallel processing and memory sharing. The advantage is that only small changes are required to convert existing Fortran code to support a robust and potentially efficient parallelism.
A CAF program is interpreted as if it was replicated a number of times and all copies were executed asynchronously. The number of copies is decided at execution time. Each copy (called image) has its own private variables. The variable syntax of Fortran language is extended with indexes in square brackets (called co-dimension) representing a reference to data distributed across images.
By default, the CAF is using Message Passing Interface (MPI) for lower-level communication, so there are some similarities with MPI.
Read more here.
Indexing of Coarray Images¶
Indexing of individual images can be shown on the simple Hello World program:
program hello_world implicit none print *, 'Hello world from image ', this_image() , 'of', num_images() end program hello_world
- num_images() - returns the number of all images
- this_image() - returns the image index - numbered from 1 to num_images()
Co-Dimension Variables Declaration¶
Coarray variables can be declared with the codimension[*] attribute or by adding trailing index [*] after the variable name. Notice, the * character always has to be in the square brackets.
integer, codimension[*] :: scalar integer :: scalar[*] real, dimension(64), codimension[*] :: vector real :: vector(64)[*]
Because each image is running on its own, the image synchronization is needed to ensure, that all altered data is distributed to all images. Synchronization can be done across all images or only between selected images. Be aware, that selective synchronization can lead to the race condition problems like deadlock.
program synchronization_test implicit none integer :: i ! Local variable integer :: numbers[*] ! Scalar coarray ! Genereate random number on image 1 if (this_image() == 1) then numbers = floor(rand(1) * 1000) ! Distribute information to other images do i = 2, num_images() numbers[i] = numbers end do end if sync all ! Barrier to synchronize all images print *, 'The random number is', numbers end program synchronization_test
- sync all - Synchronize all images between each other
- sync images(*) - Synchronize this image to all other
- sync images(index) - Synchronize this image to image with index
number is the local variable while number[index] accesses the variable in a specific image. number[this_image()] is the same as number.
Compile and Run¶
Currently, version 1.8.10 compiled with the OpenMPI 1.10.7 library is installed on the cluster. To load the OpenCoarrays module, type:
$ ml OpenCoarrays/1.8.10-GCC-6.3.0-2.27
Compile CAF Program¶
The preferred method for compiling a CAF program is by invoking the caf compiler wrapper. The above mentioned Hello World program can be compiled as follows:
$ caf hello_world.f90 -o hello_world.x
The input file extension .f90 or .F90 are to be interpreted as Fortran 90. If the input file extension is .f or .F the source code will be interpreted as Fortran 77.
Another method for compiling is by invoking the mpif90 compiler wrapper directly:
$ mpif90 hello_world.f90 -o hello_world.x -fcoarray=lib -lcaf_mpi
Run CAF Program¶
A CAF program can be run by invoking the cafrun wrapper or directly by the mpiexec:
$ cafrun -np 4 ./hello_world.x Hello world from image 1 of 4 Hello world from image 2 of 4 Hello world from image 3 of 4 Hello world from image 4 of 4 $ mpiexec -np 4 ./synchronization_test.x The random number is 242 The random number is 242 The random number is 242 The random number is 242
-np 4 is number of images to run. The parameters of cafrun and mpiexec are the same.
For more information about running CAF program, follow Running OpenMPI - Salomon.