Porting a Fortran plasma simulation to Exascale on AMD GPUs using both OpenMP and Kokkos

This paper presents the 2-step work undertaken to port Gysela, a petascale Fortran simulation code for turbulence in tokamak plasmas, to GPUs. The initial porting process using OpenMP offloading allowed for good performance in most of the code with speedups ranging from × 1 to × 12.5, with the exception of the collision operator, which became a major bottleneck. This performance critical operator was then rewritten in C++ using Kokkos. The new version is known as KoLiOp, and is one of the code modules with the largest speedups relative to the CPU baseline. We explain our strategy in both phases of development and provide an in-depth analysis of how we leveraged each framework for overall performance. The techniques detailed are applicable to other codes seeking to use a portability layer. Finally, we present a comparative benchmark run on the CPU (AMD Genoa), GPU (MI250X) and APU (MI300A) partitions of the Adastra machine.