RIKEN Center for Computational Science (R-CCS), Japan
The multi-scale full-f simulation of the next generation experimental fusion reactor ITER based on a five dimensional (5D) gyrokinetic model is one of the most computationally demanding problems in fusion science. In this work, a Gyrokinetic Toroidal 5D Eulerian code (GT5D) is accelerated by a new mixed-precision communication-avoiding (CA) Krylov method. The bottleneck of global collective communication on accelerated computing platforms is resolved using a CA Krylov method. In addition, a new FP16 preconditioner, which is designed using the new support for FP16 SIMD operations on A64FX, reduces both the number of iterations (halo data communication) and the computational cost. The performance of the proposed method for ITER size simulations with approximately 0.1 trillion grids on 1440 CPUs/GPUs on Fugaku and Summit shows 2.8x and 1.9x speedups, respectively, above the conventional non-CA Krylov method, and excellent strong scaling is obtained up to 5760 CPUs/GPUs.