Communication-overlap techniques for improved strong scaling of gyrokinetic Eulerian code beyond 100k cores on the K-computer
Yasuhiro Idomura, M. Nakata, Susumu Yamada et al. · The International Journal of High Performance Computing Applications · 2013
Plasma turbulence research based on five-dimensional (5D) gyrokinetic simulations is one of the most critical and demanding issues in fusion science. To pioneer new physics regimes both in problem sizes and in timescales, an improvement of strong scaling is essential. Overlap of computations and communications using non-blocking MPI communication schemes is a promising approach to improving strong scaling, but it often fails on practical applications with conventional MPI libraries. In this work, this classical issue is resolved by developing communication-overlap techniques with additional MPI support for non-blocking communication routines and with heterogeneous OpenMP threads, which work even on conventional MPI libraries and network hardware. These techniques dramatically improved the parallel efficiency of a gyrokinetic toroidal 5D Eulerian code GT5D on the K-computer, which has a dedicated network, and on the Helios system which has a commodity network. On the K-computer, excellent strong scaling was achieved beyond 100k cores whilst keeping a sustained performance of [Formula: see text]10% ([Formula: see text]307 TFlops using 196,608 cores), and simulations for next-generation large-scale fusion experiments are significantly accelerated. This performance is 16[Formula: see text] sped up compared with the maximum performance reported at the 2011 International Conference for High Performance Computing, Networking, Storage and Analysis ([Formula: see text]19 TFlops using 16,384 cores of the BX900 cluster) (Idomura, 2011).
Cited by 24 works. doi:10.1177/1094342013490973
Authors
- Yasuhiro Idomura
- M. Nakata
- Susumu Yamada
- Masahiko Machida
- Toshiyuki Imamura
- M. Nunami
- Hikaru Inoue
- Shigenobu Tsutsumi
- Ikuo Miyoshi
- Naoyuki Shida
References
- A high-performance, portable implementation of the MPI message passing interface standard
- Variational Iterative Methods for Nonsymmetric Systems of Linear Equations
- Comparisons and physics basis of tokamak transport models and turbulence simulations
- Foundations of nonlinear gyrokinetic theory
- Size Scaling of Turbulent Transport in Magnetically Confined Plasmas
- Numerical simulation of ion-temperature-gradient-driven modes
- Gyrokinetic simulations of turbulent transport
- Additive Semi-Implicit Runge–Kutta Methods for Computing High-Speed Nonequilibrium Reactive Flows
- Gyroradius Scaling of Electron and Ion Transport
- RDMA read based rendezvous protocol for MPI over InfiniBand