Multilevel parallelization for simulating compressible turbulent flows on most kinds of hybrid supercomputers

• Published in: Computers & fluids Vol. 173; pp. 171 - 177
• Main Authors: Gorobets, A., Soukov, S., Bogdanov, P.
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier Ltd 15.09.2018; Elsevier BV
• Subjects: Compressibility; Computational fluid dynamics; Computer simulation; Costs; Data transfer (computers); Finite volume method; Fluid dynamics; GPU; Heterogeneous computing; OpenCL; Parallel CFD; Parallel processing; Supercomputers; Turbulence models; Turbulent flow; Turbulent flows; Unstructured mesh; Parallel CFD; Unstructured mesh; OpenCL; Heterogeneous computing; GPU; Turbulent flows
• ISSN: 0045-7930; 1879-0747
• DOI: 10.1016/j.compfluid.2018.03.011

•Fully-portable multilevel MPI+OpenMP+OpenCL parallelization with overlap of communications and computations.•Modeling of compressible turbulent flows on unstructured hybrid meshes.•Heterogeneous computing on CPU and GPU.•Performance on different architectures, such as multicore CPUs, MICs, GPUs of AMD and NVIDIA.•Scalability tests using tens of thousands of CPU cores, hundreds of GPUs. The paper describes a multilevel MPI+OpenMP+OpenCL parallelization approach that provides complete portability across a wide range of hybrid supercomputer architectures. A parallel CFD algorithm for heterogeneous computing of turbulent flows is presented. It simulates the compressible Navier–Stokes equations using a cell-centered finite-volume method with polynomial reconstruction on unstructured hybrid meshes. A two-level partitioning is used for the workload distribution among computing devices of hybrid nodes. The overlap of communications and computations hides the data transfer expenses. The scalability is tested on various HPC systems including a fat node with 8 GPUs and supercomputers using up to 320 GPUs. Comparison of performance is presented for multicore CPUs, Intel Xeon Phi, various GPUs of AMD and NVIDIA. The heterogeneous execution using CPUs and GPUs is studied in detail.