CG-Kit: Code Generation Toolkit for performant and maintainable variants of source code applied to Flash-X hydrodynamics simulations

• Published in: Future generation computer systems Vol. 163; no. C; p. 107511
• Main Authors: Rudi, Johann, Lee, Youngjun, Chadha, Aidan H., Wahib, Mohamed, Weide, Klaus, O’Neal, Jared P., Dubey, Anshu
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.02.2025; Elsevier
• Subjects: Algorithmic portability; Code generation; Control flow graph; Multiphysics simulation; Performance portability; Syntax tree; Code generation; Syntax tree; Algorithmic portability; Performance portability; Multiphysics simulation; Control flow graph
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2024.107511

CG-Kit is a new Code Generation tool-Kit that we have developed as a part of the solution for portability and maintainability for multiphysics computing applications. The development of CG-Kit is rooted in the urgent need created by the shifting landscape of high-performance computing platforms and the algorithmic complexities of a particular large-scale multiphysics application: Flash-X. To efficiently use computing resources on a heterogeneous node, an application must have a map of computation to resources and a mechanism to move the data and computation to the resources according to the map. Most existing performance portability solutions are focussed on abstracting the expression of computations so that a unified source code can be specialized to run on different resources. However, such an approach is insufficient for a code like Flash-X, which has a multitude of code components that can be assembled in various permutations and combinations to form different instances of applications. Similar challenges apply to any code that has composability, where a single specified way of apportioning work among devices may not be optimal. Additionally, use cases arise where the optimal control flow of computation may differ for different devices while the underlying numerics remain identical. This combination leads to unique challenges including handling an existing large code base in Fortran and/or C/C++, subdivision of code into a great variety of units supporting a wide range of physics and numerical methods, different parallelization techniques for distributed and shared memory systems and accelerator devices, and heterogeneity of computing platforms requiring coexisting variants of parallel algorithms. All of these challenges demand that scientific software developers apply existing knowledge about domain applications, algorithms, and computing platforms to determine custom abstractions and granularity for code generation. There is a critical lack of tools to tackle those problems. CG-Kit is designed to fill this gap by providing a user with the ability to express their desired control flow and computation-to-resource map in the form a pseudocode-like recipe. It consists of standalone tools that can be combined into highly specific and, we argue, highly effective portability and maintainability toolchains. Here we present the design of our new tools: parametrized source trees, control flow graphs, and recipes. The tools are implemented in Python. They are agnostic to the programming language of the source code targeted for code generation. We demonstrate the capabilities of the toolkit with two examples, first, multithreaded variants of the basic AXPY operation, and second, variants of parallel algorithms within a hydrodynamics solver, called Spark, from Flash-X that operates on block-structured adaptive meshes. •Scientific computing applications demand portability across computing platforms.•Variants of algorithms or implementations provide adaptability to new HPC platforms.•Scalable implementation of variants is enabled by code generation.•CG-Kit users decide on abstractions and desired granularity for code generation.•Reductions in lines of C/C++/Fortran code of over 60% are demonstrated.