I/O performance challenges at leadership scale

• Published in: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis pp. 1 - 12
• Main Authors: Lang, Samuel, Carns, Philip, Latham, Robert, Ross, Robert, Harms, Kevin, Allcock, William
• Format: Conference Proceeding
• Language: English
• Published: New York, NY, USA ACM 14.11.2009
• Series: ACM Conferences
• Subjects: Benchmark testing; Computer architecture; File systems; General and reference > Cross-computing tools and techniques > Performance; Hardware; Ions; Leadership; Scalability; Scientific computing; Servers; Software; Software and its engineering > Software creation and management > Software verification and validation > Operational analysis; Software and its engineering > Software organization and properties > Contextual software domains > Operating systems > Communications management; Software and its engineering > Software organization and properties > Software system structures > Distributed systems organizing principles
• ISBN: 1605587443; 9781605587448
• ISSN: 2167-4329
• DOI: 10.1145/1654059.1654100

Today's top high performance computing systems run applications with hundreds of thousands of processes, contain hundreds of storage nodes, and must meet massive I/O requirements for capacity and performance. These leadership-class systems face daunting challenges to deploying scalable I/O systems. In this paper we present a case study of the I/O challenges to performance and scalability on Intrepid, the IBM Blue Gene/P system at the Argonne Leadership Computing Facility. Listed in the top 5 fastest supercomputers of 2008, Intrepid runs computational science applications with intensive demands on the I/O system. We show that Intrepid's file and storage system sustain high performance under varying workloads as the applications scale with the number of processes.