Scalable work stealing

• Published in: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis pp. 1 - 11
• Main Authors: Dinan, James, Larkins, D. Brian, Sadayappan, P., Krishnamoorthy, Sriram, Nieplocha, Jarek
• Format: Conference Proceeding
• Language: English
• Published: New York, NY, USA ACM 14.11.2009
• Series: ACM Conferences
• Subjects: ARMCI; Benchmark testing; Computational modeling; dynamic load balancing; Electronics packaging; General and reference > Cross-computing tools and techniques > Performance; global arrays; Load management; Load modeling; Parallel processing; Parallel programming; PGAS; Program processors; Scalability; Servers; Software and its engineering > Software creation and management > Software verification and validation > Operational analysis; Software and its engineering > Software organization and properties > Contextual software domains > Operating systems > Process management; Software and its engineering > Software organization and properties > Software system structures > Distributed systems organizing principles; task pools; Theory of computation > Design and analysis of algorithms > Approximation algorithms analysis > Scheduling algorithms; Theory of computation > Design and analysis of algorithms > Online algorithms > Online learning algorithms > Scheduling algorithms; Theory of computation > Theory and algorithms for application domains > Machine learning theory > Reinforcement learning > Sequential decision making; work stealing; ARMCI; global arrays; work stealing; PGAS; task pools; dynamic load balancing
• ISBN: 1605587443; 9781605587448
• ISSN: 2167-4329
• DOI: 10.1145/1654059.1654113

Irregular and dynamic parallel applications pose significant challenges to achieving scalable performance on large-scale multicore clusters. These applications often require ongoing, dynamic load balancing in order to maintain efficiency. Scalable dynamic load balancing on large clusters is a challenging problem which can be addressed with distributed dynamic load balancing systems. Work stealing is a popular approach to distributed dynamic load balancing; however its performance on large-scale clusters is not well understood. Prior work on work stealing has largely focused on shared memory machines. In this work we investigate the design and scalability of work stealing on modern distributed memory systems. We demonstrate high efficiency and low overhead when scaling to 8,192 processors for three benchmark codes: a producer-consumer benchmark, the unbalanced tree search benchmark, and a multiresolution analysis kernel.