Dynamic load balancing of MPI+OpenMP applications

• Published in: International Conference on Parallel Processing, 2004. ICPP 2004 pp. 195 - 202 vol.1
• Main Authors: Corbalan, J., Duran, A., Labarta, J.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2004
• Subjects: Computer architecture; Concurrent computing; Distributed computing; Load management; Parallel processing; Parallel programming; Power system management; Programming profession; Resource management; Runtime library
• ISBN: 9780769521978; 0769521975
• ISSN: 0190-3918
• DOI: 10.1109/ICPP.2004.1327921

The hybrid programming model MPI+OpenMP are useful to solve the problems of load balancing of parallel applications independently of the architecture. Typical approaches to balance parallel applications using two levels of parallelism or only MPI consist of including complex codes that dynamically detect which data domains are more computational intensive and either manually redistribute the allocated processors or manually redistribute data. This approach has two drawbacks: it is time consuming and it requires an expert in application analysis. In this paper we present an automatic and dynamic approach for load balancing MPI+OpenMP applications. The system calculates the percentage of load imbalance and decides a processor distribution for the MPI processes that eliminates the computational load imbalance. Results show that this method can balance effectively applications without analyzing nor modifying them and that in the cases that the application was well balanced does not incur in a great overhead for the dynamic instrumentation and analysis realized.