Exploiting partial replication in unbalanced parallel loop scheduling on multicomputers

• Published in: Microprocessing and microprogramming Vol. 41; no. 8-9; pp. 645 - 658
• Main Authors: Orlando, Salvatore, Perego, Raffaele
• Format: Journal Article
• Language: English
• Published: 01.04.1996
• ISSN: 0165-6074
• DOI: 10.1016/0165-6074(96)00002-6

The problem of scheduling parallel loops whose iterations operate on large array data structures and are characterized by highly varying execution times (unbalanced or non-uniform parallel loops) is considered. A general parallel loop implementation template for message-passing distributed-memory multiprocessors (multicomputers) is presented. Assuming that it is impossible to statically determine the distribution of the computational load on the data accessed, the template exploits a hybrid scheduling strategy. The data are partially replicated on the processor's local memories and iterations are statically scheduled until first load imbalances are detected. At this point an effective dynamic scheduling technique is adopted to move iterations among nodes holding the same data.