Allocating Independent Subtasks on Parallel Processors

• Published in: IEEE transactions on software engineering Vol. SE-11; no. 10; pp. 1001 - 1016
• Main Authors: Kruskal, C.P., Weiss, A.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.10.1985; Institute of Electrical and Electronics Engineers; IEEE Computer Society
• Subjects: Applied sciences; Computer aided instruction; Computer programming; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Concurrent computing; Efficiency; Engineering; Exact sciences and technology; Finishing; Mathematical models; Parallel processing; Partial differential equations; Performance analysis; Processor scheduling; Queueing analysis; Queuing; Random variables; Reliability theory; scheduling; Software; Software engineering; Statistical analysis; Statistical distributions; Theory; Parallel processing; Processor; Performance analysis; Synchronization; Integrated planning; Queue
• ISSN: 0098-5589; 1939-3520
• DOI: 10.1109/TSE.1985.231547

When using MIMD (multiple instruction, multiple data) parallel computers, one is often confronted with solving a task composed of many independent subtasks where it is necessary to synchronize the processors after all the subtasks have been completed. This paper studies how the subtasks should be allocated to the processors in order to minimize the expected time it takes to finish all the subtasks (sometimes called the makespan). We assume that the running times of the subtasks are independent, identically distributed, increasing failure rate random variables, and that assigning one or more subtasks to a processor entails some overhead, or communication time, that is independent of the number of subtasks allocated. Our analyses, which use ideas from renewal theory, reliability theory, order statistics, and the theory of large deviations, are valid for a wide class of distributions. We show that allocating an equal number of subtasks to each processor all at once has good efficiency. This appears as a consequence of a rather general theorem which shows how some consequences of the central limit theorem hold even when we cannot prove that the central limit theorem applies.