Efficient shared memory and RDMA based collectives on multi-rail QsNet SMP clusters

• Published in: Cluster computing Vol. 11; no. 4; pp. 341 - 354
• Main Authors: Qian, Ying, Afsahi, Ahmad
• Format: Journal Article
• Language: English
• Published: 01.12.2008
• ISSN: 1386-7857
• DOI: 10.1007/s10586-008-0065-8

Clusters of Symmetric Multiprocessors (SMP) are more commonplace than ever in achieving high-performance. Scientific applications running on clusters employ collective communications extensively. Shared memory communication and Remote Direct Memory Access (RDMA) over multi-rail networks are promising approaches in addressing the increasing demand on intra-node and inter-node communications, and thereby in boosting the performance of collectives in emerging multi-core SMP clusters. In this regard, this paper designs and evaluates two classes of collective communication algorithms directly at the Elan user-level over multi-rail Quadrics QsNet with message striping: 1) RDMA-based traditional multi-port algorithms for gather, all-gather, and all-to-all collectives for medium to large messages, and 2) RDMA-based and SMP-aware multi-port all-gather algorithms for small to medium size messages.