Accelerating distributed deep neural network training with pipelined MPI allreduce - Castelló, Adrián; Quintana-Ortí, Enrique S.; Duato, José (Cluster computing, 2021)
Keywords: Algorithms; Application programming interface; Artificial neural networks; Bandwidths; Communication; Computer Communication Networks; Computer Science; Energy consumption; Neural networks; Operating Systems; Processor Architectures; Synchronism; Training; Allreduce; Deep learning; Distributed training; Collective communication primitives; Message Passing Interface (MPI)