A Combinatorial Design for Cascaded Coded Distributed Computing on General Networks

• Published in: IEEE transactions on communications Vol. 69; no. 9; pp. 5686 - 5700
• Main Authors: Woolsey, Nicholas, Chen, Rong-Rong, Ji, Mingyue
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.09.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Cascaded coded distributed computing; coded multicasting; Combinatorial analysis; Communication; communication load; computation load; Computational modeling; Computer networks; Distributed computing; Distributed processing; Encoding; heterogeneity; Heterogeneous networks; Information theory; Lattices; low-complexity; Nodes; Task analysis; Urban areas
• ISSN: 0090-6778; 1558-0857
• DOI: 10.1109/TCOMM.2021.3087788

Coding theoretic approaches have been developed to significantly reduce the communication load in modern distributed computing system. In particular, coded distributed computing (CDC) introduced by Li et al. can efficiently trade computation resources to reduce the communication load in MapReduce like computing systems. For the more general cascaded CDC, Map computations are repeated at <inline-formula> <tex-math notation="LaTeX">r </tex-math></inline-formula> nodes to significantly reduce the communication load among nodes tasked with computing <inline-formula> <tex-math notation="LaTeX">Q </tex-math></inline-formula> Reduce functions <inline-formula> <tex-math notation="LaTeX">s </tex-math></inline-formula> times. In this paper, we propose a novel low-complexity combinatorial design for cascaded CDC which 1) determines both input file and output function assignments, 2) requires significantly less number of input files and output functions, and 3) operates on heterogeneous networks where nodes have varying storage and computing capabilities. We provide an analytical characterization of the computation-communication tradeoff, from which we show the proposed scheme can outperform the state-of-the-art scheme proposed by Li et al. for the homogeneous networks. Further, when the network is heterogeneous, we show that the performance of the proposed scheme can be better than its homogeneous counterpart. In addition, the proposed scheme is optimal within a constant factor of the information theoretic converse bound while fixing the input file and the output function assignments.