FedMDC: Enabling Communication-Efficient Federated Learning over Packet Lossy Networks via Multiple Description Coding

• Published in: Proceedings (IEEE International Conference on Multimedia and Expo) pp. 1 - 7
• Main Authors: Guan, Yixuan, Liu, Xuefeng, Ren, Tao, Niu, Jianwei
• Format: Conference Proceeding
• Language: English
• Published: IEEE 15.07.2024
• Subjects: Communication Overhead; Decoding; Distortion; Encoding; Federated learning; Multiple Description Coding; Packet Erasure; Propagation losses; Quantization (signal); Reliability; Resilience; Resource management; Synchronization
• ISSN: 1945-788X
• DOI: 10.1109/ICME57554.2024.10687793

Federated learning (FL) generally suffers significant communication overhead from high-traffic gradient synchronization. The majority of existing studies on this problem aim at compressing gradients under the premise of reliable transmission. While transmission reliability can be ensured via TCP by default, the notably increased latency and retransmitted packets are prohibitive for most clients in FL. To tackle this issue, we propose FedMDC, a retransmission-free compression framework for FL over packet lossy networks. Given clients' limited resources, FedMDC adopts multiple description coding to encode gradients into redundant descriptions for erasure resilience simply through multiplying an overcomplete matrix; and then quantizes these descriptions for compression. To further reduce quantization distortion and computational overhead, a reduced decoding algorithm is developed by decoding the aggregation of all clients' encodings in conjunction with a customized dither quantization design. Besides, FedMDC explicitly supports adaptive bitrates subject to clients' heterogeneous communication budgets, which maximize resource utilization to facilitate distortion reduction and accelerate model convergence. Theoretical analysis and experimental results both demonstrate the effectiveness of our scheme.