Asynchronous Message-Passing and Zeroth-Order Optimization Based Distributed Learning With a Use-Case in Resource Allocation in Communication Networks

• Published in: IEEE transactions on signal and information processing over networks Vol. 10; pp. 916 - 931
• Main Authors: Behmandpoor, Pourya, Moonen, Marc, Patrinos, Panagiotis
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: asynchronous distributed optimization; bounded delay; Communication; Communication networks; Communications networks; Computer aided instruction; Convergence; Cost function; Deep learning; deep learning-based resource allocation; Delays; Distance learning; Distributed learning and adaptation; Federated learning; Machine learning; Message passing; Optimization; Parameters; Receivers; Resource allocation; Resource management; Training; Transmitters; zeroth-order optimization
• ISSN: 2373-776X; 2373-7778
• DOI: 10.1109/TSIPN.2024.3487421

Distributed learning and adaptation have received significant interest and found wide-ranging applications in machine learning and signal processing. While various approaches, such as shared-memory optimization, multi-task learning, and consensus-based learning (e.g., federated learning and learning over graphs), focus on optimizing either local costs or a global cost, there remains a need for further exploration of their interconnections. This paper specifically focuses on a scenario where agents collaborate towards a common task (i.e., optimizing a global cost equal to aggregated local costs) while effectively having distinct individual tasks (i.e., optimizing individual local parameters in a local cost). Each agent's actions can potentially impact other agents' performance through interactions. Notably, each agent has access to only its local zeroth-order oracle (i.e., cost function value) and shares scalar values, rather than gradient vectors, with other agents, leading to communication bandwidth efficiency and agent privacy. Agents employ zeroth-order optimization to update their parameters, and the asynchronous message-passing between them is subject to bounded but possibly random communication delays. This paper presents theoretical convergence analyses and establishes a convergence rate for nonconvex problems. Furthermore, it addresses the relevant use-case of deep learning-based resource allocation in communication networks and conducts numerical experiments in which agents, acting as transmitters, collaboratively train their individual policies to maximize a global reward, e.g., a sum of data rates.