Towards Resource-Efficient Edge AI: From Federated Learning to Semi-Supervised Model Personalization

• Published in: IEEE transactions on mobile computing Vol. 23; no. 5; pp. 6104 - 6115
• Main Authors: Zhang, Zhaofeng, Yue, Sheng, Zhang, Junshan
• Format: Magazine Article
• Language: English
• Published: Los Alamitos IEEE 01.05.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Algorithms; Computational modeling; Constraint modelling; Data models; Device heterogeneity; edge intelligence; Federated learning; Heterogeneity; Internet of Things; Machine learning; Performance evaluation; semi-supervised learning; Servers; Training
• ISSN: 1536-1233; 1558-0660
• DOI: 10.1109/TMC.2023.3316189

A central question in edge intelligence is "how can an edge device learn its local model with limited data and constrained computing capacity?" In this study, we explore the approach where a global model initialization is first obtained by running federated learning (FL) across multiple edge devices, based on which a semi-supervised algorithm is devised for a single edge device to carry out quick adaptation with its local data. Specifically, to account for device heterogeneity and resource constraints, a global model is first trained via FL, where each device conducts multiple local updates only for its customized subnet. A subset of devices can be selected to upload updates for aggregation during each training round. Further, device scheduling is optimized to minimize the training loss of FL, subject to resource constraints, based on the carefully crafted reward function defined as the one-round progress of FL each device can provide. We examine the convergence behavior of FL for the general non-convex case. For semi-supervised model personalization, we use the FL-based model initialization as a teacher network to impute soft labels on unlabeled data, thereby addressing the insufficiency of labeled data. Experiments are conducted to evaluate the performance of the proposed algorithms.