Distillate a Sparse-Meta Time Series Classifier for Open Radio Access Network-Based Cellular Vehicle-to-Everything

• Published in: IEEE transactions on vehicular technology Vol. 73; no. 7; pp. 9262 - 9271
• Main Authors: Sun, Le, Liang, Jiancong, Muhammad, Ghulam
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.07.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Ablation; Algorithms; Catastrophic forgetting; Cellular radio; cellular vehicle-to-everything (CV2x); Classification; Deep learning; Distillation; Knowledge management; Machine learning; Mathematical models; meta-transfer learning; Metalearning; open radio access network (ORAN); Open RAN; Radio access networks; sparse Model-Agnostic Meta-Learning (sparse-MAML); Teachers; Time series; Time series analysis; Training; User experience; Vehicle-to-everything
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2023.3323279

Deep learning-based univariate time series classification can improve the user experience of Open Radio Access Network (ORAN)-based Cellular Vehicle-to-Everything (CV2x). However, few institutes researching ORAD-based CV2x can satisfy the enormous demand of labeled data. This issue is known as few-shot learning. Thus, we deeply explore the issue of few-shot learning for ORAE-based CV2x. Meta-transfer learning is a good alternative to solving few-shot learning. Most of them, however, are still plagued by catastrophic forgetting. Numerous studies have demonstrated that deliberately applying gradient sparsity can significantly increase a meta-model's capacity for generalization. In this article, we propose a pre-training framework named Distilling for Sparse-Meta-transfer Learning (DSML). It is a combination and enhancement of meta-transfer learning, multi-teacher knowledge distillation, and sparse Model-Agnostic Meta-Learning (sparse-MAML). It utilizes multi-teacher knowledge distillation to address the catastrophic forgetting in the meta-learning phase. Simultaneously, it utilizes sigmoid function to fundamentally address the gradient anomaly problem of sparse-MAML. We conducted ablation experiments on Sparse-MAML and prove that it can actually increase the meta-model's generalization capacity. We also compare DSML with the state-of-the-art algorithm in the univariate time series classification field. The results demonstrate that DSML performs better. Finally, we present two case studies of applying DSML to ORAN-based CV2x.