Continual meta-learning algorithm

• Published in: Applied intelligence (Dordrecht, Netherlands) Vol. 52; no. 4; pp. 4527 - 4542
• Main Authors: Jiang, Mengjuan, Li, Fanzhang, Liu, Li
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.03.2022; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Computer Science; Deep learning; Feature extraction; Machine learning; Machines; Manufacturing; Mechanical Engineering; Neural networks; Processes; Deep learning; Catastrophic forgetting; Continual meta-learning algorithm; Neural network; Meta-learning
• ISSN: 0924-669X; 1573-7497
• DOI: 10.1007/s10489-021-02543-8

Deep learning has accomplished impressive excellence in many fields. However, its achievement relies on a vast amount of marker data and when there is insufficient labeled data, the phenomenon of over-fitting will occur. On the other hand, the real world tends to be so non-stationary that neural networks cannot learn continuously like humans. The specific manifestation is that learning new tasks leads to a significant decrease in its performance on old tasks. In responding to the above problem, this paper proposes a new algorithm CMLA ( C ontinual M eta- L earning A lgorithm) based on meta-learning. CMLA cannot only extract the key features of the sample, but also optimize the update method of the task gradient by introducing the cosine similarity judgment mechanism. The algorithm is tested on miniImageNet and Fewshot-CIFAR100 ( C anadian I nstitute F or A dvanced R esearch), and the outcome clearly reveals the effectiveness and superiority of the CMLA in comparison with other advanced systems. Especially compared to MAML ( M odel- A gnostic M eta- L earning) with standard four-layer convolution, the accuracy of 1 shot and 5 shot is improved by 15.4% and 16.91% respectively under the setting of 5-way on miniImageNet. CMLA not only reduces the instability of the adaptation process, but also solves the stability-plasticity dilemma to a certain extent, achieving the goal of continual learning.