psoResNet: An improved PSO-based residual network search algorithm

• Published in: Neural networks Vol. 172; p. 106104
• Main Authors: Wang, Dianwei, Zhai, Leilei, Fang, Jie, Li, Yuanqing, Xu, Zhijie
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.04.2024
• Subjects: Algorithms; Benchmarking; Image classification; Neural network optimization; Neural Networks, Computer; Particle swarm optimization; Residual networks; Residual networks; Particle swarm optimization; Neural network optimization; Image classification
• ISSN: 0893-6080; 1879-2782; 1879-2782
• DOI: 10.1016/j.neunet.2024.106104

Neural Architecture Search (NAS) methods are widely employed to address the time-consuming and costly challenges associated with manual operation and design of deep convolutional neural networks (DCNNs). Nonetheless, prevailing methods still encounter several pressing obstacles, including limited network architecture design, excessively lengthy search periods, and insufficient utilization of the search space. In light of these concerns, this study proposes an optimization strategy for residual networks that leverages an enhanced Particle swarm optimization algorithm. Primarily, low-complexity residual architecture block is employed as the foundational unit for architecture exploration, facilitating a more diverse investigation into network architectures while minimizing parameters. Additionally, we employ a depth initialization strategy to confine the search space within a reasonable range, thereby mitigating unnecessary particle exploration. Lastly, we present a novel approach for computing particle differences and updating velocity mechanisms to enhance the exploration of updated trajectories. This method significantly contributes to the improved utilization of the search space and the augmentation of particle diversity. Moreover, we constructed a crime-dataset comprising 13 classes to assess the effectiveness of the proposed algorithm. Experimental results demonstrate that our algorithm can design lightweight networks with superior classification performance on both benchmark datasets and the crime-dataset.