Knowledge reconstruction assisted evolutionary algorithm for neural network architecture search

• Published in: Knowledge-based systems Vol. 264; p. 110341
• Main Authors: An, Yang, Zhang, Changsheng, Zheng, Xuanyu
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 15.03.2023
• Subjects: Differential evolution; Knowledge reconstruction; Network morphism; Neural architecture search; Differential evolution; Network morphism; Knowledge reconstruction; Neural architecture search
• ISSN: 0950-7051; 1872-7409
• DOI: 10.1016/j.knosys.2023.110341

Neural architecture search (NAS) aims to provide a manual-free search method for obtaining robust and high-performance neural network structures. However, limited search space, weak empirical reusability, and low search efficiency limit the performance of NAS. This study proposes an evolutionary knowledge-reconstruction-assisted method for neural network architecture searches. First, a search space construction method based on network blocks with a-priori knowledge of the network morphism is proposed. This can reduce the computational burden and the time required for the search process while increasing the diversity of the search space. Next, a hierarchical variable-length coding strategy is designed for application to the complete evolutionary algorithm; this strategy divides the neural network into two layers for coding, satisfies the need for decoding with neural network weights, and achieves coding of neural network structures with different depths. Furthermore, the complete differential evolution algorithm is used as the search strategy, thus providing a new possibility of using the search space based on network morphism for applications related to evolutionary algorithms. In addition, the results of comparison experiments conducted on CIFAR10 and CIFAR100 indicate that the neural networks obtained using this method achieve similar or better classification accuracy compared with other neural network structure search algorithms and manually designed networks, while effectively reducing computational time and resource requirements. •A priori knowledge-based search space construction method. Unlike traditional methods, the search space is constructed by reusing part of the architecture and weights of a neural network that has already verified its performance. This method ensures the performance of the network and saves computational resources.•A hierarchical variable-length coding method is designed. It encodes the candidate solutions based on different levels. The encoding with weights is effectively achieved while ensuring the ability to explore for the optimal network depth.•In the proposed algorithm, for the first time, the search space based on network morphism is combined with a complete evolutionary algorithm to improve the diversity of populations. It also provides more possibilities for improvement of evolutionary based neural network architecture search algorithms.