An Improved Cuckoo Search Algorithm for Optimization of Artificial Neural Network Training

• Published in: Neural processing letters Vol. 55; no. 9; pp. 12093 - 12120
• Main Authors: Maddaiah, Pedda Nagyalla, Narayanan, Pournami Pulinthanathu
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.12.2023; Springer Nature B.V
• Subjects: Accuracy; Algorithms; Artificial Intelligence; Artificial neural networks; Back propagation; Benchmarks; Bias; Communication; Complex Systems; Computational Intelligence; Computer Science; Critical point; Design engineering; Entropy (Information theory); Fault diagnosis; Feature selection; Heuristic methods; Minima; Mutation; Neural networks; Optimization; Optimization algorithms; Root-mean-square errors; Search algorithms; Steepest descent method; Voronoi graphs; Cuckoo search; Metaheuristic; Voronoi diagram; Artificial neural network; Numerical optimization
• ISSN: 1370-4621; 1573-773X
• DOI: 10.1007/s11063-023-11411-0

Artificial neural networks are widely used for solving engineering design problems of various disciplines due to its simplicity, efficiency, and adaptability. It predicts promising and accurate results. Artificial neural network solves these problems with weights and biases obtained in the training process. In training, the weights and biases have to be updated such that the difference between predicted and actual values has to be minimized. The artificial neural network uses stochastic gradient steepest descent methods to update the weights and biases for optimizing it. These methods are good at finding the optimum solution. However, they suffer from the drawbacks of vanishing gradient at local minima and critical points and are sensitive to initial weights and biases. As a result, it falls into local minima, the training time becomes high, and accuracy becomes low. One of the best solutions to overcome these problems is to use metaheuristics algorithms instead of stochastic gradient descent methods. Among metaheuristics, the cuckoo search algorithm is widely used in many applications due to its simplicity and efficiency. In this work, we proposed an improved Cuckoo search algorithm by incorporating Voronoi diagram with Cuckoo search to strengthen the weak areas of Cuckoo search and to overcome the addressed problems of the artificial neural network. The proposed Cuckoo search algorithm performance is tested on higher dimensional benchmark functions and on benchmark data sets. Moreover, its performance is compared with variants of Cuckoo search and other metaheuristic algorithms. The proposed algorithm has shown better results in terms of the number of generations, accuracy, cross-entropy, and root mean square error (RMSE).