Optimizing the neural network hyperparameters utilizing genetic algorithm

• Published in: Journal of Zhejiang University. A. Science Vol. 22; no. 6; pp. 407 - 426
• Main Authors: Nikbakht, Saeid, Anitescu, Cosmin, Rabczuk, Timon
• Format: Journal Article
• Language: English
• Published: Hangzhou Zhejiang University Press 01.06.2021; Springer Nature B.V; Faculty of Civil Engineering,Ton Duc Thang University,Ho Chi Minh City,Vietnam; Institute of Structural Mechanics,Bauhaus-Universit?t Weimar,Weimar 99423,Germany%Division of Computational Mechanics,Ton Duc Thang University,Ho Chi Minh City,Vietnam
• Subjects: Accuracy; Algorithms; Artificial neural networks; Civil Engineering; Classical and Continuum Physics; Energy methods; Engineering; Genetic algorithms; Industrial Chemistry/Chemical Engineering; Learning algorithms; Machine learning; Mechanical Engineering; Neural networks; Neurons; Optimization; Stress concentration; Stress distribution; Timoshenko beams; 遗传算法; Neural network (NN); Hyperparameters; 机器学习; TP18; Genetic algorithm; Machine learning; 神经网络; 超参数; Neural network(NN)
• ISSN: 1673-565X; 1862-1775
• DOI: 10.1631/jzus.A2000384

Neural networks (NNs), as one of the most robust and efficient machine learning methods, have been commonly used in solving several problems. However, choosing proper hyperparameters (e.g. the numbers of layers and neurons in each layer) has a significant influence on the accuracy of these methods. Therefore, a considerable number of studies have been carried out to optimize the NN hyperparameters. In this study, the genetic algorithm is applied to NN to find the optimal hyperparameters. Thus, the deep energy method, which contains a deep neural network, is applied first on a Timoshenko beam and a plate with a hole. Subsequently, the numbers of hidden layers, integration points, and neurons in each layer are optimized to reach the highest accuracy to predict the stress distribution through these structures. Thus, applying the proper optimization method on NN leads to significant increase in the NN prediction accuracy after conducting the optimization in various examples.