An optimized RBF neural network algorithm based on partial least squares and genetic algorithm for classification of small sample

• Published in: Applied soft computing Vol. 48; pp. 373 - 384
• Main Authors: Jia, Weikuan, Zhao, Dean, Ding, Ling
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.11.2016
• Subjects: Genetic algorithm; Partial least squares; PLS-GA-RBF algorithm; RBF neural network; Small sample classification; RBF neural network; PLS-GA-RBF algorithm; Genetic algorithm; Partial least squares; Small sample classification
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2016.07.037

[Display omitted] This paper’s Graphical abstract When using the RBF neural network to deal with small samples with high feature dimension and few numbers, too many inputs are difficult to determine the numbers of hidden layer neurons, it influences the design structure of the network, the redundancies or correlative data will influence the training of the network, and relatively few number of samples make network train non-completed or over-fitted, thereby affecting the operating efficiency and recognition accuracy of neural network. For the problem of small sample classification, two aspects of RBF neural network are optimized. Firstly, the original data reduces their feature dimension by PLS algorithm, then the low dimensional data is used as network input, it regard as external optimization. And then, using genetic algorithm to optimize RBF, the optimization way adopts hybrid coding and simultaneous evolving for hidden layer neurons and connection weights, this step regard as internal optimization. By these two consecutive optimizations, an optimized RBF neural network algorithm based on PLS and GA (PLS-GA-RBF algorithm) for small sample is established, which facilitates the hidden layer of network design, and improves the network training speed and generalization ability, thereby improving the operating efficiency and recognition accuracy of the network. The new algorithm is ingenious combination of the advantages of three algorithms, it realize the external optimization by PLS and internal optimization by GA. PLS-GA-RBF algorithm can fit more complex nonlinear recognition problems, and is more suitable for the small sample classification, which with high feature dimension and fewer numbers. In order to verify the reliability of the PLS-GA-RBF algorithm, multiple instances is used to validate and analysis. In this paper, four different experiments are arranged; among them are three small sample test and one large sample test. The purpose of the arrangement large sample test is to compare of validation. The result is satisfactory, which means the new algorithm has unique superiority in dealing with the small sample. •The nature of small sample is well-analyzed.•PLS is employed to reduce feature dimension of small sample, which obtained the relatively ideal low-dimensional data as the inputs of neural network.•Unlike previous studies, the optimized GA-RBF algorithm is adopts the way of hybrid coding and simultaneous evolving for hidden layer neurons and connection weights.•By two consecutive optimization, combining the advantages of three algorithms of PLS, GA, and RBF, a reliable small sample classification algorithm (PLS-GA-RBF) is established.•Four different groups of experiments are arranged to valuate the classification ability of PLS-GA-RBF algorithm. Radial basis function (RBF) neural network can use linear learning algorithm to complete the work formerly handled by nonlinear learning algorithm, and maintain the high precision of the nonlinear algorithm. However, the results of RBF would be slightly unsatisfactory when dealing with small sample which has higher feature dimension and fewer numbers. Higher feature dimension will influence the design of neural network, and fewer numbers of samples will cause network training incomplete or over-fitted, both of which restrict the recognition precision of the neural network. RBF neural network has some drawbacks, for example, it is hard to determine the numbers, center and width of the hidden layer’s neurons, which constrain the success of training. To solve the above problems, partial least squares (PLS) and genetic algorithm(GA)are introduced into RBF neural network, and better recognition precision will be obtained, because PLS is good at dealing with the small sample data, it can reduce feature dimension and make low-dimensional data more interpretative. In addition, GA can optimize the network architecture, the weights between hidden layer and output layer of the RBF neural network can ease non-complete network training, the way of hybrid coding and simultaneous evolving is adopted, and then an accurate algorithm is established. By these two consecutive optimizations, the RBF neural network classification algorithm based on PLS and GA (PLS-GA-RBF) is proposed, in order to solve some recognition problems caused by small sample. Four experiments and comparisons with other four algorithms are carried out to verify the superiority of the proposed algorithm, and the results indicate a good picture of the PLS-GA-RBF algorithm, the operating efficiency and recognition accuracy are improved substantially. The new small sample classification algorithm is worthy of further promotion.