Evolution strategies based adaptive L p LS-SVM

• Published in: Information sciences Vol. 181; no. 14; pp. 3000 - 3016
• Main Authors: Wei, Liwei, Chen, Zhenyu, Li, Jianping
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 15.07.2011
• Subjects: Adaptive penalty; Classification; Evolution strategies; Least squares support vector machine; Adaptive penalty; Least squares support vector machine; Evolution strategies; Classification
• ISSN: 0020-0255; 1872-6291
• DOI: 10.1016/j.ins.2011.02.029

Not only different databases but two classes of data within a database can also have different data structures. SVM and LS-SVM typically minimize the empirical ϕ-risk; regularized versions subject to fixed penalty ( L 2 or L 1 penalty) are non-adaptive since their penalty forms are pre-determined. They often perform well only for certain types of situations. For example, LS-SVM with L 2 penalty is not preferred if the underlying model is sparse. This paper proposes an adaptive penalty learning procedure called evolution strategies (ES) based adaptive L p least squares support vector machine (ES-based L p LS-SVM) to address the above issue. By introducing multiple kernels, a L p penalty based nonlinear objective function is derived. The iterative re-weighted minimal solver (IRMS) algorithm is used to solve the nonlinear function. Then evolution strategies (ES) is used to solve the multi-parameters optimization problem. Penalty parameter p, kernel and regularized parameters are adaptively selected by the proposed ES-based algorithm in the process of training the data, which makes it easier to achieve the optimal solution. Numerical experiments are conducted on two artificial data sets and six real world data sets. The experiment results show that the proposed procedure offer better generalization performance than the standard SVM, the LS-SVM and other improved algorithms.