Kernel Path for ν-Support Vector Classification

• Published in: IEEE transaction on neural networks and learning systems Vol. 34; no. 1; pp. 490 - 501
• Main Authors: Gu, Bin, Xiong, Ziran, Li, Xiang, Zhai, Zhou, Zheng, Guansheng
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Analytical models; Classification; Computational modeling; Computer applications; Equivalence; Kernel; Kernel functions; Kernel path; Machine learning algorithms; model selection; Parameters; piecewise non-linear; Regularization; Static VAr compensators; support vector machine (SVM); Support vector machines; Training
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3097248

It is well known that the performance of a kernel method is highly dependent on the choice of kernel parameter. However, existing kernel path algorithms are limited to plain support vector machines (SVMs), which has one equality constraint. It is still an open question to provide a kernel path algorithm to <inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>-support vector classification (<inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>-SVC) with more than one equality constraint. Compared with plain SVM, <inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>-SVC has the advantage of using a regularization parameter <inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula> for controlling the number of support vectors and margin errors. To address this problem, in this article, we propose a kernel path algorithm (KP<inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>SVC) to trace the solutions of <inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>-SVC exactly with respect to the kernel parameter. Specifically, we first provide an equivalent formulation of <inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>-SVC with two equality constraints, which can avoid possible conflicts during tracing the solutions of <inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>-SVC. Based on this equivalent formulation of <inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>-SVC, we propose the KP<inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>SVC algorithm to trace the solutions with respect to the kernel parameter. However, KP<inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>SVC traces nonlinear solutions of kernel method rather than the errors of loss function, and it is still a challenge to provide the algorithm that guarantees to find the global optimal model. To address this challenging problem, we extend the classical error path algorithm to the nonlinear kernel solution paths and propose a new kernel error path (KEP) algorithm that ensures to find the global optimal kernel parameter by minimizing the cross validation error. We also provide the finite convergence analysis and computational complexity analysis to KP<inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>SVC and KEP. Extensive experimental results on a variety of benchmark datasets not only verify the effectiveness of KP<inline-formula> <tex-math notation="LaTeX">\nu </tex-math></inline-formula>SVC but also show the advantage of applying KEP to select the optimal kernel parameter.