An explainable multi-sparsity multi-kernel nonconvex optimization least-squares classifier method via ADMM

• Published in: Neural computing & applications Vol. 34; no. 18; pp. 16103 - 16128
• Main Authors: Zhang, Zhiwang, He, Jing, Cao, Jie, Li, Shuqing, Li, Xingsen, Zhang, Kai, Wang, Pingjiang, Shi, Yong
• Format: Journal Article
• Language: English
• Published: London Springer London 01.09.2022; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Classification; Classifiers; Computational Biology/Bioinformatics; Computational geometry; Computational Science and Engineering; Computer Science; Convexity; Data mining; Data Mining and Knowledge Discovery; Feature extraction; Image Processing and Computer Vision; Iterative methods; Kernel functions; Least squares; Machine learning; Optimization; Optimization techniques; Original Article; Performance prediction; Probability and Statistics in Computer Science; Sparsity; Support vector machines; Multiple kernel learning; Nonconvex optimization; Sparse learning; Explainable; Least squares; Classification
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-022-07282-6

Convex optimization techniques are extensively applied to various models, algorithms, and applications of machine learning and data mining. For optimization-based classification methods, the sparsity principle can greatly help to select simple classifier models, while the single- and multi-kernel methods can effectively address nonlinearly separable problems. However, the limited sparsity and kernel methods hinder the improvement of the predictive accuracy, efficiency, iterative update, and interpretable classification model. In this paper, we propose a new Explainable Multi-sparsity Multi-kernel Nonconvex Optimization Least-squares Classifier (EM 2 NOLC) model, which is an optimization problem with a least-squares objective function and multi-sparsity multi-kernel nonconvex constraints, aiming to address the aforementioned issues. Based on reconstructed multiple kernel learning (MKL), the proposed model can extract important instances and features by finding the sparse coefficient and kernel weight vectors, which are used to compute importance or contribution to classification and obtain the explainable prediction. The corresponding EM 2 NOLC algorithm is implemented with the Alternating Direction Method of Multipliers (ADMM) method. On the real classification datasets, compared with the three ADMM classifiers of Linear Support Vector Machine Classifier, SVMC, Least Absolute Shrinkage and Selection Operator Classifier, the two MKL classifiers of SimpleMKL and EasyMKL, and the gradient descent classifier of Feature Selection for SVMC, our proposed EM 2 NOLC generally obtains the best predictive performance and explainable results with the least number of important instances and features having different contribution percentages.