A generalized l 2,p-norm regression based feature selection algorithm

• Published in: Journal of applied statistics Vol. 50; no. 3; pp. 703 - 723
• Main Authors: Zhi, X., Liu, J., Wu, S., Niu, C.
• Format: Journal Article
• Language: English
• Published: England Taylor & Francis 2023
• Subjects: 2,p; Feature selection; iterative re-weighted least squares; least square QR decomposition; norm; sparse regression; iterative re-weighted least squares; Feature selection; least square QR decomposition; l2,p-norm; sparse regression
• ISSN: 0266-4763; 1360-0532
• DOI: 10.1080/02664763.2021.1975662

Feature selection is an important data dimension reduction method, and it has been used widely in applications involving high-dimensional data such as genetic data analysis and image processing. In order to achieve robust feature selection, the latest works apply the or -norm of matrix to the loss function and regularization terms in regression, and have achieved encouraging results. However, these existing works rigidly set the matrix norms used in the loss function and the regularization terms to the same or -norm, which limit their applications. In addition, the algorithms for solutions they present either have high computational complexity and are not suitable for large data sets, or cannot provide satisfying performance due to the approximate calculation. To address these problems, we present a generalized -norm regression based feature selection ( -RFS) method based on a new optimization criterion. The criterion extends the optimization criterion of ( -RFS) when the loss function and the regularization terms in regression use different matrix norms. We cast the new optimization criterion in a regression framework without regularization. In this framework, the new optimization criterion can be solved using an iterative re-weighted least squares (IRLS) procedure in which the least squares problem can be solved efficiently by using the least square QR decomposition (LSQR) algorithm. We have conducted extensive experiments to evaluate the proposed algorithm on various well-known data sets of both gene expression and image data sets, and compare it with other related feature selection methods.