Sparse Variable PCA Using Geodesic Steepest Descent

• Published in: IEEE transactions on signal processing Vol. 56; no. 12; pp. 5823 - 5832
• Main Authors: Ulfarsson, M.O., Solo, V.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.12.2008; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Bayesian methods; Biological and medical sciences; Computer simulation; Computerized, statistical medical data processing and models in biomedicine; Covariance matrix; Criteria; Descent; Exact sciences and technology; Geodesic; geometric; Information, signal and communications theory; Input variables; l_{1}; LASSO; Magnetic analysis; Magnetic resonance imaging; Mathematical models; Medical management aid. Diagnosis aid; Medical sciences; Miscellaneous; Optimization; Personal communication networks; Principal component analysis; principal component analysis (PCA); Reduction; regularization; Signal processing; Signal processing algorithms; sparse; Sparse matrices; Studies; Telecommunications and information theory; Tuning; Variables; Vectors; Logarithmic function; Steepest descent method; principal component analysis (PCA); regularization; Functional analysis; Algorithm; Nuclear magnetic resonance imaging; Modeling; Optimization; Statistical method; Linear combination; sparse; Statistical model; LASSO; Geodesic; Signal processing; geometric; Medical imagery; li; Principal component analysis; Functional imaging
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2008.2006587

Principal component analysis (PCA) is a dimensionality reduction technique used in most fields of science and engineering. It aims to find linear combinations of the input variables that maximize variance. A problem with PCA is that it typically assigns nonzero loadings to all the variables, which in high dimensional problems can require a very large number of coefficients. But in many applications, the aim is to obtain a massive reduction in the number of coefficients. There are two very different types of sparse PCA problems: sparse loadings PCA (slPCA) which zeros out loadings (while generally keeping all of the variables) and sparse variable PCA which zeros out whole variables (typically leaving less than half of them). In this paper, we propose a new svPCA, which we call sparse variable noisy PCA (svnPCA). It is based on a statistical model, and this gives access to a range of modeling and inferential tools. Estimation is based on optimizing a novel penalized log-likelihood able to zero out whole variables rather than just some loadings. The estimation algorithm is based on the geodesic steepest descent algorithm. Finally, we develop a novel form of Bayesian information criterion (BIC) for tuning parameter selection. The svnPCA algorithm is applied to both simulated data and real functional magnetic resonance imaging (fMRI) data.