Do Algorithms and Barriers for Sparse Principal Component Analysis Extend to Other Structured Settings?

• Published in: IEEE transactions on signal processing Vol. 72; pp. 3187 - 3200
• Main Authors: Wang, Guanyi, Lou, Mengqi, Pananjady, Ashwin
• Format: Journal Article
• Language: English
• Published: New York IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; computational hardness; Computational modeling; Estimation; Indexes; Iterative methods; nonconvex iterative optimization; Principal component analysis; Principal components analysis; Signal processing algorithms; structured sparsity; Vectors
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2024.3421618

We study a principal component analysis problem under the spiked Wishart model in which the structure in the signal is captured by a class of union-of-subspace models. This general class includes vanilla sparse PCA as well as its variants with graph sparsity. With the goal of studying these problems under a unified statistical and computational lens, we establish fundamental limits that depend on the geometry of the problem instance, and show that a natural projected power method exhibits local convergence to the statistically near-optimal neighborhood of the solution. We complement these results with end-to-end analyses of two important special cases given by path and tree sparsity in a general basis, showing initialization methods and matching evidence of computational hardness. Overall, our results indicate that several of the phenomena observed for vanilla sparse PCA extend in a natural fashion to its structured counterparts.