Iterative reweighted least squares for matrix rank minimization

• Published in: 2010 48th Annual Allerton Conference on Communication, Control, and Computing pp. 653 - 661
• Main Authors: Mohan, K, Fazel, M
• Format: Conference Proceeding
• Language: English; Japanese
• Published: IEEE 01.09.2010
• Subjects: Approximation algorithms; Clustering algorithms; Compressed sensing; Convergence; Minimization; Null space; Projection algorithms
• ISBN: 1424482151; 9781424482153
• DOI: 10.1109/ALLERTON.2010.5706969

The classical compressed sensing problem is to find the sparsest solution to an underdetermined system of linear equations. A good convex approximation to this problem is to minimize the ℓ 1 norm subject to affine constraints. The Iterative Reweighted Least Squares (IRLSp) algorithm (0 <; p ≤ 1), has been proposed as a method to solve the ℓ p (p ≤ 1) minimization problem with affine constraints. Recently Chartrand et al observed that IRLS-p with p <; 1 has better empirical performance than ℓ 1 minimization, and Daubechies et al gave `local' linear and super-linear convergence results for IRLS-p with p = 1 and p <; 1 respectively. In this paper we extend IRLS-p as a family of algorithms for the matrix rank minimization problem and we also present a related family of algorithms, sIRLS-p. We present guarantees on recovery of low-rank matrices for IRLS-1 under the Null Space Property (NSP). We also establish that the difference between the successive iterates of IRLS-p and sIRLS-p converges to zero and that the IRLS-0 algorithm converges to the stationary point of a non-convex rank-surrogate minimization problem. On the numerical side, we give a few efficient implementations for IRLS-0 and demonstrate that both sIRLS-0 and IRLS-0 perform better than algorithms such as Singular Value Thresholding (SVT) on a range of `hard' problems (where the ratio of number of degrees of freedom in the variable to the number of measurements is large). We also observe that sIRLS-0 performs better than Iterative Hard Thresholding algorithm (IHT) when there is no apriori information on the low rank solution.