Null space conditions and thresholds for rank minimization

• Published in: Mathematical programming Vol. 127; no. 1; pp. 175 - 202
• Main Authors: Recht, Benjamin, Xu, Weiyu, Hassibi, Babak
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer-Verlag 01.03.2011; Springer Nature B.V
• Subjects: Algorithms; Asymptotic methods; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Control theory; Exact solutions; Full Length Paper; Heuristic; Infinity; Linear operators; Machine learning; Mathematical analysis; Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematical programming; Mathematics; Mathematics and Statistics; Mathematics of Computing; Minimization; Numerical Analysis; Optimization; Phase transitions; Studies; Theoretical; 15A52; Random matrices; Matrix norms; Convex optimization; 90C25; Gaussian processes; Rank; 90C59; Compressed sensing
• ISSN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-010-0422-2

Minimizing the rank of a matrix subject to constraints is a challenging problem that arises in many applications in machine learning, control theory, and discrete geometry. This class of optimization problems, known as rank minimization, is NP-hard, and for most practical problems there are no efficient algorithms that yield exact solutions. A popular heuristic replaces the rank function with the nuclear norm—equal to the sum of the singular values—of the decision variable and has been shown to provide the optimal low rank solution in a variety of scenarios. In this paper, we assess the practical performance of this heuristic for finding the minimum rank matrix subject to linear equality constraints. We characterize properties of the null space of the linear operator defining the constraint set that are necessary and sufficient for the heuristic to succeed. We then analyze linear constraints sampled uniformly at random, and obtain dimension-free bounds under which our null space properties hold almost surely as the matrix dimensions tend to infinity. Finally, we provide empirical evidence that these probabilistic bounds provide accurate predictions of the heuristic’s performance in non-asymptotic scenarios.