Relationship between the optimal solutions of least squares regularized with ℓ0-norm and constrained by k-sparsity

• Published in: Applied and computational harmonic analysis Vol. 41; no. 1; pp. 237 - 265
• Main Author: Nikolova, Mila
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.07.2016; Elsevier
• Series: Sparse Representations with Applications in Imaging Science, Data Analysis and Beyond
• Subjects: [formula omitted]-regularization; Computer Science; Engineering Sciences; Globally optimal solutions; k-sparsity constraint; Mathematics; Numerical Analysis; Optimal solution analysis; Parameter selection; Quasi-equivalence between nonconvex problems; Signal and Image processing; Sparse recovery; Under-determined linear systems; Optimal solution analysis; Globally optimal solutions; Under-determined linear systems; Quasi-equivalence between nonconvex problems; Parameter selection; Sparse recovery; ℓ0-regularization; k-sparsity constraint; optimal solution analysis; globally optimal solutions; sparse recovery; quasi-equivalence between nonconvex problems; L0 -regularization; under-determined linear systems; parameter selection
• ISSN: 1063-5203; 1096-603X; 1096-603X
• DOI: 10.1016/j.acha.2015.10.010

Two widely used models to find a sparse solution from a noisy underdetermined linear system are the constrained problem where the quadratic error is minimized subject to a sparsity constraint, and the regularized problem where a regularization parameter balances the minimization of both quadratic error and sparsity. However, the connections between these two problems have remained unclear so far. We provide an exhaustive description of the relationship between their globally optimal solutions. A partial equivalence between them always exists. We exhibit a sequence of critical parameters that partitions the positive axis into a certain number of intervals. For every regularization parameter inside an interval, there is a sparsity level such that the regularized problem and the constrained problem have the same global minimizers. At the values of the critical parameters, the optimal set of the regularized problem contains two optimal sets of the constrained problem. When the length of the sequence of critical parameters equals the number of all sparsity levels, both problems are quasi-completely equivalent. The critical parameters are obtained from the optimal values of the constrained problem.