For most large underdetermined systems of equations, the minimal 1-norm near-solution approximates the sparsest near-solution

• Published in: Communications on pure and applied mathematics Vol. 59; no. 7; pp. 907 - 934
• Main Author: Donoho, David L.
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.07.2006
• ISSN: 0010-3640; 1097-0312
• DOI: 10.1002/cpa.20131

We consider inexact linear equations y ≈ Φx where y is a given vector in ℝn, Φ is a given n × m matrix, and we wish to find x0,ϵ as sparse as possible while obeying ‖y − Φx0,ϵ‖2 ≤ ϵ. In general, this requires combinatorial optimization and so is considered intractable. On the other hand, the 𝓁1‐minimization problem $${\rm min} \; \|x\|_{1}\;\;\; {\rm subject \; to}\;\;\; \|y - \Phi{x}\|_{2} \leq \epsilon$$ is convex and is considered tractable. We show that for most Φ, if the optimally sparse approximation x0,ϵ is sufficiently sparse, then the solution x1,ϵ of the 𝓁1‐minimization problem is a good approximation to x0,ϵ. We suppose that the columns of Φ are normalized to the unit 𝓁2‐norm, and we place uniform measure on such Φ. We study the underdetermined case where m ∼ τn and τ > 1, and prove the existence of ρ = ρ(τ) > 0 and C = C(ρ, τ) so that for large n and for all Φ's except a negligible fraction, the following approximate sparse solution property of Φ holds: for every y having an approximation ‖y − Φx0‖2 ≤ ϵ by a coefficient vector x0 ∈ ℝm with fewer than ρ · n nonzeros, $$ \|x_{1,\epsilon} - x_{0}\|_{2} \leq C \cdot \epsilon .$$ This has two implications. First, for most Φ, whenever the combinatorial optimization result x0,ϵ would be very sparse, x1,ϵ is a good approximation to x0,ϵ. Second, suppose we are given noisy data obeying y = Φx0 + z where the unknown x0 is known to be sparse and the noise ‖z‖2 ≤ ϵ. For most Φ, noise‐tolerant 𝓁1‐minimization will stably recover x0 from y in the presence of noise z. We also study the barely determined case m = n and reach parallel conclusions by slightly different arguments. Proof techniques include the use of almost‐spherical sections in Banach space theory and concentration of measure for eigenvalues of random matrices. © 2006 Wiley Periodicals, Inc.