L Regularization: Convergence of Iterative Half Thresholding Algorithm

• Published in: IEEE transactions on signal processing Vol. 62; no. 9; pp. 2317 - 2329
• Main Authors: Jinshan Zeng, Shaobo Lin, Yao Wang, Zongben Xu
• Format: Journal Article
• Language: English; Japanese
• Published: IEEE 01.05.2014
• Subjects: Convergence; Convex programming; iterative half thresholding algorithm; Iterative methods; L_{q}/L_{1/2} regularization; nonconvex regularization
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2014.2309076

In recent studies on sparse modeling, the nonconvex regularization approaches (particularly, L q regularization with q ∈ (0,1)) have been demonstrated to possess capability of gaining much benefit in sparsity-inducing and efficiency. As compared with the convex regularization approaches (say, L 1 regularization), however, the convergence issue of the corresponding algorithms are more difficult to tackle. In this paper, we deal with this difficult issue for a specific but typical nonconvex regularization scheme, the L 1/2 regularization, which has been successfully used to many applications. More specifically, we study the convergence of the iterative half thresholding algorithm (the half algorithm for short), one of the most efficient and important algorithms for solution to the L 1/2 regularization. As the main result, we show that under certain conditions, the half algorithm converges to a local minimizer of the L 1/2 regularization, with an eventually linear convergence rate. The established result provides a theoretical guarantee for a wide range of applications of the half algorithm. We provide also a set of simulations to support the correctness of theoretical assertions and compare the time efficiency of the half algorithm with other known typical algorithms forL 1/2 regularization like the iteratively reweighted least squares (IRLS) algorithm and the iteratively reweighted l 1 minimization (IRL1) algorithm.