Dictionary Learning for Sparse Coding: Algorithms and Convergence Analysis

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 38; no. 7; pp. 1356 - 1369
• Main Authors: Bao, Chenglong, Ji, Hui, Quan, Yuhui, Shen, Zuowei
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Convergence; convergence analysis; Dictionaries; dictionary learning; Encoding; Image coding; Learning systems; non-convex optimization; Numerical analysis; Optimization; sparse coding; non-convex optimization; convergence analysis; sparse coding; Dictionary learning
• ISSN: 0162-8828; 1939-3539; 2160-9292; 1939-3539
• DOI: 10.1109/TPAMI.2015.2487966

In recent years, sparse coding has been widely used in many applications ranging from image processing to pattern recognition. Most existing sparse coding based applications require solving a class of challenging non-smooth and non-convex optimization problems. Despite the fact that many numerical methods have been developed for solving these problems, it remains an open problem to find a numerical method which is not only empirically fast, but also has mathematically guaranteed strong convergence. In this paper, we propose an alternating iteration scheme for solving such problems. A rigorous convergence analysis shows that the proposed method satisfies the global convergence property: the whole sequence of iterates is convergent and converges to a critical point. Besides the theoretical soundness, the practical benefit of the proposed method is validated in applications including image restoration and recognition. Experiments show that the proposed method achieves similar results with less computation when compared to widely used methods such as K-SVD.