Efficient Low-Rank Approximation of Matrices Based on Randomized Pivoted Decomposition

• Published in: IEEE transactions on signal processing Vol. 68; pp. 3575 - 3589
• Main Authors: Kaloorazi, Maboud F., Chen, Jie
• Format: Journal Article
• Language: English
• Published: New York IEEE 2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Acoustics; Algorithms; Approximation; Approximation algorithms; Decomposition; dimension reduction; image recovery; low-rank approximation; Machine learning algorithms; Mathematical analysis; Matrix decomposition; Randomization; randomized numerical linear algebra; rank-revealing factorization; Signal processing algorithms; Sparse matrices; Subspaces; Surges; Upper bounds
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2020.3001399

Given a matrix <inline-formula><tex-math notation="LaTeX">\bf A</tex-math></inline-formula> with numerical rank <inline-formula><tex-math notation="LaTeX">k</tex-math></inline-formula>, the two-sided orthogonal decomposition (TSOD) computes a factorization <inline-formula><tex-math notation="LaTeX">{\bf A} = {\bf UDV}^T</tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX">{\bf U}</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">{\bf V}</tex-math></inline-formula> are orthogonal, and <inline-formula><tex-math notation="LaTeX">{\bf D}</tex-math></inline-formula> is (upper/lower) triangular. TSOD is rank-revealing as the middle factor <inline-formula><tex-math notation="LaTeX">{\bf D}</tex-math></inline-formula> reveals the rank of <inline-formula><tex-math notation="LaTeX">\bf A</tex-math></inline-formula>. The computation of TSOD, however, is demanding. In this paper, we present an algorithm called randomized pivoted TSOD (RP-TSOD), where the middle factor is lower triangular. Key in our work is the exploitation of randomization, and RP-TSOD is primarily devised to efficiently construct an approximation to a low-rank matrix. We provide three different types of bounds for RP-TSOD: (i) we furnish upper bounds on the error of the low-rank approximation, (ii) we bound the <inline-formula><tex-math notation="LaTeX">k</tex-math></inline-formula> approximate principal singular values, and (iii) we derive bounds for the canonical angles between the approximate and the exact singular subspaces. Our bounds describe the characteristics and behavior of our proposed algorithm. Through numerical tests, we show the effectiveness of the devised bounds as well as our proposed algorithm.