Parallel Tensor Train Rounding using Gram SVD

• Published in: Proceedings - IEEE International Parallel and Distributed Processing Symposium pp. 930 - 940
• Main Authors: Al Daas, Hussam, Ballard, Grey, Manning, Lawton
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.05.2022
• Subjects: Approximation algorithms; Approximation error; Distributed processing; Iterative methods; Matrix decomposition; Scalability; Tensors
• ISSN: 1530-2075
• DOI: 10.1109/IPDPS53621.2022.00095

Tensor Train (TT) is a low-rank tensor representation consisting of a series of three-way cores whose dimensions specify the TT ranks. Formal tensor train arithmetic often causes an artificial increase in the TT ranks. Thus, a key operation for applications that use the TT format is rounding, which truncates the TT ranks subject to an approximation error guarantee. Truncation is performed via SVD of a highly structured matrix, and current rounding methods require careful orthogonalization to compute an accurate SVD. We propose a new algorithm for TT-Rounding based on the Gram SVD algorithm that avoids the expensive orthogonalization phase. Our algorithm performs less computation and can be parallelized more easily than existing approaches, at the expense of a slight loss of accuracy. We demonstrate that our implementation of the rounding algorithm is efficient, scales well, and consistently outperforms the existing state-of-the-art parallel implementation in our experiments.