Deep Unfolding for Snapshot Compressive Imaging

• Published in: International journal of computer vision Vol. 131; no. 11; pp. 2933 - 2958
• Main Authors: Meng, Ziyi, Yuan, Xin, Jalali, Shirin
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.11.2023; Springer; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Artificial neural networks; Coders; Computer Imaging; Computer Science; Hardware; Image Processing and Computer Vision; Image reconstruction; Imaging systems; Neural networks; Pattern Recognition; Pattern Recognition and Graphics; Software; Vision; Deep learning; Compressive imaging; Snapshot; Compressive sensing; Convolution neural network; Generative alternating projection; Convergence; Denoising
• ISSN: 0920-5691; 1573-1405
• DOI: 10.1007/s11263-023-01844-4

Snapshot compressive imaging (SCI) systems aim to capture high-dimensional ( ≥ 3 D) images in a single shot using 2D detectors. SCI devices consist of two main parts: a hardware encoder and a software decoder. The hardware encoder typically consists of an (optical) imaging system designed to capture compressed measurements. The software decoder, on the other hand, refers to a reconstruction algorithm that retrieves the desired high-dimensional signal from those measurements. In this paper, leveraging the idea of deep unrolling, we propose an SCI recovery algorithm, namely GAP-net, which unfolds the generalized alternating projection (GAP) algorithm. At each stage, GAP-net passes its current estimate of the desired signal through a trained convolutional neural network (CNN). The CNN operates as a denoiser projecting the estimate back to the desired signal space. For the GAP-net that employs trained auto-encoder-based denoisers, we prove a probabilistic global convergence result. Finally, we investigate the performance of GAP-net in solving video SCI and spectral SCI problems. In both cases, GAP-net demonstrates competitive performance on both synthetic and real data. In addition to its high accuracy and speed, we show that GAP-net is flexible with respect to signal modulation implying that a trained GAP-net decoder can be applied in different systems. Our code is available at https://github.com/mengziyi64/GAP-net .