HyperReconNet: Joint Coded Aperture Optimization and Image Reconstruction for Compressive Hyperspectral Imaging

• Published in: IEEE transactions on image processing Vol. 28; no. 5; pp. 2257 - 2270
• Main Authors: Wang, Lizhi, Zhang, Tao, Fu, Ying, Huang, Hua
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Aperture; Apertures; coded aperture optimization; Compressive hyperspectral imaging; Convolution; convolution neural network; hyperspectral image greconstruction; Hyperspectral imaging; Image coding; Image reconstruction; Machine learning; Methods; Neural networks; Optimization; Spatial resolution; State of the art; Three-dimensional displays
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2018.2884076

Coded aperture snapshot spectral imaging (CASSI) system encodes the 3D hyperspectral image (HSI) within a single 2D compressive image and then reconstructs the underlying HSI by employing an inverse optimization algorithm, which equips with the distinct advantage of snapshot but usually results in low reconstruction accuracy. To improve the accuracy, existing methods attempt to design either alternative coded apertures or advanced reconstruction methods, but cannot connect these two aspects via a unified framework, which limits the accuracy improvement. In this paper, we propose a convolution neural network-based end-to-end method to boost the accuracy by jointly optimizing the coded aperture and the reconstruction method. On the one hand, based on the nature of CASSI forward model, we design a repeated pattern for the coded aperture, whose entities are learned by acting as the network weights. On the other hand, we conduct the reconstruction through simultaneously exploiting intrinsic properties within HSI-the extensive correlations across the spatial and spectral dimensions. By leveraging the power of deep learning, the coded aperture design and the image reconstruction are connected and optimized via a unified framework. Experimental results show that our method outperforms the state-of-the-art methods under both comprehensive quantitative metrics and perceptive quality.