Design of multispectral array imaging system based on depth-guided network

• Published in: Optics and lasers in engineering Vol. 175; p. 108026
• Main Authors: Yan, Gangqi, Song, Yansong, Zhang, Bo, Liang, Zonglin, Piao, Mingxu, Dong, Keyan, Zhang, Lei, Liu, Tianci, Wang, Yanbai, Li, Xinghang, Hu, Wenyi
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2024
• Subjects: Deep learning; Demosaicing; Guiding features; Multispectral imaging system; Demosaicing; Deep learning; Multispectral imaging system; Guiding features
• ISSN: 0143-8166; 1873-0302; 1873-0302
• DOI: 10.1016/j.optlaseng.2024.108026

•Image reconstruction techniques in the multispectral domain were investigated.•Design of an eight-band multispectral filter array imaging system.•A deep guidance network modeling algorithm is proposed.•Outperforms other existing methods in both quantitative and qualitative results. Imaging techniques using multispectral filter arrays (MSFA)have become a research hotspot with the rapid development of spectroscopic techniques. Among them, exploiting the correlation of color channels in the raw data and reconstructing raw images with high sparsity is a bottleneck and constraint in multi-band MSFA imaging systems. Therefore, this paper proposes a 4 × 4 eight-band MSFA imaging system containing a high sampling rate all-pass band. The all-pass band with a 1/2 high sampling rate contains rich color texture information to provide more features. A depth-guided reconstruction network (DGRN), including a depth-guided model (DGM) and a channel adaptive convolution model (CACM), is established to reconstruct the original spectral images. DGM extracts the color texture information of all-pass band images as the guide feature, which is combined with the initially processed eight-band shallow features to be the input of CACM to assign different guide features to different bands adaptively for learning and aggregation. The spatial correlation and spectral correlation of multiple bands are jointly learned using spectral and spatial properties to make the network flexible for MSFA imaging systems. The experimental results show that the method can effectively remove the artifacts of reconstructed images and improve the edge texture clarity.