Spectral-Spatial Interaction Network for Multispectral Image and Panchromatic Image Fusion

• Published in: Remote sensing (Basel, Switzerland) Vol. 14; no. 16; p. 4100
• Main Authors: Nie, Zihao, Chen, Lihui, Jeon, Seunggil, Yang, Xiaomin
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.08.2022
• Subjects: Computer vision; Data integration; Decomposition; Deep learning; Design; Feature extraction; Image processing; information exchange; Information transfer; Machine learning; Methods; Multisensor fusion; multispectral imagery; Neural networks; Optimization; panchromatic imagery; pansharpening; Quality assessment; Remote sensing; Spatial data; spectral-spatial attention; spectral-spatial interaction network; Wavelet transforms
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs14164100

Recently, with the rapid development of deep learning (DL), an increasing number of DL-based methods are applied in pansharpening. Benefiting from the powerful feature extraction capability of deep learning, DL-based methods have achieved state-of-the-art performance in pansharpening. However, most DL-based methods simply fuse multi-spectral (MS) images and panchromatic (PAN) images by concatenating, which can not make full use of the spectral information and spatial information of MS and PAN images, respectively. To address this issue, we propose a spectral-spatial interaction Network (SSIN) for pansharpening. Different from previous works, we extract the features of PAN and MS, respectively, and then interact them repetitively to incorporate spectral and spatial information progressively. In order to enhance the spectral-spatial information fusion, we further propose spectral-spatial attention (SSA) module to yield a more effective spatial-spectral information transfer in the network. Extensive experiments on QuickBird, WorldView-4, and WorldView-2 images demonstrate that our SSIN significantly outperforms other methods in terms of both objective assessment and visual quality.