Deep Spectral Blending Network for Color Bleeding Reduction in PAN-Sharpening Images

• Published in: IEEE transactions on geoscience and remote sensing Vol. 62; pp. 1 - 14
• Main Authors: Kim, Hyun-Ho, Kim, Munchurl
• Format: Journal Article
• Language: English
• Published: New York IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Attention mechanism; Bleeding; Blending; Blurring; Color; color bleeding; Colour; Complexity; Complexity theory; Convolutional neural networks; deep learning (DL); Errors; Feature extraction; Floating point arithmetic; Hemorrhaging; Image acquisition; Image color analysis; Image quality; Image resolution; Interpolation; Noise levels; panchromatic (PAN)-sharpening; Registration; remote sensing; Satellite imagery; Satellites; Sensors; Signal to noise ratio
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2024.3378158

High-resolution (HR) satellites generally transmit multispectral (MS) images at a lower resolution than that of panchromatic (PAN) images. However, satellite image users often prefer MS images to have the same resolution as the corresponding PAN images. Therefore, PAN-sharpening (PS), a technique for obtaining HRMS images by utilizing low-resolution MS (LRMS) images and HRPAN images, has been a subject of study for several decades. Nevertheless, in most PS methods, various considerations are often ignored, including disparities in physical sensor locations, sensor distortions, geometric variations among acquired images, and registration errors. Owing to these missed factors, increasing the resolution by generating PS images from MS images results in increased registration errors, leading to color bleeding. Furthermore, when obtaining PS images from LRMS images, interpolation of spectral information can lead to image blurring. To address these issues, we propose a novel spectral blending network (SBN) that incorporates spectral alignment blocks (SABs) and a half-instance and half-attention block (HHB) to alleviate both color bleeding and registration errors, producing high-quality PS images with low complexity, respectively. Our SBN achieves superior performance with 1.40-4.55 dB higher peak signal-to-noise ratio (PSNR) for KOMPSAT-3A data and 1.11-3.27 dB higher PSNR for WorldView-III data, as well as with significantly lower computational complexity 42.5%-99.3% lower floating point operations per second (FLOPs) than other state-of-the-art (SOTA) methods.