Hyperspectral and multispectral image fusion: When model-driven meet data-driven strategies

• Published in: Information fusion Vol. 116; p. 102803
• Main Authors: Yan, Hao-Fang, Zhao, Yong-Qiang, Chan, Jonathan Cheung-Wai, Kong, Seong G., EI-Bendary, Nashwa, Reda, Mohamed
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.04.2025
• Subjects: Data-driven; Hyperspectral and multispectral image fusion; Model-data-driven; Model-driven; Technical review; Technical review; Model-data-driven; Data-driven; Model-driven; Hyperspectral and multispectral image fusion
• ISSN: 1566-2535
• DOI: 10.1016/j.inffus.2024.102803

Hyperspectral image (HSI) and Multispectral Image (MSI) fusion aims at combining a high-resolution MSI (HR MSI) with a low-resolution HSI (LR HSI), resulting in a fused image that contains the spatial resolution of the former and the spectral resolution of the latter. This approach offers a cost-effective alternative to directly acquiring high-resolution HSIs (HR HSIs). In this survey, we offer an extensive literature review tailored for students and professionals seeking deeper insights into the subject matter. We delve into existing HSI-MSI fusion methods and revealed a spectrum of approaches, ranging from model-driven techniques (extended CS and MRA, Bayesian, matrix factorization, and tensor representation) to data-driven methods (CNN, GAN, and Transformer) and model-data-driven approaches (model-guided networks and semi-supervised or unsupervised methods). This exploration aims to optimize fusion strategies for various applications. This paper not only provides a comprehensive overview of HSI-MSI fusion strategies, but also summarizes and contrasts their unique characteristics, benefits, and limitations. Additionally, it reviews image quality evaluation indices (both full-reference and no-reference) and widely used datasets. Furthermore, using hybrid data, large-view-field satellite data and real satellite data pairs, the reduced-resolution and full-resolution experimental comparison analysis of various algorithms from three strategies are carried out. Finally, the paper identifies unresolved challenges and outlines potential future research directions in this evolving field.