A Two-Stage Underwater Enhancement Network Based on Structure Decomposition and Characteristics of Underwater Imaging

• Published in: IEEE journal of oceanic engineering Vol. 46; no. 4; pp. 1213 - 1227
• Main Authors: Wu, Shengcong, Luo, Ting, Jiang, Gangyi, Yu, Mei, Xu, Haiyong, Zhu, Zhongjie, Song, Yang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.10.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Ablation; Artificial neural networks; Attenuation; Color; Colour; Convolutional neural network (CNN); Decomposition; Distortion; Estimation; Generative adversarial networks; Hafnium; Image color analysis; Image contrast; Image enhancement; Image quality; Image restoration; Imaging techniques; Light attenuation; Scattering; structure decomposition; Theoretical analysis; Underwater; underwater image enhancement (UIE); Underwater structures
• ISSN: 0364-9059; 1558-1691
• DOI: 10.1109/JOE.2021.3064093

Due to the scattering and attenuation of light into the water, the underwater image usually appears with color distortion, blurred details, and low contrast. To address these problems, a novel two-stage underwater image convolutional neural network (CNN) based on structure decomposition (UWCNN-SD) for underwater image enhancement is proposed by considering the characteristics of underwater imaging. Specifically, the raw underwater image is decomposed into high-frequency and low-frequency based on theoretical analysis of the underwater imaging. Then, a two-stage underwater enhancement network including a preliminary enhancement network and a refinement network is proposed. In the first stage, the preliminary enhancement network, which contains the high-frequency and the low-frequency enhancement networks, is proposed. The high-frequency part is enhanced directly by a deep learning network, and the low-frequency enhancement network is based on the underwater imaging, which is integrated transmission map and background light into joint component map. In the second stage, the refinement network is designed to further optimize the color of the underwater image by considering complexity of underwater imaging. The experimental results of synthetic and real-world underwater images/videos demonstrate that the proposed UWCNN-SD method can perform color correction and enhancement on different types of underwater images. The ablation study verifies the effectiveness of each component, and application tests further illustrate that the proposed UWCNN-SD method can obtain underwater images with higher visual quality. The trained model is available at: https://github.com/wushengcong/UWCNN-SD .