Deep hybrid model for single image dehazing and detail refinement

• Published in: Pattern recognition Vol. 136; p. 109227
• Main Authors: Jiang, Nanfeng, Hu, Kejian, Zhang, Ting, Chen, Weiling, Xu, Yiwen, Zhao, Tiesong
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2023
• Subjects: Details refinement; Haze removal; Image processing; Image processing; Details refinement; Haze removal
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2022.109227

•We propose a hybrid network, which employs two sub-networks to handle haze removal and image details refinement, respectively. Each sub-network has its own contributions to enhance image. One sub-network is used to reconstruct the basic dehazed image and the other one further enhances the contrast and details of the dehazed result. The whole dehazing framework adopts multi-term losses to optimize model, and this dehazing process also can be treated as a coarse-to-fine manner.•To obtain basic dehazed result, we combine the Squeeze-and-Excitation (SE) with residual learning to aggregate more spatial contextual information, which can be used to remove basic haze components. To preserve more image details, we utilize dilated convolution operation to obtain larger fields and design multi-scale dilated layers to conduct image details refinement.•Our method can effectively remove complex haze and preserve more image details. Particularly, the detail refinement sub-network can be detached into other existing SID methods to improve their dehazing performance. Experimental results substantial improvements both on quantitative indicators and visual effects over the current state-of-the-art technologies. Besides, the proposed method can promote other computer vision tasks, such like object detection. Deep learning technologies have been applied in Single Image Dehazing (SID) tasks successfully. However, most SID algorithms seldom consider to refine image details during dehazing. Therefore, there exist some detail-loss regions in dehazed results. To solve this issue, we design a deep hybrid network to improve dehazing performance and remedy the loss of details. Different from existing algorithms that usually ignore detail refinement and adopt a unified framework to remove haze, we propose to treat dehazing and detail refinement as two separate tasks, so that each task could be solved via different ways. Particularly, we design two sub-networks with a multi-term loss function. First, for removing haze effectively, we introduce the Squeeze-and-Excitation (SE) to design a haze residual attention sub-network, which is used to reconstruct the dehazed image. Second, as for remedying details, we take the previous dehazed image as the input to a detail refinement sub-network, where the image details can be enhanced via multi-scale contextual information aggregation. Through the joint training of two sub-network, the haze can be removed clearly and the image details can be preserved well. Moreover, the detail refinement sub-network can be detached into other existing dehazing methods to improve their model performances. Extensive experiments also verify the superiority of our proposed network against recently proposed state-of-the-arts.