Joint image upsampling with affinity learning

• Published in: Neurocomputing (Amsterdam) Vol. 653; p. 131159
• Main Authors: Yang, Yang, Luo, Shuai, Qiu, Shuailong, Zeng, Lanling, Wang, Xinyu, Wang, Xinsheng
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 07.11.2025
• Subjects: Convolutional neural network; Depth map upsampling; Image processing; Joint image upsampling; Joint image upsampling; Image processing; Convolutional neural network; Depth map upsampling
• ISSN: 0925-2312
• DOI: 10.1016/j.neucom.2025.131159

Joint image upsampling is of significance to various fields of image processing and computer vision. Traditional joint upsampling methods mostly attempt to model the relations between the low-resolution guidance and target images, then the relations are upsampled and applied to the high-resolution guidance image. However, we argue that the upsampling of the relations could also be subject to loss of information. In this paper, we propose to discover the task-dependent affinities between the low-resolution and high-resolution guidance images, then map the affinities to the low-resolution target image. We show that the proposed scheme is effective for joint image upsampling. Based on this principle, we propose a deep neural network that learns the affinities based on the convolutional spatial propagation network and applies them to the low-resolution target images for upsampling. Compared to the state-of-the-art joint upsampling networks, our model is more interpretable. We have experimented with the proposed upsampling method on a variety of low-level vision and image processing tasks, including depth map upsampling, image smoothing upsampling, style transfer upsampling, and image dehazing upsampling. The results indicate the superiority of the proposed joint upsampling method over the state-of-the-art methods. Furthermore, the proposed method is highly efficient, rendering real time upsampling into 720P color images on a modern GPU. Therefore, it is practical for real usage. Our code and trained models are available at https://github.com/LiQingCode/JUAL. [Display omitted] •We formulate image smoothing as a downsampling-and-upsampling process.•We show that a mild smoothing renders accurate restoration of the target image.•We propose a CNN to learn the affinities for joint image upsampling.•Extensive experimental results validate the superiority of the proposed method.