LineDL: Processing Images Line-by-Line With Deep Learning

• Published in: IEEE transactions on image processing Vol. 32; pp. 3150 - 3162
• Main Authors: Huang, Yujie, Chen, Wenshu, Peng, Liyuan, Liu, Yuhao, Wang, Mingyu, Zhang, Xiao-Ping, Zeng, Xiaoyang
• Format: Journal Article
• Language: English
• Published: United States IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Computational modeling; Deep learning; denoising; Electronic devices; Image coding; Image compression; Image processing; Image quality; line-by-line; Noise reduction; Radio frequency; Superresolution; Task analysis
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2023.3277394

Although deep learning-based (DL-based) image processing algorithms have achieved superior performance, they are still difficult to apply on mobile devices (e.g., smartphones and cameras) due to the following reasons: 1) the high memory demand and 2) large model size. To adapt DL-based methods to mobile devices, motivated by the characteristics of image signal processors (ISPs), we propose a novel algorithm named LineDL. In LineDL, the default mode of the whole-image processing is reformulated as a line-by-line mode, eliminating the need to store large amounts of intermediate data for the whole image. An information transmission module (ITM) is designed to extract and convey the interline correlation and integrate the interline features. Furthermore, we develop a model compression method to reduce the model size while maintaining competitive performance; that is, knowledge is redefined, and compression is performed in two directions. We evaluate LineDL on general image processing tasks, including denoising and superresolution. The extensive experimental results demonstrate that LineDL achieves image quality comparable to that of state-of-the-art (SOTA) DL-based algorithms with a much smaller memory demand and competitive model size.