Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline

• Published in: Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 1648 - 1657
• Main Authors: Liu, Yu-Lun, Lai, Wei-Sheng, Chen, Yu-Sheng, Kao, Yi-Lung, Yang, Ming-Hsuan, Chuang, Yung-Yu, Huang, Jia-Bin
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.01.2020
• Subjects: Cameras; Dynamic range; Image reconstruction; Image sensors; Pipelines; Quantization (signal); Training
• ISSN: 1063-6919
• DOI: 10.1109/CVPR42600.2020.00172

Recovering a high dynamic range (HDR) image from a single low dynamic range (LDR) input image is challenging due to missing details in under-/over-exposed regions caused by quantization and saturation of camera sensors. In contrast to existing learning-based methods, our core idea is to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDR-to-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We then propose to learn three specialized CNNs to reverse these steps. By decomposing the problem into specific sub-tasks, we impose effective physical constraints to facilitate the training of individual sub-networks. Finally, we jointly fine-tune the entire model end-to-end to reduce error accumulation. With extensive quantitative and qualitative experiments on diverse image datasets, we demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.