Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System

• Published in: IEEE access Vol. 6; pp. 52058 - 52072
• Main Authors: Jang, Hanbyol, Bang, Kihun, Jang, Jinseong, Hwang, Dosik
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Cameras; Color; Color difference; deep neural network; Digital imaging; Displays; Dynamic range; high dynamic range; human visual system; Image color analysis; Image resolution; inverse tone mapping operator; Machine learning; Mapping; Neural networks; Visual perception; Visual systems
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2018.2870295

Conventional digital displays show images with much less dynamic range than that of human visual perception. High-dynamic range (HDR) displays are being developed for viewing images with higher dynamic range than that of conventional low-dynamic range (LDR) images. However, to view existing LDR images on HDR displays, an inverse tone mapping operator (ITMO), which is a process to extend the dynamic range of LDR images, is required. In this paper, we propose an adaptive ITMO by effectively learning the differences between LDR and HDR images using a sequential learning process: dynamic range learning followed by color difference learning. Our proposed method enables visualization of colors similar to real-world colors better than conventional ITMOs by learning color differences based on the human visual system properties. For the objective comparison, seven different evaluation metrics optimized for HDR image evaluation were used. Our method resulted in 10%-25% improved HDR-visual difference predictor-2.2 values over those of other ITMOs. Other metrics also demonstrated the superior performance of our method. For the subjective comparison, eight human observers evaluated the estimated HDR images in terms of color appearance and overall preferences. Our method received an average 4.7 out of 5 score, whereas other ITMOs received below 3.5 scores in the evaluation of color appearance. The objective and subjective evaluations' results showed that our proposed method outperformed the conventional ITMOs in estimating the dynamic range and color appearance of ground-truth HDR images.