Robust super-resolution in a multiview setup based on refined high-frequency synthesis

• Published in: 2012 IEEE 14th International Workshop on Multimedia Signal Processing pp. 7 - 12
• Main Authors: Richter, Thomas, Seiler, Jurgen, Schnurrer, Wolfgang, Kaup, Andre
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.09.2012
• Subjects: Cameras; Gain; Image analysis; Image resolution; PSNR; Robustness; Visualization
• ISBN: 9781467345705; 1467345709
• DOI: 10.1109/MMSP.2012.6343407

Increasing image sharpness and thus improving the visual quality is an important task in multiview image and video processing. We propose a novel super-resolution approach for multiview images in a mixed-resolution setup that is robust to various depth map distortions. The considered distortion scenarios may be caused by an inaccurate calibration of the depth camera or a limitation of depth range. Our method is based on a refined high-frequency synthesis that relies on a blockwise and depth-dependant low-frequency registration. The refinement step efficiently adapts the high-frequency content from a neighboring high-resolution camera to a low-resolution view and thereby compensates the displacement caused by depth inaccuracies. In case of undistorted depth maps, the results show that our algorithm leads to a PSNR gain of up to 1.33 dB with respect to a comparable unrefined super-resolution approach for a mixed-resolution multiview video plus depth format. Compared to the initial low-resolution view, a PSNR gain of up to 2.61 dB is obtained. In case of distorted depth maps, a PSNR gain of even 4.78 dB is achieved with respect to the reference superresolution algorithm. The PSNR gains get confirmed by the corresponding SSIM values which manifest a similar behaviour. The improvement of visual quality is also convincingly for all considered scenarios.