Calibration and low-level data fusion algorithms for a parallel 2D/3D-camera

• Published in: Information fusion Vol. 12; no. 1; pp. 37 - 47
• Main Authors: Hanning, T., Lasaruk, A., Tatschke, T.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 2011
• Subjects: Calibration; Fusion of range and image data; Nearest neighbor; Nearest neighbor by segmentation; Nearest neighbor; Fusion of range and image data; Calibration; Nearest neighbor by segmentation
• ISSN: 1566-2535; 1872-6305
• DOI: 10.1016/j.inffus.2010.01.006

In this article we propose a calibration algorithm and three low-level data fusion algorithms for a parallel 2D/3D-camera system. A parallel 2D/3D-camera is a hardware setup of a range camera and a high-resolution gray-value camera spatially related to each other by a fixed translation. The proposed calibration algorithm utilizes the fact that for known calibration patterns the range reconstruction accuracy of the gray-value camera is significantly higher than that of the range sensor. Using the calibrated 2D/3D-camera we identify the range pixels within the gray-value image for each pair of acquired camera images. We present three low-level data fusion approaches assigning range information to each gray-value pixel based on different neighborhood relations to the identified range pixels: one-nearest neighbor, nearest neighbors in the surrounding Delaunay-triangle, and nearest neighbors constrained by a gray-value image segmentation. We demonstrate the applicability, efficiency and accuracy of our calibration and fusion algorithms on real and synthetic data. Our real experiments are performed on a 2D/3D-camera comprising a Siemens 64 × 8-pixel time-of-flight range camera developed within the European project PReVENT (UseRCams) and a common gray-value camera.