Volumetric calibration and registration of multiple RGBD-sensors into a joint coordinate system

Abstract

We present an integrated approach for the calibration and registration of color and depth (RGBD) sensors into a joint coordinate system. Our application domain is 3D telepresence where users in front of a three-dimensional display need to be captured from all directions. The captured data is used to virtually reconstruct the group of people at a remote location. One key requirement of such applications is that contributions from different color and depth cameras match, as closely as possible, in spatially overlapping or adjacent regions. Our method employs a tracked checker-board to establish a number of correspondences between positions in color and depth camera space and in world space. These correspondences are used to construct a single calibration and registration volume per RGBD sensor which maps raw depth sensor values in a single step into a joint coordinate system and to their associated color values. This approach considerably reduces reconstruction latency by omitting expensive image rectification processes during runtime. Furthermore, our evaluation demonstrates a high measurement accuracy with an average 3D error below 3 mm and an average texture deviation smaller than 0.5 pixels for a space of about 1.5 m × 1.8 m × 1.5 m.