Three-dimensional registration and shape reconstruction from depth data without matching: A PDE approach

Abstract

The widespread availability of depth sensors like the Kinect camera makes it easy to gather three-dimensional (3D) data. However, accurately and efficiently merging large datasets collected from different views is still a core problem in computer vision. This question is particularly challenging if the relative positions of the views are not known, if there are few or no overlapping points, or if there are multiple objects. Here, we develop a method to reconstruct the 3D shapes of objects from depth data taken from different views whose relative positions are not known. Our method does not assume that common points in the views exist nor that the number of objects is known a priori. To reconstruct the shapes, we use partial differential equations (PDE) to compute upper and lower bounds for distance functions, which are solutions of the Eikonal equation constrained by the depth data. To combine various views, we minimize a function that measures the compatibility of relative positions. As we illustrate in several examples, we can reconstruct complex objects, even in the case where multiple views do not overlap, and, therefore, do not have points in common. We present several simulations to illustrate our method including multiple objects, non-convex objects, and complex shapes. Moreover, we present an application of our PDE approach to object classification from depth data. D. Gomes was partially supported by baseline and start-up funds, from King Abdullah University of Science and Technology (KAUST). J. Saúde was partially supported by by the Portuguese Foundation for Science and Technology through the Carnegie Mellon Portugal Program under the Grant SFRH/BD/52162/2013.