An Approach for Shape from Surface Normals with Local Discontinuity Detection

Abstract

We present a multi-modal surface reconstruction approach, which utilizes direct surface orientation measurements along with luminance information to obtain high quality 3D reconstructions. The proposed approach models local surface geometry as a set of intersecting natural cubic splines estimated through least squares fitting of our input pixel-wise surface normal measurements. We use this representation to detect discontinuities and segment our scene into disjoint continuous surfaces, which are constructed by an aggregation of connected local surface geometry elements. In order to obtain absolute depth estimates, we introduce the concept of multi-view patch sweeping, where we search for the most photo-consistent patch displacement along a viewing ray. Our approach improves on existing shape from normals methods by enabling absolute depth estimates for scenes with multiple objects. Furthermore, in contrast to existing multi-view stereo methods, we are able to reconstruct textureless regions through the propagation of relative surface orientation measurements. Experiments on synthetic and real data are presented to validate our proposal.