On Combining a Semi-Calibrated Stereo Camera and Massive Parallelism for Fast Plane Extraction

Abstract

We present a novel methodology that combines stereo vision and parallel processing, based on GPU and the use of binary descriptors, for fast plane extraction. Typical stereo algorithms require an image rectification stage that has to run in a frame-to-frame basis, increasing the computational burden and with the possibility of compromising high frame rate operation. Hence, we propose to use a semi-calibrated stereo approach, meaning that only calibration of extrinsic parameters of the stereo rig is carried out, thus avoiding a rectification process of the frames captured by the stereo camera. For the latter, we rely on feature matching of salient points detected on the stereo images, from which image correspondences are obtained. These correspondences are triangulated in order to generate a point cloud that is passed to a plane fitting module. As feature matching is a heavy task, we present a novel GPU architecture to accelerate such process, thus achieving real-time performance of up to 50 fps for the whole process. To demonstrate our approach, we also present an augmented reality application that exploits the planes extracted with our proposed methodology.