Event-driven stereo vision algorithm based on silicon retina sensors

Abstract

In this paper a new stereo matching concept for event-driven silicon retinae is presented. The main contribution of the proposed approach is the correlation of incoming events. As a novelty, not only the spatial information is used, but also the time of occurrence of the events as a part of the similarity measure. Stereo matching is used in depth generating camera systems for solving the correspondence problem and for 3D reconstruction of the sensed environment. In fact, using conventionally frame-based cameras, this is a time consuming and computationally expensive task, especially for high frame rates and spatial resolutions. An event-based silicon retina delivers events only on illumination changes and completely asynchronous in time. The sensor provides no frames, but a time-continuous data stream of intensity differences and thus inherently reduces the visual information to a minimum. This paper focuses on an event-based stereo matching algorithm implemented in hardware on a field programmable gate array (FPGA) that allows a reliable matching of the sparse input event data. Furthermore, the approach is compared to other standard frame-based and event driven stereo methods. The results show that the achieved depth map outperforms other algorithms in terms of accuracy and the calculation performance of the hardware architecture is in the range or still higher than state-of-the-art computing platforms.