Dense-depth-net: a spatial-temporal approach on depth completion task

Abstract

Depth completion is essential functionality in the perception system of an autonomous vehicle. With various convolution neural networks (CNN), scene geometric representation has been studied extensively under supervised learning or self-supervised learning. This paper utilizes recurrent neural networks (RNNs) to investigate temporal information from camera video sequences, which can help mitigate the mismatch between two consecutive data frames. Our paper proposed an architecture consisting of two sequence processing: the spatial exploitation stage built from a two-branches network and the temporal exploitation stage, a novel convolutional LSTM (ConvLSTM). Furthermore, we take the ability of long short-term memory (LSTM)-based RNNs to estimate a one-step depth map as an additional role of the representations of objects not only in a data frame but also in its temporal neighborhood. Moreover, the proposed ConvLSTM network demonstrated to have the option to make depth forecasts for future or occluded parts of an image frame. We evaluate the performance of the proposed architecture on the KITTI dataset and achieve the result proving to improve accuracy via a supervised-learning.