Guided Depth Inpainting in ToF Image Sensing Based on Near Infrared Information

Abstract

Accurate depth estimation is crucial in various computer vision applications, such as robotics, augmented reality, or autonomous driving. Despite the common use of Time-of-Flight (ToF) sensing systems, they still face challenges such as invalid pixels and missing depth values, particularly with low light reflectance, distant objects, or light-saturated conditions. Cameras using indirect ToF technology provide depth maps along with active infrared brightness images, which can offer a potential guide for depth restoration in fusion approaches. This study proposes a method for depth completion by combining depth and active infrared images in ToF systems. The approach is based on a belief propagation strategy to extend valid nearby information in missing depth regions, using the infrared gradient for depth consistency. Emphasis is placed on considering object edges, especially those coinciding with depth discontinuities, to approximate missing values. Empirical results demonstrate the efficiency and simplicity of the proposed algorithm, showcasing superior outcomes compared to other reference guided depth inpainting methods.