Hybrid depth-event pose estimation for online dense reconstruction in challenging conditions

In this paper, we present a novel dense SLAM system based on depth-event fusion, aiming to address the challenge of online dense reconstruction in challenging environments. To achieve robust camera tracking, we devise a hybrid depth-event pose estimation framework based on random optimization, which estimates all states jointly. Notably, we introduce an innovative 3D-2D edge alignment method based on particle swarm optimization, specifically tailored for event cameras, to tackle the highly non-linear pose estimation problem. Furthermore, we implement a dynamic update mechanism for both geometric and intensity edges of the 3D reconstruction, enabling efficient and accurate management of edge information. Our method represents the first depth-event dense SLAM system employing a random optimization paradigm, achieving robust performance even with high-speed camera motion, specifically linear velocities exceeding 1 m/s and/or angular velocities exceeding 2 rad/s. The system achieves accurate and globally consistent dense mapping with a maximum spatial resolution of 2 mm, while maintaining real-time performance at approximately 30 FPS for simultaneous localization and 3D reconstruction. Through extensive evaluations on synthetic and real-world datasets, particularly on our newly constructed DEveSet dataset, we demonstrate the superior performance of our proposed method compared to state-of-the-art techniques such as InfiniTAM, ROSEFusion, and DEVO. Contact us for access to the DEveSet download link.