Accurate and automatic spatiotemporal calibration for multi-modal sensor system based on continuous-time optimization

Abstract

Current intelligent robotic applications, such as unmanned aerial vehicles (UAV) and autonomous driving, generally rely on multi-modal sensor fusion to continuously strive towards higher levels of autonomy. To achieve this goal, accurate and consistent inter-sensor spatiotemporal relationship is a fundamental prerequisite for fusing heterogeneous sensor information. Nevertheless, current calibration frameworks typically necessitate specialized tools or additional infrastructures, rendering them labor-intensive and only applicable to certain sensor combinations. To address this issue, we propose an accurate and easy-to-use spatiotemporal calibration framework tailored to current primary sensors, including inertial measurement unit (IMU), LiDAR, camera and Radar. This calibration framework can be seamlessly extended to other sensors that could independently recover ego-motion or ego-velocity, such as wheel odometry and GPS devices. A rigorous multistage initialization approach is first developed to obtain reasonable initial guesses of spatiotemporal parameters without relying on prior knowledge of environmental information or specialized movements. Leveraging the IMU-centric principle, the spatiotemporal parameters of other sensors relative to IMU can be jointly optimized and refined via continuous-time batch estimation without sharing the overlapping field-of-views (FoVs) among exteroceptive sensors. A comprehensive series of experiments is carried out to quantitatively evaluate the proposed method in both simulation and real-world scenarios. The results demonstrate that the proposed method could achieve comparable calibration accuracy against state-of-the-art target-based calibration methods and outperform targetless calibration methods in terms of consistency and repeatability.