Motion-Based Camera-LiDAR Online Calibration With 3D Camera Ground

This paper addresses a novel camera-LiDAR online targetless calibration method. As the installation of heterogeneous sensors in autonomous vehicles is increasing, the importance of sensor calibration is also growing. An automatic sensor calibration function is essential for safe autonomous driving, responding to the sensor's geometric change. In this paper, background knowledge of sensor calibration is explained, and related papers are referenced. Depth estimation and semantic segmentation models are learned to detect significant regions from the camera using open and custom datasets. This paper describes the online calibration procedures based on motion-based calibration between the camera and LiDAR. Motion-based calibration faces a challenge for lack of roll and pitch variation, and our method overcomes this challenge by estimating a camera 3D ground plane from semantic segmentation and depth estimation and simultaneously detecting a 3D LiDAR ground plane. LiDAR-to-camera roll, pitch, and z values are then extracted using the camera and LiDAR ground plane, and the motion-based calibration problem is solved and optimized using those values as constraints. Experimental results conducted on urban roads and the proving ground C-track using our autonomous vehicles showed that the proposed method is quantitatively and qualitatively improving the existing method.