LiDAR-Camera joint obstacle detection algorithm for railway track area

The application of multi-sensor systems in railway security holds significant research potential. We propose a joint decision-making strategy for obstacle detection (OD) in track areas that integrates LiDAR and camera sensors. LiDAR can accurately detect the geometric information of obstacles in the region without being affected by lighting conditions. To accurately assess the danger level of obstacles, we introduce a camera sensor to classify obstacles based on known locations, thereby enhancing the accuracy of detecting potentially hazardous obstacles. We first use LiDAR to obtain the point cloud data for the detection area. A point cloud algorithm designed explicitly for static obstacle recognition is applied to extract obstacle point cloud information. Using an external transformation relationship, we capture the corresponding obstacle images, fusing them with the point cloud data to serve as input images for a neural network. Subsequently, we introduce MS-YOLO-DLKA, an image OD network that combines a multi-scale feature extraction module (MS-Block) and a large convolution kernel module (D-LKA) based on YOLOv5. On our railway track obstacle dataset, the network achieved an accuracy of 85 %, a recall rate of 95.8 %, and a mAP value of 0.91, outperforming several SOTA (state-of-the-art) networks regarding comprehensive application performance. In test scenarios, our equipment has achieved OD within a range of 50 m for obstacles as small as 20 cm × 20 cm × 20 cm, providing a new railway security and monitoring solution.