Multi-Modal Attention Guided Real-Time Lane Detection

Multimodal data fusion is becoming a trend for the field of autonomous driving, especially for lane detection. In the process of driving, sensors often encounter problems such as modality imbalance, changing illumination and so on. Therefore, it is worthwhile to study the problems of applying multimodal fusion for lane detection and modality imbalance in the fusion process. In this paper, we propose a novel multimodal model for lane detection, in which attention mechanism is embedded into network to balance multimodal feature fusion and to improve detection capability. In addition, we use multi-frame input and long short-term memory (LSTM) network to solve the shadow interference, vehicles occlusion and mark degradation. At the same time, the network can be applied to the task of lane detection. In order to verify the effect of multimodal application and attention mechanism on fusion, we have designed adequate experiments on processed continuous scene KITTI dataset. The results show that precision increases by about 15% when LiDAR is added compared with RGB only. Besides, attention mechanism obviously improves the performance of multi-modal detection by balancing multi-modal features.