Multimodal Semantic Fusion for Vehicle Driving Based on Residual Connections and Attention Mechanism

Cameras and 3DLidar are indispensable sensors in vehicle driving, in which cameras can provide rich color and texture information, and 3DLidar can capture environmental information with higher precision and longer distances, especially in depth measurement. Sensors can complement each other in the detection of the environment, so how to efficiently fuse the two modal information has become a research hotspot. In this paper, a module based on residual network and attention mechanism is designed to fuse the features of the dual-stream network, enhance the deep features through the direct connection, and change the feature combination form to achieve a better road environment semantic segmentation effect.