End-to-end control of autonomous vehicles based on deep learning with visual attention

In this paper, we propose an end-to-end controller for self-driving vehicles based on visual attention. Attention strategy is used to weight the high-dimensional feature information extracted by convolutional neural networks (CNNs), and then the vehicle's velocity and steering wheel angle are predicted by different recurrent neural networks (RNNs). The end-to-end controller is trained on Comma.ai dataset and can effectively reduce the mean absolute error (MAE). The result shows that compared with other models, the end-to-end control model based on visual attention can achieve better control effects of vehicle's speed and steering wheel angle.