Towards neuroimaging real-time driving using Convolutional Neural Networks

The majority of previous attempts to autonomous driving in the TORCS environment involve the use of precalculated data such as the exact distance to opponents or the actual position of the car with respect to the center of the track. As humans, we drive based on locally available information that is gathered by our senses, mainly sight. This information is not precomputed and it is left to the agent to make sense of it. This work explores and evaluates the development of autonomous steering using visual-only input in real time driving. Convolutional Neural Networks (CNNs) have been proven to excel in categorical image classification, but little has been done on how they could perform in continuous spaces such as deciding steering values. The results presented here show that CNNs are indeed capable of performing well on such spaces and can be trained from example. Various modifications proved to enhance network accuracy and hence driving performance, such as applying edge detection filters to the input image, using a weighted average method to select network outputs and including unusual situations in the training data, making the neurovisual agent much more robust and capable of higher generalization power.