Teaching Vehicles to Steer Themselves with Deep Learning

Traditional approaches for steering a vehicle using machine vision require large amounts of robust hand-crafted software which is both time consuming and expensive. The presented method uses a deep neural network to teach cars to steer themselves without any additional software. We created a labeled dataset for the ACTor (Autonomous Campus TranspORt) electric vehicle by pairing real world images taken during a drive with the associated steering wheel angle. We trained a model end to end using modern deep learning techniques including convolutional neural networks and transfer learning to automatically detect relevant features in the input and provide a predicted output. This means that no traditional hand engineered algorithm features were required for this implementation. We currently use an pretrained inception network on the ImageNet dataset to leverage the high level features learned from ImageNet to the steering problem through transfer learning. We removed the top portion of the network and replaced it with a linear regression node to provide the output. The model is trained end to end using backpropagation. The trained model is integrated with vehicle software on ROS (Robot Operating System) to read image data and provide a corresponding steering angle in real time. The current model achieves 15.2 degree error on average. As development continues the model may replace the current lane centering software and will be used for IGVC Self-Drive competition and campus transportation.