Autonomous Navigation via a Deep Q Network with One-Hot Image Encoding

Abstract

Common autonomous driving techniques employ various combinations of convolutional and deep neural networks to safely and efficiently navigate unique road and traffic conditions. This paper investigates the feasibility of employing a reinforcement learning (RL) model for autonomous navigation using a low dimensional input. While many navigation applications generate each individual state as a function of a frame's raw pixel information, we use a deep Q network (DQN) with reduced input dimensionality to train a mobile robot to continuously remain within a lane around an elliptical track. We accomplish this by using a one-hot encoding scheme that assigns a binary variable to each element in a square array. This value is a function of whether the input frame detects the presence of a lane boundary. Our ultimate goal was to determine the minimum number of training samples required to consistently train the robot to complete one cycle around the track, from multiple starting positions and directions, without crossing a lane boundary. We found that by intelligently balancing exploration and exploitation of its environment, as well as the rewards for staying in the lane, the robot was able to achieve its goal with a small number of samples.