Point-to-Point Car Racing: an Initial Study of Evolution Versus Temporal Difference Learning

Abstract

This paper considers variations on an extremely simple form of car racing, the challenge being to visit as many way-points as possible in a fixed amount of time. The simplicity of the models enables a very thorough evaluation of various learning algorithms and control architectures, and enables other researchers to work on the same models with relative ease. The models are used to compare the performance of various hand-programmed controllers, and neural networks trained using evolution, and using temporal difference learning. Comparisons are also made between state-based and action-based controller architectures. The best controllers were obtained using evolution to learn the weights of state-evaluation neural networks, and these were greatly superior to human drivers