Simulation of Self-driving System by implementing Digital Twin with GTA5

Abstract

Computer simulation based on digital twin is an essential process when designing self-driving cars. However, designing a simulation program that is exactly equivalent to real phenomena can be arduous and cost ineffective because many things have to be implemented. In this paper, we propose a method using the online game ‘GTA5’ as a groundwork for autonomous vehicle simulation. As ‘GTA5’ has a variety of well-implemented objects, people, and roads, it can be considered a suitable tool for simulation. By using OpenCV to capture the GTA5 game screen and analyzing images with YOLO and TensorFlow [1] based on Python, we can build quite an accurate object recognition system. This can lead to the writing of algorithms for object avoidance and lane recognition. Once these algorithms have been completed, vehicles in GTA5 can be controlled through codes composed of the basic functions of autonomous driving, such as collision avoidance and lane-departure prevention.