Action Replication in GTA5 using Posenet Architecture with LSTM Cells

Abstract

Playing video games by doing physical activity in an environment instead of keyboard or game controllers is not new. There are multiple products accessible in the market which are already doing well. But they all rely on some expensive sensors (Motion sensors, accelerometer, radar, infrared, etc.) with a separate processing unit to control games by physical activity in real-time. They perform excellently but cost too much that not everyone can afford. Despite that, they are compatible with only fewer games and users can't modify them to play games that they want. This paper introduces a method to control games by doing physical activities with just a Smartphone or web camera without any separate processing unit or expensive sensors. The product will be the only software that will use a camera to analyze physical activities with the help of some Deep Learning algorithms to control games in real-time. The user will have the ability to tune the system according to the game and the way they want to play. The network was trained on 70% of data and tested on 30% of the data logging 96.01 % accuracy when validated.