Deep Q-learning for Control: Technique and Implementation Considerations on a Physical System: Active Automotive Rear Spoiler Case

Abstract

Deep Q-learning is the combination of artificial neural networks advantages (ANNs) with Q-learning. ANNs have expanded the possibilities on a variety of algorithms by enhancing their capabilities and surpassing their limitations. This is the case of reinforcement learning. Nowadays, Deep Q-learning is used in a variety of applications in different fields, including the development of intelligent algorithms to control physical systems. Deep Q-learning has demonstrated the possibility of achieving effective results by solving specific tasks that are highly complex to model through classical approaches. An important drawback is that these models require an elaborated implementation process, and several design decisions must be taken in order to achieve reliable results. Often, developers might find the design process mostly experimental rather than ruled-based. Addressing this problem, the present work describes in detail the implementation process of Deep Q-learning to control a physical system, proposes considerations and analysis parameters for each of the main steps. Demonstrated in the development of the “Active automotive rear spoiler”, the results present a methodology that successfully guides towards a proper implementation of Deep Q-learning. The knowledge of this paper should not be taken as a recipe, but rather as an evaluation reference to equip the reinforcement learning developers with tools for the development of projects.